Technology continues to advance at a rapid pace, and yet PC human-machine interfaces (HMIs) haven't changed much since the addition of the mouse. It's difficult to imagine a computer without a keyboard or mouse, and perhaps that's why various attempts to replace them have been unsuccessful. But according to former Intel-Israel President Mooly Eden, the way we currently interact with technology is unnatural, and technology companies intend to do something about it.
In fact, HMI has advanced considerably over the past decade, and while it doesn't seem that the keyboard and mouse are going anywhere, users seem to increasingly depend on other HMI technology for their various computing needs. From voice recognition and haptic interfaces to augmented reality and gesture sensing, HMIs are becoming increasingly "human."
Let's have a look at some of the most popular and innovative HMI technologies available today, and some that aren't quite ready for market yet, to assess how far human computer interaction has come, and where it's going next.
Dragon NaturallySpeaking
Nuance's Dragon NaturallySpeaking products have been around for a while and have gradually increased in popularity across all age groups and geographical locations. The voice recognition software transcribes user speech in seconds, and allows users to execute computer commands using only their voice. Commands include transcription, editing text, sending emails and instant messages, and even searching the web (much like Amazon's Echo).
The real secret behind the success of the Dragon software is its ability to make interacting with the computer more natural. While the Echo is a rather nifty gadget, Dragon poises itself as functional software with true value in today's market. Whether you're a parent with too much on your hands, or a corporate executive, the program allows you to execute various tasks using just your voice.
The keyboard and mouse are excellent input devices in that we can access our computer intimately, quietly. Voice recognition HMIs are meant for busy days and fast-paced environments. Together, the HMIs work quite well.
Geomagic haptic devices and software
Another technology on the horizon is haptic computer interface. Haptic interfaces enable a sense of touch otherwise impossible when interacting with a screen. While haptic has only really become commonplace with video games and emerging technology for the disabled, one company is successfully bringing haptic to the masses.
Geomagic creates haptic devices and open-source software to add the third dimension to design. The company claims its devices allow developers to "feel" their designs, as if they were creating with clay instead of code. The software can do anything from jewelry and medical 3D simulation to 3D printing, architecture, and 3D game design.
The magic occurs in being able feel the creations to ensure accuracy before production. Better yet, clients can feel the designs too, adding a unique and competitive advantage to a creative entrepreneur or firm.
And while the design field is niche compared to the vast array of digital consumers, the technology has potential. Imagine having the ability to feel the material of a product digitally, prior to purchase. Online sales of material goods, such as clothing and furniture, could be greatly enhanced, and other sectors, such as movies and gaming, would never be the same.
Microsoft HoloLens
There's been a lot of buzz around Microsoft's forthcoming HoloLens. The augmented reality (AR) system promises to transform your world by turning any surface into a computer screen. The AR device seems as if it might be one of the most successful HMI launches of the decade, but its potential for success really comes down to what it does for users.
The innovation behind Microsoft's HoloLens is based on eliminating the computer altogether. The days of sitting at a desk and interfacing with a machine with our hands is over. The Hololens combines voice and gesture recognition with impeccable augmented reality display to give users the first glimpse of what the future might look like.
The difficulty in getting rid of the keyboard and mouse is it makes interacting with a computer unnatural. Keyboards and mice are unnatural in and of themselves, but if you have been using these HMIs since childhood, they are natural to you. Hololens takes it a step further by making technology a part of you that you can take with you anywhere you go, and Microsoft isn't the only company pushing this technology.
CastAR, and Magic Leap are other companies on the forefront of making AR commonplace in today's market. The reason is the same. Augmented reality offers a unique opportunity to allow consumers to interact with computers in a new, yet natural, way. If there is to be a forthcoming HMI replacement for the keyboard and mouse, it lies in creating a more reliable computing experience on the whole.
Kinnect for Windows
The Microsoft Kinnect isn't just for the Xbox anymore. Microsoft is developing the gesture sensing technology for computers, too. While the platform isn't likely to replace any HMIs, it does make the Kinnect more accessible to more people—namely, the elderly.
Since Microsoft made mention of Kinnect for Windows, an unexpected market has emerged. There is a very large population of elderly people who need functional exercise to remain healthy, or as a form of therapy. With the help of Memore and the Kinnect, these persons are now getting the help they need at home, by following virtual cues that assess movement. As technology advances, it could even provide updates to physicians to ensure patients are performing assigned exercises correctly and keeping to a schedule.
The Kinnect for Windows doesn't only provide value to the elderly. It's nice to think we may watch videos, listen to music, and scroll news feeds with gestures alone. The technology won't replace the keyboard or mouse for things like typing and web browsing yet, but it is a nice addition to the user experience.
Leap Motion
The Leap Motion controller is a gesture sensing and virtual reality platform that relies on infrared technology to enable users to access their computers using only their hands. Users can pinch, pull, and swipe their way across computer screens. As you can see in the image from Leap Motion, this HMI isn't meant to replace the keyboard and mouse, but complement the devices.
Leap Motion prides itself on its success with gesture sensing. The infrared cameras housed in the small device create a 150 degree field of "view" that's nearly eight feet across. Even if you're a good distance away from your computer, say DJing, showing off a design to clients, or playing a first-person shooter, your computer can recognize your gestures and transform them into computer commands.
The way in which the Leap Motion controller adds useful functionality with relative ease is part of its success. The accuracy of the technology is being duplicated in other industries as well, and we may well see it revolutionize the car industry, professional sector, and advanced learning.
Multi-touch screens and trackpads
If there are recent HMIs that are giving the classic keyboard and mouse serious competition, it's trackpads and multi-touch surfaces. Apple is so confident that it will enhance the user experience that its newest mouse includes a multi-touch surface, as does its popular new Magic Trackpad.
Apple's Magic collection of HMIs all feature wireless, rechargeable, and multi-touch surfaces to give users more power and control over their computing, with smaller and more natural gestures.
Multi-touch screens are also becoming increasingly common. Although Microsoft's Windows 8 computer/tablet didn't do as well as the company hoped, it is still a clue as to where the market is headed. Consumers want to be able to access technology and information more naturally, and with less effort.
Brain-computer interface
Brain-computer interface (BCI) is one of the most interesting HMI advancements in recent times, and getting a lot of attention. The technologies typically rely on EEG monitoring to transform thoughts into executable computer commands, and the sensors can be invasive or non-invasive. The technology shows much promise for paraplegic patients and other disabled persons, but it has larger implications, too. The list of BCI devices recently developed is enormous, from mind-controlled cars, robotics, games, and of course, computing.
BrainGate, based at Brown University, is working on a line of invasive BCIs that, in a way, act as the central nervous system. The sensors are projected to allow paraplegics to override disabilities and execute literally any thought into a real action, whether that means controlling a robotic arm or browsing the web. It's hard to say when it will be market ready, but if successful, it will literally change the way we think about computer interaction.
Artificial intelligence
If HMIs aren't changing in the way we want, maybe AI has been the solution all along. The difficulty in trying to influence HMIs is that everyone is different. A child and a business professional will interact with computers differently; as will a creative freelancer and a college student. HMIs are not one-size-fits-all, and any innovation that is to go head to head with the keyboard and mouse will have to make up for this.
AI offers a unique solution, in that machines may learn the habits of the user, and adapt accordingly. Nuance's Dragon programs offer this functionality, as do some mobile interfaces. Imagine if computers also adapted to your habits. If you prefer gesture motion or voice recognition, it could enhance the features for that interaction and meet your needs individually
In one example, a user reprogrammed his phone to input and output Star Trek commands. Makers create similar hacks everyday. If these kinds of changes can be mass produced, we'd really be onto something in HMI technology.
Putting it all together
The death knell hasn't quite sounded for keyboard and mouse yet; in fact, I suspect they'll remain supreme, but the interface horizon will get more and more crowded with newer innovations. The progression of HMIs promise users a cooler experience than ever imagined, but these newer gadgets are not without their own problems.
Emerging HMIs lack the intimacy familiar solutions provide. No one wants to read their news feed on a quiet Saturday morning using a clunky headset. And most people wouldn't want to always have to verbally communicate to their computer to engage with it. These smaller, everyday computer interactions must be considered and replaced with better options if we're to truly see a shift in human-computer interaction.
As technology continues to advance, we'll likely see a disappearance of the computer as we know it, and with it will go its endless array of accessories. Until then, we'll enjoy our keyboards and mice.