The future will be augmented
Microsoft's upcoming product, the HoloLens, is thought by many to be the true future of personal computing. Not only does it augment reality (a feat formerly reserved for the likes of hallucinogens and sweat lodges), but it supports voice commands and gesture recognition as well. This combination of features will make any developer weak in the knees, and early previews from the Holographic Academy Microsoft hosted in April were overwhelmingly positive. By showcasing Unity 5 and Visual Studio as the primary development environments, Microsoft is intent on catering to the needs of the indie crowd. Not only does the app creation experience seem surprisingly elegant, but the included support of the Unity 5 game engine will allow new developers to take an almost drag-and-drop approach to bringing 3D models to life.
Despite this intriguing peek into HoloLens development, it is important to frame our expectations for the device within the capabilities of the hardware. While virtual reality headsets like the Oculus Rift have the luxury of being tethered to powerful gaming rigs, the HoloLens is designed to be completely self-contained. Even if Microsoft were to somehow fit a GTX 970 (recommended GPU for powering the Rift) inside the headset, the human brain does not make for an ideal passive cooling solution. Microsoft hasn't released an official specs list for the headset just yet, but the company claimed it will have significantly more "computing power" than the average laptop. While that could mean the CPU and GPU performance will compete with high-end laptops, it is more likely a nod to the inclusion of the custom holographic processing unit (HPU). This chip is responsible for converting the constant flood of sensor data into a simplified stream of spatial information. In doing so, the HPU should not only ease the burden on the CPU and GPU, but Microsoft claims you won't need a PhD in physics to make sense of the chip's output.
Keeping in mind the importance of factors like heat dissipation, weight, and battery life, it wouldn't be surprising to see a chip similar to the Intel Core M filling the CPU role within the headset. Combined with the aforementioned HPU and next-gen Intel HD graphics, Microsoft's computational power claim would still technically hold true. After all, in terms of raw computing power, the average laptop doesn't have (or need) a holographic processing unit. There are laptops out there that already show that Intel's low-power components can perform competently on passive cooling, and the form factor of the headset will demand parts that run cool and sip power. While this configuration might not be capable of pushing enough pixels for modern games, when used within the HoloLens it probably won't have to. Unlike a desktop gaming experience that has to render an entire virtual world, the novelty of the HoloLens lies within its ability to render a select number of virtual objects within the real world. Simply put, developers will likely forego modeling detailed environments, and instead just find ways to make use of real surroundings.
This bring us to a particularly impressive depth sensor used within the HoloLens. With the release of the Kinect 2, Microsoft managed to engineer a time-of-flight sensor that is capable of mapping an entire scene with a single pulse of light. The HoloLens will reportedly be using the same technology, but with a wider field of view and lower power consumption. Because we've had our hands on the Kinect 2 development kit since last July, we can make a few fairly safe assumptions about the HoloLens. In terms of limitations, the Kinect 2 sensor had a depth range of about 15 feet [PDF] and though it performed well in most lighting conditions, it did sometimes struggle with dark clothing (objects of low reflectivity). Even if Microsoft managed to bump up the resolution of the sensor in the HoloLens, we should expect there to be similar restrictions in our augmented world. Still, real-time spatial mapping is no small feat, and this is an area that Microsoft is likely far ahead of its AR competitors.
Technical details like these will ultimately determine what kind of augmented reality experiences developers will be able to create in 2016. While most VR headsets are focused on revolutionizing 3D gaming, the HoloLens hardware seems better fit to revolutionize basic personal computing. Even Microsoft is touting the headset as a business/enterprise solution that will neatly fit into its own mobile-centric universal app platform. As we learn more about the HoloLens, it becomes clear that Microsoft isn't attempting to compete with VR, at least not yet. What becomes less clear is how developers will choose to leverage the hardware. What should we expect to see in the first wave of augmented reality applications? Will it be enough to push the technology forward? We can only speculate at this point, which is why we've come up with the following list of augmented reality experiences we would realistically expect to see following the release of the HoloLens.
The Facebook Wall – Social media will always have a strong influence on which tech reigns supreme, and augmented reality should have no issue enhancing the experience. In other words, your Facebook wall could begin to take on a very literal meaning.
Help me, Obi-Wan Kenobi – Though we still don't have the droids we're looking for, we are about to get a legitimate form of holographic chat. The depth sensor inside the Kinect 2 already made capturing holographic video possible, and with the HoloLens we finally have a way to project it onto our coffee tables.
Personal HUD – Have you ever wanted to feel like Master Chief as you strut the halls of your apartment? The HoloLens will allow you to pin all sorts of information to your peripheral vision, no matter how useless or distracting.
Tabletop Gaming – Blizzard has reported that over 30 million people have played Hearthstone, their massively popular virtual card game. So making the jump to a platform that supports actual tabletops seems like a no-brainer here.
2.5D Gaming – It's doubtful that the HoloLens will have the chops necessary to play games like the latest Call of Duty, but that doesn't mean it can't augment your current 2D gaming experience. Once the headset identifies the surface of your television, it could communicate with your console to bridge the gap in very interesting ways.
Try Before You Buy – There is nothing more disheartening then spending an entire Saturday building an Ikea dresser only to find out it's too large. The HoloLens could make holographic shopping a reality, and allow you to preview an endless amount of holographic décor in the process.
Relocate – Feeling claustrophobic in your tiny, windowless studio apartment? Let the HoloLens open it up a bit by pinning a few holographic windows to your walls. It won't be long before popular destinations around the world become available, and Microsoft has already demonstrated the ability to peer into your own Minecraft realm.
Get Smart – An obvious adaptation for augmented reality, education might stand to benefit the most from the HoloLens. Whether it's bringing a textbook to life or performing holographic dissections, it's clear that the younger generations will have it better than we did.
