We’re entering an era when entertainment will not only be seen or heard, but experienced. In 2017, we’ll have even higher expectations for our next phone, especially when it comes to how we’re amused, inspired, and informed. That’s something Qualcomm Technologies kept in mind when developing our next-generation processor: the Qualcomm Snapdragon 835 is purpose-built for immersion. It is designed to pack in cutting-edge technologies that work together to provide the next-level experience power users crave.
Many of us have heard the terms “AR” and “VR” mentioned. Some of us have even experienced one or the other. But how many of us fully understand what these technologies are and how they’ll impact our lives?
AR, or augmented reality, superimposes content over the real world. As of now, germinal parts of AR are present in apps like Pokémon Go (think about that Squirtle your smartphone is showing right in front of you, for example) and some smartphones like the Google Tango-enabled Lenovo PHAB2 Pro and ASUS ZenFone AR. And even with more advanced products like the recently announced ODG R-8 and R-9 smartglasses — both powered by Snapdragon 835 — AR is still in its infancy. In the coming years, AR will absorb scores of mobile technologies like 3D audio, stunning graphics, and accurate motion tracking (just to name a few) and be placed in one imperceptible device that replaces all others.
Virtual reality (VR), on the other hand, simulates our presence in a real or virtual world and requires full immersion — through visuals, audio, and intuitive actions — to be believable. VR has benefited from aggressive development in recent years, moving beyond the early cumbersome, wired head-mounted displays with controllers. With the Snapdragon 835 processor, each of the necessary components for immersion is at a point where we can experience advanced VR right from our mobile devices. Here’s how:
For VR to be convincing, content needs to be so vibrant and realistic that it’s indistinguishable from what we see in the real world. And for AR to seamlessly overlay objects in our view, its images need to naturally blend with what we see in reality. For that to happen, mobile devices need a processor that delivers extreme pixel quantity and quality.
Snapdragon 835 processors are engineered to provide immersive visuals with a 360-degree spherical field of view of the VR and AR world, in realistic 3D. It supports stereoscopic display, which mimics how we view our real-life surroundings. To provide the illusion of depth, slightly offset images are shown to each eye. And foveated rendering is designed to provide higher pixel resolution where the eye (fovea) is focused and lower resolution everywhere else on the screen, so visuals look sharp and the HMD (Head Mounted Display) device stays cool.
Premium VR and AR also require higher performance and more life-like colors for virtual worlds to look like a second reality. To that end, the Snapdragon 835 is designed to display up to 60X more colors and render graphics 25 percent faster than the Snapdragon 820. That translates to graphics rendered at higher frame rates with true-to-life pixels so that virtual worlds are believable and no awkward lag exists when we turn our heads.
Vibrant, realistic visuals aren’t the only component needed to make AR and VR immersive. Sound must also be true to life. In other words, the audio should mirror our human hearing capabilities in the real world, mimicking the sounds we hear all around us, and even adjusting the sound appropriately as we move in the virtual and augmented world.
The Snapdragon 835, with the integrated Qualcomm Hexagon 682 DSP, is engineered to deliver heightened sounds with the latest Qualcomm Aqstic audio codec and software. It adds height information, in order to play audio from not only at the front, back, left, or right, but also above and below the user. Imagine playing a racing game and suddenly hearing a police siren: you should know exactly where the sound is coming from even as you’re speeding ahead.
Hi-fi audio formats like native DSD, which previously could only be found in audiophile-grade SACDs (Super Audio CD), are also supported, so you now can enjoy some of the most immersive audio quality on the go. The Snapdragon 835’s crystal-clear voice technology, enhanced by noise cancellation and speech recognition, is designed so you can effectively use voice commands at any time and chat with others while interacting in virtual and augmented worlds.
Snapdragon 835 processors also support object-based audio formats like Dolby Atmos, so you can experience accurately mapped, true 3D surround sound. 3D audio, whether object or scene based, uses head-related transfer functions (HRTF) and maps the audio to your headphones, so that your ear hears sounds coming from correct directions, even as you turn your head up, down, right, or left. Hearing a dog howling in the distance, a honking cab speeding past, and even a suspicious rustle in the trees — these are details that can make or break VR and AR, and Snapdragon 835 processors help bring them to life.
A 3D Audio SDK will be released to assist developers and OEMs in their creation of audibly immersive experiences with an anticipated release in February 2017. This developers kit will include support for object-based and ambisonic audio, reverb effects, and headphone reproduction through HRTF.
What separates a movie, or even a 360-degree video, from a more realistic experience? Interactions that are so intuitive that it becomes second nature. In VR and AR, 6-DOF (Degrees Of Freedom) motion tracking allows you not only to look up, down, left, and right, but also to walk forward and backward in your changing VR environment. The Hexagon DSP runs a visual-inertial odometry (VIO) algorithm which can figure out where you are in the virtual or real world and provide a 6-DOF head pose. VIO is designed to provide precise on-device motion tracking by fusing information from a camera and motion sensors, and estimating relative position and orientation of a moving device.
As you move through the virtual or augmented world, the motion-to-photon latency — the time it takes for your movement to be fully reflected on the screen — needs to be short, so that the experience is imperceptible from reality. Even more importantly, perhaps, minimizing motion-to-photon latency helps cut down on motion sickness. The Snapdragon 835 processors is engineered to reduce motion-to-photon latency by 20% over the Snapdragon 820 and 821 – all the way to 15 milliseconds. The processor also supports Q-Sync, an adaptive display technology that refreshes at the same frame rate that the GPU renders, so the display is smooth and jank-free.
The Snapdragon 835 is designed to provide the forward-thinking technologies needed to make our virtual worlds real, combining superior visuals and sound with truly intuitive interaction. This year, significant progress will be made in VR, and AR too. Augmented reality is the natural next step for mobile devices and will continue to develop in parallel. It might be challenging to develop and strike the balance between immersion, cognition, and connectivity, but we’re up to the task.
We’ll be right there in the thick of it, not just with our innovative processor, but with the release of the Qualcomm Snapdragon VR SDK, our VR reference platform, and support for Google Daydream. And with the emergence of 5G, both VR and AR technologies will soon have the connectivity foundation to build upon and thrive on.
Check back on our Snapdragon blog for in-depth tech explainers on each of the Snapdragon 835 processor’s other key components — capture, connectivity, security, and machine learning. You can catch up on our battery deep dive now.
And sign up for our Snapdragon Insider newsletter to be among the first to get the latest Snapdragon news.