OnQ Blog

Guest blog: Leap Motion and Qualcomm Technologies - A template for the future of mobile VR

Mar 23, 2017

Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.

Martin Schubert is a VR Designer, Developer and Evangelist at Leap Motion. With a background in architecture and spatial studies, Martin designs and develops Leap Motion enabled VR experiences such as Weightless and Geometric which let users reach out and interact with virtual worlds using their bare hands. Read on to get Martin’s thoughts on the future of Mobile VR.

Virtual and augmented reality are the next major stage, radically advancing our concept of what using a computer means and blurring the lines between the physical and digital worlds. However, adoption of these technologies has been slow as desktop VR systems are expensive and difficult to set up, while mobile VR has a comparatively limited feature set.

Until now, the quality of mobile VR experiences has been lacking due to two big limitations: they only have rotational tracking (so you can only look around, not move around) and input is limited to gaze, touchpad, or more recently 3 degrees-of-freedom (3DoF) remotes. The lack of positional tracking for headsets and inputs is a serious problem for immersion and interaction alike. It can feel like being trapped in the center of a spherical movie, unable to move freely to explore these new 3D worlds or directly interact with virtual objects.

Now in collaboration with Leap Motion — and using the new Leap Motion hand tracking technology built specifically for mobile VR — Qualcomm’s Mobile VR Platform based on the Snapdragon™ 835 VR HMD reference design seeks to bridge the gap between the immersive power of desktop and the convenience and portability of mobile VR. With the second generation of mobile VR just around the corner, it provides a template for the future thanks to two major advances.

The first is inside-out positional tracking, which allows users to lean, stand up, and walk around without the need for external cameras or sensors. Already a standard on desktop VR, this 6 DoF movement is crucial for creating the sense of depth that compelling content needs to be experienced in full 3D.

People also want to directly interact with these virtual worlds, which is where the second advance comes in. To make these experiences possible, we developed the Leap Motion Mobile VR Platform, designed to be embedded directly in the upcoming generation of headsets.

The Snapdragon 835 VR HMD dev kit features a 180x180o field of view, with range beyond arm’s length. We track each hand as 26 individual data points with position and orientation at 100 frames per second. This total hand presence unlocks a wide range of experiences where we can grab, punch, throw, and manipulate virtual objects and interfaces.

The combination of these technologies allows developers to create truly compelling interactive room-scale (and beyond!) VR experiences that users can reach out and touch. Our showcase demo Blocks, running on the Snapdragon 835 HMD, is a glimpse at the possibilities available to VR developers.

You can create a block out of nothing by pinching your fingers together. Grab that block with your bare hand. Toss it a few yards. Totally untethered from a PC, walking around the world of Blocks has never felt more magical. Combining 6DoF tracking, natural hand input, and the Snapdragon 835 HMD dev kit has unlocked interactions and game mechanics previously only seen on desktop VR systems. From throwing balls at targets in zero gravity to expressing yourself in massive multiplayer environments, hands greatly expand the possibilities of mobile VR interactions.

The potential for mobile multi-user experiences is immense. Imagine a team of onsite construction professionals who want to show their client an interactive version of the finished project, all while standing on the recently completed foundation. The client could walk through the virtual spaces at 1:1 scale, tapping surfaces to change colors or finishes. Or a medical team might examine a patient's MRI or CT scans in full 3D before entering the operating theatre, pulling apart layers of information and discussing them collaboratively. And of course, gaming on mobile VR can now move beyond gaze shooters and single-button inputs to explore the mechanics made possible by the rich input of the human hand.

The future of virtual and augmented reality relies on our ability to create natural interaction and compelling immersion. With touchless input and 6 DoF head tracking, the Qualcomm’s Snapdragon 835 VR development kit is an important step towards this future.

As with all new platforms, the success of mobile VR will depend on compelling content created by innovative developers. We’ve been working hard to enable you to do just that by building tools like the Interaction Engine, a layer that exists between game engine physics systems and real world hand physics. This and many other developer resources can be found on our developer site, including documentation for our Unity and Unreal integration.