Oct 24, 2014
Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.
Did you know you can now build applications like these for see-through digital eyewear?
- A technician follows step-by-step visual instructions to replace a failed module in an elevator.
- A gamer plays in a hybrid landscape of augmented reality (AR) content, virtual reality content and physical toys.
- A student conducts virtual experiments in an augmented science textbook.
- A shopper gets a 3D preview of a product without opening the package.
These immersive experiences are a perfect fit for see-through digital eyewear, which is why we announced the Qualcomm® Vuforia™ SDK for Digital Eyewear. It leverages the powerful vision capabilities of our popular Vuforia SDK, which has a global ecosystem of more than 100,000 registered developers and has powered more than 10,000 Android and iOS mobile apps.
At Uplinq™ 2014, Qualcomm’s annual mobile developer conference, Vuforia Product Manager Giuliano Maciocci and Principal Engineer Andrew Everitt presented “Bringing Mobile Vision to Wearables,” explaining the technical challenges, key use cases and hardware platforms involved in implementing computer vision (CV) and AR on stereoscopic displays right in front of people’s eyes. Here are some of the highlights of their presentation.
Challenges Of Line-Of-Sight Content In See-Through Digital Eyewear
First, Giuliano emphasized that see-through, stereoscopic digital eyewear is a very different paradigm for a developer than a small mobile screen in a corner of the user’s vision. Stereo, stereoscopic digital eyewear in fact presents the key challenge of precisely aligning content – be it context-aware instructions, callouts, or 3D models, – in a direct line between the eye and a real-world physical target, in a way that accommodates to each user’s eyes.
As manufacturers like Epson, Samsung and ODG blaze the trails of digital eyewear, Giuliano explained the technical obstacles that Vuforia SDK for Digital Eyewear overcomes so that developers can focus on simply displaying content:
- Facial geometry is different for every human, so a good digital eyewear solution needs to take into account variables such as the distance between the eyes, or the way in which the glasses sit on the bridge of the nose. A good digital eyewear solution needs to understand the complex relationship between a real world target, the camera, the display and the eye, to ensure that digital content aligns properly with real-world targets.
- Binocular vision is tricky. If images are not correctly balanced, then the result can be eye strain and a poor user experience.
- Tracking latency becomes more apparent. Users are moving their head, and the eyewear display needs to keep up.
- User interaction enters a completely new world. There is no keyboard, mouse, or touchscreen, so developers need to find new ways of interacting with digital content.
Under The Hood – How We Do It
Vuforia for Digital Eyewear addresses all of those challenges for both users and developers. We’ll cover three of its main features in this post, then point you to the presentation video for more information.
- User calibration – The Vuforia SDK for Digital Eyewear accounts for the fact that everyone’s eyes are different. This is the key challenge to see-through, line of site vision experiences, so we’ve rolled a quick four-step calibration process into a run-once app that generates and stores a unique calibration profile for each user of a digital eyewear device. The profile calculates the relationships among the target, the device’s camera, the display and the user’s eye. The calibration process takes 30 to 60 seconds, and if you’d rather build and brand your own calibration app or roll the calibration steps into your app’s workflow, you can use the calibration API we provide to do just that. Once a calibration is performed, all applications using the Vuforia SDK for Digital Eyewear will automatically pick up the sored calibration profile and adapt their content accordingly.
- Stereoscopic rendering – If working in Unity, you don’t need to recreate your existing content for binocular digital eyewear. The Vuforia SDK for Digital Eyewear includes a drag-and-drop Unity camera prefab that supports the simple conversion of Unity content to stereo – and calibration profile support is built right in.
- Video background removal – In mobile AR use cases, you can simply align AR content with the video feed from the device’s camera. But in see-through digital eyewear, you need to align AR content directly with a real-world object on the other side of the display. Vuforia for Digital Eyewear does this by turning the video background off or on, depending on the type of eyewear.
We’ve designed all the features in the Vuforia SDK for Digital Eyewear so that you don’t have to worry about a comfortable user experience. You can just focus on content.
The Digital Eyewear Experience Zone at Uplinq 2014 included demonstrations of the Vuforia SDK for Digital Eyewear on the Epson Moverio BT-200, the ODG R-7 (powered by the Qualcomm Snapdragon™ 805 processor) and the Samsung Gear VR Innovator Edition. The demonstrations showcased use cases spanning from machine maintenance to interactive gaming, as well as hybrid AR/VR experiences.
The use cases are real and the technology is ready. Are you?
- Come up with your own cool, vision-based experience for see-through digital eyewear.
- Sign up for our beta program for the Digital Eyewear SDK.
- Watch the 32 minute video below, which includes more about how we do it all, and provides a walk-through of both the Digital Eyewear SDK and the Unity Extension.