Enabling augmented reality in unknown environments.

Qualcomm Research’s computer vision efforts are focused on developing novel technology to enable augmented reality (AR) experiences in unknown environments. A critical step in enabling such experiences involves tracking the camera pose with respect to the scene. In target-based AR, a known object in the scene is used to compute the camera pose in relation to it. The known object is most commonly a planar object, however it can also be a 3D object whose model of geometry and appearance is available to the AR application. Tracking the camera pose in unknown environments can be a challenge. In order to mitigate this challenge, We have developed a leading technology known as SLAM, which enables AR experiences on mobile devices in unknown environments.

Qualcomm Research has designed and demonstrated novel techniques for modeling an unknown scene in 3D and using the model to track the pose of the camera with respect to the scene. This technology is a keyframe-based SLAM solution that assists with building room-sized 3D models of a particular scene. The system runs a computationally intensive non-linear optimization called bundle adjustment to ensure that models with a high-level of accuracy are generated. This optimization is significantly improved using SIMD on NEON to make certain that a smooth operation occurs on mobile devices.

The system is designed to support a responsive AR application by delivering a camera pose in real-time at 30 Hz. In order to realize this technical challenge, the proprietary SLAM solution utilizes multiple cores in a multi-threaded architecture.

During fast motion tracking failures are typically experienced. In order to recover from such tracking failures, the system has a state-of-the-art relocalization routine which was developed to quickly compute an approximate pose from the camera when images are blurred or otherwise corrupted. The system also delivers a triangulated 3D mesh of the environment for the application to utilize in order to deliver a realistic augmented experience that is blended with the real scene.

A key factor in the development of this complex and responsive system, was for it to be very easy to use right from the start. As such, a unique initialization method for the SLAM system was developed which hides the user from the complexities of initializing a SLAM system. A camera pose is available to the application in the first frame, that enables applications to hide the initialization of the system from the user.

Qualcomm Research is a division of Qualcomm Technologies, Inc.

3D Reconstruction Demo Video

Feb 24, 2015


©2021 Qualcomm Technologies, Inc. and/or its affiliated companies.

References to "Qualcomm" may mean Qualcomm Incorporated, or subsidiaries or business units within the Qualcomm corporate structure, as applicable.

Qualcomm Incorporated includes Qualcomm's licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm's engineering, research and development functions, and substantially all of its products and services businesses. Qualcomm products referenced on this page are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

Materials that are as of a specific date, including but not limited to press releases, presentations, blog posts and webcasts, may have been superseded by subsequent events or disclosures.

Nothing in these materials is an offer to sell any of the components or devices referenced herein.