Jun 9, 2016
Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.
The Qualcomm Snapdragon 652 processor was designed to deliver the best in high performance mobile user experiences and was recently selected by Lenovo for the world's first consumer Tango-enabled device to deliver augmented reality (AR) experiences in a smartphone form factor. The Lenovo PHAB2 Pro was launched June 9th at Lenovo Tech World 2016 in San Francisco to an excited crowd wanting to learn more about Tango and the new Lenovo device.
The collaboration between Lenovo and Qualcomm using Tango technology was previously announced at CES 2016 where Snapdragon was announced as the mobile processor powering the future PHAB2 Pro due to several key benefits:
- Precise time-stamped sensor
- Efficient hardware integration with no incremental hardware or co-processor required
- Advanced camera and sensor processing capabilities
- Customized Tango software optimizations
Tango delivers a fast, smooth smartphone AR experience, where 3D AR graphic elements track more naturally to physical surroundings. To make that happen, Tango requires all sensor data to be timestamped precisely to a common timeline. These sensors include the rear camera, depth camera, motion tracking camera, front camera, accelerometer, gyroscope, and display.
The need for accurate timestamping is so critical it requires updates at least every 50 microseconds (µs) or things start to go, well… not where you want them to be. Without this microsecond accurate timestamping, where a device is and where it “thinks” it is will drift farther and farther apart, resulting in objects on the screen suddenly jumping from one position in the room to another.
To give you a feel for how accurate 50 µs is, GPS satellites orbiting the earth have to be corrected by around 40 µs every day to account for the effects of time and space that Einstein’s theory of general relativity predicted. 50 µs is so small, we practically needed to get Einstein involved to calculate it! Because Snapdragon processors were already built to support GPS as well as the most demanding mobile networks worldwide and designed to support leading sensor technologies, the Snapdragon 652 has a built-in high-frequency clock providing uniform timestamping for all key subsystems, including camera, sensor core, modem, and DSP. This includes camera frames, accelerometer, and gyroscope information, all of which are critical for the precise functioning of Tango’s advanced sensor fusion algorithms. All of the latest Snapdragon processors do this with no additional hardware components needed, saving hardware cost, integration expense, and simplifying certification for OEMs and operators.
Efficient hardware integration and processing
In the PHAB2 Pro, thanks to the integrated Snapdragon 652 processor, users get a smooth, fast experience, where 3D graphic overlays match up smoothly and precisely with the surrounding physical environment. This is due not only to the timestamping happening every 50 µs in Snapdragon, but also because Qualcomm Technologies designed the processor to efficiently distribute computer vision workloads, particularly the 6 degrees of freedom—or “6DoF”—positioning used in Tango and other AR applications. Snapdragon processors distribute this processing across the Qualcomm Hexagon digital signal processor (DSP), CPU, and Qualcomm Adreno GPU. While the Hexagon DSP helps determine exactly where and how the device is moving through 3D time and space, the CPU and GPU are free to handle displaying the enhanced visualized 3D information on the screen and running other processes needed to deliver an immersive Tango user experience.
Other companies approach advanced capabilities like Tango by using specialized co-processors. Many people are surprised to learn that these same services can be performed by a single, integrated and highly efficient mainstream processor such as the Snapdragon 652. We talked about the highly integrated Hexagon DSP, CPU and Adreno GPU. We also integrate the Imaging Signal Processor (ISP) and sensor hub into each Snapdragon processor. By integrating all of these components into the Snapdragon processor in hardware and software, we enable the components to run more efficiently and talk to each other more effectively, allowing for fast and smooth experiences with Tango technology. The DSP, camera, and sensor subsystems of Snapdragon processors are also all capable of sharing memory with each other, thereby enabling the zero-copy low-latency and power efficient transactions required by a complex framework such as Tango.
All-Ways Aware hub and advanced camera and sensor processing
At the heart of the PHAB2 Pro, the Snapdragon 652 processes all the rich sensor data being received simultaneously from multiple camera sensors, gyro and accelerometer. The built-in All-Ways Aware hub enables Snapdragon to obtain and process sensor data quickly/at low latency. Tango utilizes three different cameras—an RGB camera, a feature-tracking camera, and a depth sensor. The Snapdragon camera framework enables vendor-specific processing software (such as modules to generate a depth map from depth sensor input) to be seamlessly plugged in.
Custom software optimizations
To optimize Tango performance on the PHAB2 Pro, Qualcomm Technologies, Lenovo, and Google engineers focused on customized software optimizations since the Snapdragon 652 processor offered all the necessary Tango hardware plus leading LTE and Wi-Fi connectivity.
Qualcomm Technologies engineers were able to improve accuracy and reduce screen jitter by increasing the accuracy of time synchronization for all sensor components. Additionally, the Snapdragon feature-tracking engine is optimized to run on the DSP and Neon processors at very low power, enabling the GPU to remain fully available to actively support Tango applications and games which are expected to play a big role in the overall Tango experience.