Jan 13, 2017
Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.
At CES 2017, Qualcomm Technologies unveiled several initiatives to drive farther into the smart, connected car space, including using the Qualcomm Snapdragon 820A to power next-generation Volkswagen infotainment systems, announcing a Cellular V2X trial in Germany, revealing a new class of high-bandwidth Gigabit LTE for vehicles and demonstrating how Snapdragon 820A can bring the next level of intelligence to connected cars.
Qualcomm Snapdragon 820A goes beyond infotainment
Snapdragon 820A provides the enhanced capabilities of the complete software stack—supporting multiple operating systems and frameworks including Automotive Grade Linux (AGL), QNX and Android. The Snapdragon 820A processor is not your typical infotainment processor. It is designed to allow automotive companies to integrate machine learning-based Informational ADAS and User Interface personalization into cars with a deep learning software development kit (SDK) called the Qualcomm Snapdragon Neural Processing Engine. The Neural Processing Engine SDK is engineered to efficiently use the Snapdragon processor’s heterogeneous compute capabilities—providing auto makers a powerful, energy-efficient platform for delivering the next level of automotive intelligence. It actually opens the door to a slew of innovative informational ADAS use cases, from intelligent surround view monitoring with sensing and drivable space calculation to natural language processing/understanding, HMI customization, and personalized user profiles. The possibilities are endless.
Following are some of the demonstrations from CES 2017 that show how Qualcomm and auto technology companies are taking advantage of the on-device machine learning capabilities of Snapdragon 820A to deliver intelligent informational ADAS capabilities
Redefining driving as we know it through Qualcomm Drive Data Platform
Qualcomm demonstrated the Qualcomm Drive Data Platform, which brings together the technological innovations that will be necessary for Advanced Driver Assistance Systems (ADAS) and next-generation automotive services, such as shared mobility and autonomous driving. At CES 2017, we showcased map crowdsourcing and critical safety alert use cases—demonstrating detection of traffic signs, lane markers, road gradient, driver distraction and forward tailgating events—based on the Neural Processing Engine running on Snapdragon 820A.
Driver behavior monitoring for fleet management by Nauto
Nauto demonstrated the real-time machine learning capabilities of Snapdragon 820A for fleet management and driver monitoring use cases. The demo used a Snapdragon 820A-equipped aftermarket device mounted on the inside of a car’s windshield. The device had a dual camera system, integrated inertial sensors, GNSS, and LTE connectivity. The algorithm was running on Snapdragon 820A—fusing data from an array of smart sensors to analyze driver behavior, monitor vehicular activity and track environmental conditions in real time. All this is done using different neural network models operating simultaneously on Snapdragon 820A using the Neural Processing Engine. The demo showed the capabilities of the algorithm to continuously monitor the driver and send alerts to fleet managers when the driver is not attentive, and generate an overall safety score for the driver based on different events, such as harsh braking, tailgating and/or acceleration.
Using the Qualcomm Snapdragon 820A processor and the Snapdragon Neural Processing Engine, Nauto is able to offer a connected camera for fleets with unique, advanced features including CNNs for real-time driver distraction and tailgating detection.
Frederick Soo, CTO for Nauto
Unified deep neural network for semantic segmentation and object detection by ZongMu Tech
ZongMu demonstrated simultaneous detection of various object classes such as vehicles, pedestrians, and bicycles, as well as free-space calculation using semantic segmentation, in difficult urban scenarios. The demo used ZongMu’s proprietary unified deep neural network that outputs multi-class object detection and road semantic segmentation simultaneously. The network model was running in real-time on a Snapdragon 820A ADP-connected automotive cameras based on HDR image sensor. The system uses the Neural Processing Engine for executing the network optimally by utilizing the heterogeneous processing cores of Snapdragon 820A. ZongMu plans to keep adding more classes of object detection and semantic segmentation so that autos can fully perceive their surroundings.
We are very excited to demo our unified neural network model that performs detection and segmentation simultaneously and have it running on Snapdragon 820A in real time with an automotive grade camera. The performance we are seeing is excellent and this is achieved with very efficient power consumption. Qualcomm Snapdragon Neural Processing Engine has been a very flexible framework with a rich toolset that enables us to port our customized network model.
Rui Tang, CEO and Founder of ZongMu
Automotive Machine Vision SDK by RT-RK
RT-RK demonstrated intelligent surround view along with driver monitoring by using its Automotive Machine Vision (AMV) software development kit on top of the Qualcomm Snapdragon 820A. The AMV framework efficiently fuses information from multiple cameras and automotive sensors to bring in informational safety features to Infotainment systems. The framework allows the automotive ecosystem to build multi-sensor based informational ADAS solutions along with infotainment functionalities by utilizing the Neural Processing Engine and Symphony SDK to efficiently run perception and machine learning on the heterogeneous cores of Snapdragon 820A.
Within the Automotive Machine Vision software development kit (SDK) by RT-RK, we provided the possibility that multiple informational ADAS algorithms are executed in parallel, by means of dedicating each pipeline phase to a different computing core (CPU, DSP or GPU). AMV also abstracts all automotive sensing inputs and uses the Qualcomm Neural Processing Engine and Symphony SDK to utilize heterogeneous computing cores for machine learning and computer vision. Snapdragon 820A is a great basis for our SDK.
Nikola Teslic, CEO of RT-RK LLC
These are just some examples of how Qualcomm Technologies is bringing new levels of intelligence to the connected car, paving the way for the always connected, autonomous vehicle of the future. Learn more by visiting the Qualcomm Drive Data Platform page.