OnQ Blog

Qualcomm Zeroth is advancing deep learning in devices [VIDEO]

Enabling machines to identify and understand their environment, so that they can work smarter and more efficiently.

Mar 2, 2015

Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.

Throughout 2014, Qualcomm Research (a division of Qualcomm Technologies, Inc.)  made significant progress in its brain-inspired computing research initiative known as Zeroth. We had set out to gather feedback from a very select group of companies on the architecture and tools we have developed for biologically plausible spiking neural networks. For Qualcomm Research, academia, and the industry as a whole, research in computation with spiking neural networks is an ongoing effort.

The data and feedback gathered from these activities led the team to expand the scope of research to look not just at biologically realistic approaches but artificial neural networks as well. Both, after all, take inspiration from the brain and artificial neural networks are at the core of a branch of machine learning known as deep learning.

Time and time again, deep learning-based networks are demonstrating state-of-the-art results in pattern-matching tasks. This makes a deep learning-based approach ideal for giving our devices human-like perceptual pattern-matching capabilities.

Following are some examples of the team’s recent achievements in advancing deep learning.

On-Device deep learning

Deep learning networks to date have primarily been discussed and researched in the context of the cloud or high performance computing clusters of CPUs and GPUs. The Zeroth team has made it possible for large-scale deep-neural networks to run efficiently and on the Qualcomm Snapdragon mobile system-on-chip (SOC). This opens a whole new range of possibilities for human-like perceptual pattern matching on smartphones, robots and other devices we interact with in our daily lives.

Deep networks would still be trained on large CPU or GPU clusters with lots of data, but the trained networks can now be deployed on a Snapdragon-powered device for fully on-device processing of tasks like image classification, object recognition and face recognition, to name just a few.

Late last year the Zeroth research team demoed the Snapdragon Cargo, a robot that could learn to put away your toys (identify, classify and place toys in a bin of your choosing). The robot was able to do this based on a deep learning-based classifier that was running on a Snapdragon processor.

More recently

Another new feature of Zeroth, also functioning fully on-device, provides the ability to recognize faces in near real time on a Snapdragon-powered device using deep convolutional neural networks (CNNs) with a top level classifier that can be retrained on device. This feature has been integrated into the Snapdragon Rover and provides the robot with the ability to learn to recognize new people’s faces with just a few examples and without having to retrain the entire convolutional neural network. The same technique could also be applied to customizing and personalizing other types of recognition categories also on device. This is shown in the following video.

A preview of things to come

The cutting edge in research with neural network-based processing is for pattern matching on data that has a time component to it. Key example scenarios for this include activity recognition in video, handwriting recognition, speech and natural language processing. Approaches that include a variant of a type of neural network known as a recurrent neural network (RNN) show the most promise in achieving human-like performance for recognizing patterns over time.

The Zeroth team worked with Planet GmBH to demonstrate the power of deep convolutional recurrent neural networks running on Snapdragon and in the Zeroth Platform. The video below illustrates this technology functioning fully on-device, running offline OCR (optical character recognition) based handwriting understanding.

You can see how unconstrained English text written with pen and paper can be recognized. This is the first time fully OCR-based handwriting recognition on a mobile device has been shown to be possible. Typically text recognition solutions use either OCR for recognizing printed text and/or a digital stylus for capturing and recognizing handwriting. OCR-based handwriting recognition is considered one of the harder problems to solve in artificial intelligence.

The team has an ambitious agenda of near and long term research for further enablement of deep learning on the Snapdragon platform. They are researching deeper optimizations in hardware and software for the whole mobile SOC to make it possible to run bigger and more complex networks on the next generation of Snapdragon. 

Zeroth research has evolved from focusing on biologically realistic spiking neural networks to also include artificial neural networks for on-device deep learning. This is one major pillar to make Zeroth a platform for bringing human-like perception to the devices we all interact with every day.

Stay tuned as we’ll have more frequent updates on the progress of Zeroth this year!

Check out Snapdragon blog and learn how cognitive computing and a certain custom CPU drive next-gen Snapdragon processors. Also, we published another OnQ post this morning that discusses Qualcomm’s vision of bringing cognitive technologies to life.

Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc.


Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries ("Qualcomm"). Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.

Samir Kumar

Sr. Director, Business Development