Wouldn’t it be great to have a droid like C-3PO or R2-D2 from Star Wars to help you out? Think about it… a machine that not only follows our commands but also learns on the go and makes decisions based on past experiences.
Can machines think like humans? That’s the question I asked in my last blog post. In that post, I outlined Qualcomm’s vision of how devices, machines and other “things” are becoming smarter by acquiring human-like understanding and behaviors. This is extremely important because future mobile experiences will be defined by the ability of mobile devices to learn and adapt to the needs of the user, through harnessing the growing levels of on-device intelligence, compute and connectivity. As devices can see, hear and understand their surroundings, they will serve as natural extensions of our senses, allowing us to discover and perceive the world around us in new ways. As they become more aware of our preferences and surroundings, devices will anticipate our needs and personalize our experiences based on where we are, what we are doing and what we plan to do. And finally, as devices acquire human-like behaviors, they will enable more natural user interfaces, adaptive user experiences, and autonomous interactions between devices and things.
Having this vision not only allowed us to think about how we make devices more capable by advancing computing, connectivity and multimedia technologies, it also enabled us to go to the next level by making every aspect of our leading technologies work together in a smarter, more intuitive way. We also envision that enabling on-device intelligence will be key for providing the faster response and enhanced reliability required for future mobile experiences without compromising user privacy.
To make this vision a reality, Qualcomm Technologies introduced the Qualcomm ZerothTM Platform, designed for on-device intelligence. The Zeroth Platform takes full advantage of our highly optimized heterogeneous computing architecture to achieve the performance required for on-device cognitive capabilities within the power and thermal constraints of mobile devices. It will take the user experience to the next level by providing the foundation for more intuitive experiences and natural interactions for a range of key mobile experiences and cutting-edge cognitive capabilities, including:
- Visual perception
- Speech and audio recognition
- Always-on awareness
- Intelligent connectivity
- Intuitive security
- Immersive multimedia
- Natural device interactions
For this post, I will discuss the first two capabilities that are related to human perception: visual perception, and speech and audio recognition. Replicating our human perception is not an easy task. To put this into perspective, consider that we use 70% of our human brainpower for visual perception. To deal with this challenge, Qualcomm Technologies has been developing on-device, human-like perceptual pattern matching for smartphones, robots and other devices.
Imagine devices that can see and understand their surroundings like humans. Through on-device machine learning, computer vision and cognitive camera technologies, devices can recognize objects, read handwriting, identify people and understand the overall scene and its context. This opens new possibilities. For example, redefining smartphone photography. Your smartphone camera can capture high quality photos and videos by autonomously adjusting its settings based on its understanding of the nature of the scene—imagine the different settings used when snapping shots at a football game versus a sunny beach versus a child’s birthday party. Be sure to check the Zeroth demos at Mobile World Congress for facial recognition and scene understanding to see how we’ve started to enable this vision.
Speech and audio recognition
Imagine devices that can hear, learn and take instant actions by recognizing environmental sounds and music through always-on listening, sound localization and sound identification. In the future, on-device intelligence will enable speech understanding, and audio scene recognition. For example, your phone will recognize if you are in a train or driving your car by identifying the sound of their engines, and will know if it should raise the volume, increase/decrease display brightness, or switch to text-to-speech mode.
More to come, more to learn about Zeroth
Be sure to catch my follow up to this post when I address the other five cognitive capabilities—always-on awareness, intelligent connectivity, intuitive security, immersive multimedia and natural device interactions.
To recap, the Qualcomm Zeroth Platform will set the stage for a new level of intelligence and personalization for mobile devices through its unique on-device cognitive capabilities. Building on the scale of mobile, it will also scale to power other devices, machines, and things, including IoE, automotive, robotics, embedded computing and networking.
Check out Snapdragon blog and learn how cognitive computing and a certain custom CPU drive next-gen Snapdragon processors. Also, we published another OnQ post this morning that describes how Qualcomm Zeroth is advancing deep learning in machines.