IoT has gone from connecting ordinary devices like light bulbs, cameras, sensors and thermostats, to smart assistants, baby monitors and smart appliances. We can now use data from sensors, microphone/audio and camera/video for the continuous training and predictive capabilities of machine learning and artificial intelligence (AI).
We’re now ready to operate smart, powerful devices out at the network edge.
AI at the Edge
AI based on neural networks consists primarily of training and inferencing.
- Training: Experts build a deep neural network (DNN) model to solve a particular problem, like object classification or voice recognition. They perform the compute-intensive work of training the model in the cloud or data center.
- Inferencing: Once the model is reliable, it’s ready for deployment on IoT devices, which compare incoming data from sensors, microphones and cameras against the trained model to make "intelligent" decisions.
Based on advancements in software algorithms, improved compute resources in hardware and stronger security, the work of inferencing no longer needs to take place in the cloud; it can now take place on devices at the network edge.
What are the benefits of AI at the edge? Inferencing at the edge provides lower-latency response, lower bandwidth costs and lower storage costs compared to cloud-based solutions. For example, an object-tracking application can be much more responsive when it analyzes a video feed from the camera locally, instead of having to send the feed from the network edge to the cloud for processing and await the results. Inferencing at the edge also results in greater privacy and security, since data stays confined to the IoT device instead of being transferred over the network.
How to run AI at the edge for IOT devices
AI on IoT devices opens up vast opportunities for audio, video and sensor analytics, which can be used across connected cameras, home hubs, drones/robotics and XR as well as industrial IoT and smart cities. In upcoming blog posts I’ll describe what those opportunities mean for developers.
On-device AI needs to run on the right compute cores. A powerful, multi-core CPU alone may not be sufficient. The graphics processing unit (GPU) might be better suited for detection and tracking, while the compute-intensive workload of classification may run better on the digital signal processor (DSP). Better still, with the addition of Hexagon Vector eXtensions (HVX), the DSP can process images at high speed with extremely low power consumption.
The key to successfully deploying AI in IoT is to run the workload on the core that ensures low power consumption with optimal inferencing performance. That is why having the flexibility to run neural network models heterogeneously across various cores - GPU, DSP/HVX and CPU - is a distinct advantage for developers working on platforms from Qualcomm Technologies.
For example, AI on IoT devices can bring visual intelligence to personal, community and enterprise use cases:
- Video summarization
- Scene classification
- Text or license plate recognition
- Face detection and recognition
- Object detection and classification, with depth and location tracking
For instance, face recognition using DNNs could bring enhanced security to the enterprise if matched with the employee database. Another example would be to use DNNs for counting people and classifying them by gender and age - a boon for retail.
AI on IoT devices can also introduce audio intelligence to new use cases:
- Audio classification (for example, a baby crying, glass breaking, identifying gunshots)
- Voice activation and recognition of individual speakers
- Echo cancellation and noise suppression
- Natural language processing/Automatic speech recognition (NLP/ASR)
The Qualcomm Snapdragon Neural Processing Engine
IoT devices powered by the Qualcomm Technologies system on a chip (SoC) have 2 to 3 different cores available for running AI workloads:
- Qualcomm Kryo™ CPU
- Qualcomm Adreno™ GPU
- Qualcomm Hexagon™ DSP, including recently released versions with HVX
Qualcomm Technologies is providing the Snapdragon Neural Processing Engine (SNPE), a software framework designed to execute on each of the compute cores - GPU, DSP and CPU - that power Qualcomm Technologies SoCs. The SNPE SDK includes conversion tools for migrating existing networks based on Caffe/Caffe2 and TensorFlow to the Qualcomm Technologies SoC runtime environment. It also provides debug tools and performance optimization tools for each of the cores. Developers can optimize their applications to run the AI tasks on the core best suited to their use cases.
The combination of our highly integrated SoC and SNPE takes advantage of heterogeneous computing for high-performance, power-efficient AI on IoT edge devices, whether in robotics, the smart home or industrial applications.
Qualcomm Technologies is working to broaden the ecosystem around AI for IoT. We plan to showcase the first fruits of our collaborations exclusively in Qualcomm Technologies' meeting rooms at the Ritz Carlton (#23-26) during China Public Security Expo in Shenzhen, October 29 - November 1, 2017.
The SNPE SDK offers you deep learning at the edge on IoT devices powered by Snapdragon, which is optimized for high compute at low power consumption. Download the SDK and start experimenting with the advantages it offers your apps in performance and energy efficiency.