In his recent discussion, Cross-Device AI Acceleration, Compilation, & Execution, with Sam Charrington of TWIML AI Podcast, Jeff Gehlhaar, vice president of Technology and head of AI software platforms at Qualcomm Technologies, had an insightful dialogue about the latest developments in cross-device AI technology and how Qualcomm Technologies aims to harmonize the consumer AI experience across the Snapdragon mobile platforms.
Let's take a brief look at some of the highlights and insights from the podcast and illuminate the exciting new technologies and resources that developers and consumers can expect to see in the near future.
Compilation
Given our recent successes in the ML Commons benchmarking, Qualcomm Technologies aims to increase acceleration by further optimizing the architecture of the Snapdragon 888 mobile hardware development kit (HDK) platform. To build upon our past successes and achieve increased acceleration, Qualcomm Technologies is working to harmonize our portfolio by specializing our approach to compilation.
Currently, Qualcomm Technologies is working with both Glow and TVM compilers because each have unique elements that make them a strong suit when implemented in specific situations. Glow excels when utilized with the Qualcomm Neural Processing SDK for AI) in cloud-class large-scale inference solutions such as the Qualcomm Cloud AI 100. Whereas with TVM, we’ve found it best utilized when using the Qualcomm Hexagon DSP SDK, for the more constrained low-level optimizations in use cases where the hardware is always on.
Bridging execution framework
Qualcomm Technologies has constructed the Qualcomm AI Engine direct in order to provide cutting-edge acceleration in a package that can be utilized in the ecosystem along a number of different routes. Qualcomm Technologies aims to harmonize its portfolio with increased accessibility by providing cutting-edge hardware and software acceleration at the low level with more choice in accessibility as customers move up product offerings.
On top of our hardware, Qualcomm Technologies looks to increasingly utilize ONNX in conjunction with its APIs to function as a data interchange format that will increase its platforms’ accessibility. To further streamline the data interchange process, Qualcomm Technologies plans to integrate ONNX in the role of a runtime as a more permanent fixture of the platform. As a runtime, ONNX is engineered to orchestrate the deployment of the network in a way that utilizes the libraries on our hardware to most efficiently provide cutting-edge acceleration.
Updateable drivers
Gehlhaar points out that as customers adopt the technology and bring in use cases, Qualcomm Technologies has been able to make marked improvements on its platforms’ abilities by adjusting the software based on customer feedback. Qualcomm Technologies’ goal is to run its platforms with software that can be updated as fast as AI is changing. Updating the software rather than the hardware will make it so that consumers won’t need to go get a new phone — consumers can get an update that makes their current phone work better.
In many of Qualcomm Technologies’ products such as the Qualcomm Neural Processing SDK for artificial intelligence, there are releases to customers, but this often requires product refreshes, updates, or new apps. By working with Google and Android, Qualcomm Technologies can make updates directly to the end device to enable continuous improvement on the Snapdragon platform.
Quantization
Gehlhaar views the commercialization of the Snapdragon 888 mobile hardware development kit (HDK) as a tremendous opportunity to integrate the various tools across the platform and achieve a harmony of accessibility across the full stack. The harmony that can be achieved by this integration can allow developers to utilize the AI Model Efficiency Toolkit (AIMET) and its advanced model quantization and compression techniques for trained neural network models. By harmonizing and reducing the friction when working with the HDK, Qualcomm Technologies aims to increase the ability of developers and minimize customers’ need for data scientists to work with these toolkits.
Check it out
You can access Gehlhaar’s podcast with Sam from TWIML AI, along with Sam’s past podcasts here. For additional updates from Gehlhaar, be sure to follow his Linkedin and Twitter feed.
Snapdragon, Qualcomm Neural Processing SDK, Qualcomm Cloud AI, Qualcomm Ai Engine and Qualcomm Hexagon are products of Qualcomm Technologies, Inc. and/or its subsidiaries. AIMET is a product of Qualcomm Innovation Center, Inc.
