OnQ Blog

Cross-device AI acceleration, compilation and execution [video]

Aug 23, 2021

Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.

In his recent discussion, Cross-Device AI Acceleration, Compilation, & Execution, with Sam Charrington of TWIML AI Podcast, Jeff Gehlhaar, vice president of Technology and head of AI software platforms at Qualcomm Technologies, had an insightful dialogue about the latest developments in cross-device AI technology and how Qualcomm Technologies aims to harmonize the consumer AI experience across the Snapdragon mobile platforms.

Let's take a brief look at some of the highlights and insights from the podcast and illuminate the exciting new technologies and resources that developers and consumers can expect to see in the near future.

Compilation

Given our recent successes in the ML Commons benchmarking, Qualcomm Technologies aims to increase acceleration by further optimizing the architecture of the Snapdragon 888 mobile hardware development kit (HDK) platform. To build upon our past successes and achieve increased acceleration, Qualcomm Technologies is working to harmonize our portfolio by specializing our approach to compilation.

Currently, Qualcomm Technologies is working with both Glow and TVM compilers because each have unique elements that make them a strong suit when implemented in specific situations. Glow excels when utilized with the Qualcomm Neural Processing SDK for AI)  in cloud-class large-scale inference solutions such as the Qualcomm Cloud AI 100. Whereas with TVM, we’ve found it best utilized when using the Qualcomm Hexagon DSP SDK, for the more constrained low-level optimizations in use cases where the hardware is always on.

Bridging execution framework

Qualcomm Technologies has constructed the Qualcomm AI Engine direct in order to provide cutting-edge acceleration in a package that can be utilized in the ecosystem along a number of different routes. Qualcomm Technologies aims to harmonize its portfolio with increased accessibility by providing cutting-edge hardware and software acceleration at the low level with more choice in accessibility as customers move up product offerings.

On top of our hardware, Qualcomm Technologies looks to increasingly utilize ONNX in conjunction with its APIs to function as a data interchange format that will increase its platforms’ accessibility. To further streamline the data interchange process, Qualcomm Technologies plans to integrate ONNX in the role of a runtime as a more permanent fixture of the platform. As a runtime, ONNX is engineered to orchestrate the deployment of the network in a way that utilizes the libraries on our hardware to most efficiently provide cutting-edge acceleration.

Updateable drivers

Gehlhaar points out that as customers adopt the technology and bring in use cases, Qualcomm Technologies has been able to make marked improvements on its platforms’ abilities by adjusting the software based on customer feedback. Qualcomm Technologies’ goal is to run its platforms with software that can be updated as fast as AI is changing. Updating the software rather than the hardware will make it so that consumers won’t need to go get a new phone — consumers can get an update that makes their current phone work better.

In many of Qualcomm Technologies’ products such as the Qualcomm Neural Processing SDK for artificial intelligence, there are  releases to customers, but this often requires product refreshes, updates, or new apps. By working with Google and Android, Qualcomm Technologies can make updates directly to the end device to enable continuous improvement on the Snapdragon platform.

Quantization

Gehlhaar views the commercialization of the Snapdragon 888 mobile hardware development kit (HDK) as a tremendous opportunity to integrate the various tools across the platform and achieve a harmony of accessibility across the full stack. The harmony that can be achieved by this integration can allow developers to utilize the AI Model Efficiency Toolkit (AIMET) and its advanced model quantization and compression techniques for trained neural network models. By harmonizing and reducing the friction when working with the HDK, Qualcomm Technologies aims to increase the ability of developers and minimize customers’ need for data scientists to work with these toolkits.

Check it out

You can access Gehlhaar’s podcast with Sam from TWIML AI, along with Sam’s past podcasts here. For additional updates from  Gehlhaar, be sure to follow his Linkedin and Twitter feed.

 

Snapdragon, Qualcomm Neural Processing SDK, Qualcomm Cloud AI, Qualcomm Ai Engine and Qualcomm Hexagon are products of Qualcomm Technologies, Inc. and/or its subsidiaries. AIMET is a product of Qualcomm Innovation Center, Inc.

 

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries ("Qualcomm"). Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.

The OnQ Team

©2021 Qualcomm Technologies, Inc. and/or its affiliated companies.

References to "Qualcomm" may mean Qualcomm Incorporated, or subsidiaries or business units within the Qualcomm corporate structure, as applicable.

Qualcomm Incorporated includes Qualcomm's licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm's engineering, research and development functions, and substantially all of its products and services businesses. Qualcomm products referenced on this page are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

Materials that are as of a specific date, including but not limited to press releases, presentations, blog posts and webcasts, may have been superseded by subsequent events or disclosures.

Nothing in these materials is an offer to sell any of the components or devices referenced herein.