OnQ Blog

Developers: AI accelerated experiences and security with Windows on Snapdragon

Nov 9, 2021

Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.

The 2021 ARM DevSummit was held from October 19 to 21, 2021. At the Summit, I took part in a presentation with Marcus Perryman, Principal Engineer – Windows Team at Microsoft, titled AI Accelerated Experiences & Security with Windows on Snapdragon. Our presentation provided insight into using AI in apps for always-on, always-connected, Windows PCs powered by our Snapdragon compute platforms.

Our presentation acknowledged how society’s dependence on mobile devices continues to increase, particularly on PCs. Many aspects of our lives are experiencing this impact ranging from remote and hybrid working environments to staying connected with loved ones, anytime, anywhere. Fulfilling these requirements has fueled the demand for powerful yet energy-efficient platforms that can run apps like Teams for video chats and productivity packages like Microsoft 365 from virtually anywhere with all-day battery life.

AI is the key for rich, personal, and connected user experiences

Marcus and I discussed how the key to providing rich, personal, and connected user experiences in today’s remote climate is to augment apps with AI. Real-time AI calculations can be accelerated using on-device hardware engines when the utmost speed and performance are required with minimal power consumption. While cloud-based inference, accessible over high-speed links like 5G and Wi-Fi 6, can complement on-device AI with near-real-time performance.

For example, the Microsoft SQ1 processor, developed in collaboration with Qualcomm Technologies and used in the Microsoft Surface Pro X, runs on-device, hardware-accelerated inference for real-time gaze correction. This feature tracks a meeting participant’s eyes and digitally corrects them each frame to make the participant appear as if they’re looking at the camera, resulting in more personal and connected interaction, as shown in Figure 1:

Figure 1 – Example of real-time gaze correction using AI to make a meeting participant appear as if they’re looking at the camera.
Figure 1 – Example of real-time gaze correction using AI to make a meeting participant appear as if they’re looking at the camera.

Marcus noted that the traditional power consumption using a CPU and GPU to perform this feat is around 15 W. In comparison, the SQ1’s AI acceleration can do it with around 0.3 W of power – 50 times more efficiently. Moreover, the performance of the neural processing unit (NPU) is up to 10x faster than traditional CPUs because it is custom built to compute the complex multi-dimensional mathematical formulas required by AI neural networks.

During our presentation, we also noted numerous other existing and potential uses for AI in Windows apps, such as:

  • Backgrounds and video enhancement and replacement for video conferencing
  • Natural language processing (NLP) tasks such as real-time voice translation in video conferences, predictive suggestions in Word documents, or voice commands for voice UIs
  • Audio de-noising and cleanup to capture only the speaker’s voice
  • Security including biometrics, and behavioral analysis for threat detection
  • Computer vision tasks, including user and object detection, which can automatically lock content when a user walks away from their PC
  • Photo app filtering (e.g., changing contrasts, converting photos to sketches, etc.)

Marcus talked about how Windows 11, and apps like Office 365, are aligned to support such experiences on high-performance devices like Snapdragon compute platforms. These devices are always-connected, up-to-date, and capable of maintaining long battery life.

Resources for developers

Developers can start by visiting the Windows on Snapdragon page on QDN to access additional information. Here we have put together a list of additional tools for Windows development on QDN that you should check out.

To allow developers to convert their AI models for optimal performance on Windows PCs built around Snapdragon processors, developers can use the Qualcomm Neural Processing SDK for artificial intelligence (AI). With this SDK, developers can import models built with TensorFlow, PyTorch, Caffe, and ONNX to take advantage of hardware-accelerated, on-device AI across a variety of verticals.

Microsoft provides Windows on ARM tools and Windows on ARM documentation which developers should use to get started. And as noted in the presentation, Visual Studio 2022, currently in pre-release, will support Windows 11 and be updated monthly.

Additional blogs

  • Windows on ARM development – How do I get started with the tools?
  • On-device AI with Developer-Ready Software Stacks
  • Neural Network Optimization with AIMET
Snapdragon and Qualcomm Neural Processing SDK are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

 

Engage with us on

and

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries ("Qualcomm"). Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.

Rami Husseini

Director, Product Management

©2021 Qualcomm Technologies, Inc. and/or its affiliated companies.

References to "Qualcomm" may mean Qualcomm Incorporated, or subsidiaries or business units within the Qualcomm corporate structure, as applicable.

Qualcomm Incorporated includes Qualcomm's licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm's engineering, research and development functions, and substantially all of its products and services businesses. Qualcomm products referenced on this page are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

Materials that are as of a specific date, including but not limited to press releases, presentations, blog posts and webcasts, may have been superseded by subsequent events or disclosures.

Nothing in these materials is an offer to sell any of the components or devices referenced herein.