The Hexagon NPU is our custom-designed NPU. It is designed to inference AI workloads based on very long instruction word (VLIW) processor with specialized signal processing capabilities. It uses simultaneous multithreading (SMT) to take advantage of thread-level parallelism and hide latency. The Hexagon NPU includes scalar, vector, and tensor accelerators and a large-shared memory to move data between these accelerators and can provide massive per-clock throughput.

In 2007, the first Hexagon DSP was launched on the Snapdragon Platform — the DSP control and scalar architecture was the basis for our future NPU generations. In 2015, the Snapdragon 820 processor was announced and included our first Qualcomm AI Engine to support imaging, audio, and sensor use cases. We added the tensor accelerator to the Hexagon NPU in the Snapdragon 855 in 2018. The following year, we expanded the use cases for on-device AI on Snapdragon 865 to include AI imaging, AI video, AI speech, and always-on sensing.

In 2020, we achieved a major milestone with a revolutionary architecture update for the Hexagon NPU. We fused together the scalar, vector, and tensor accelerators for better performance and power efficiency. A large-shared memory was dedicated for the accelerators to share and move data efficiently. The fused AI accelerator architecture established a solid foundation for our NPU architecture moving forward.

The Hexagon NPU is unique through its system approach, custom design, and fast innovation. We fused together the scalar, vector, and tensor accelerators for better performance and power efficiency and on top of that, a large-shared memory cuts across these accelerators for them to share data at high speeds. The Hexagon NPU also implements micro-tile inferencing, a technique used to boost AI performance at ultra-low power. By running multiple micro tiles simultaneously, the scalar, vector, and tensor accelerators are all put to work at the same time, combining as many as 10 or more layers to eliminate almost all the intermediate memory reads and writes. Transform your AI capabilities and achieve the ultra-low power performance with the Hexagon NPU—setting a new standard in AI processing with unparalleled efficiency and speed.

AI use cases have two key challenges in common. First, their demanding and diverse computational requirements are difficult to meet in power- and thermally-constrained devices using general-purpose CPUs or GPUs, which serve multiple needs on the platform. Second, they are constantly evolving, so implementing them in purely fixed-function hardware can be impractical. As a result, a heterogeneous computing architecture with processing diversity gives the opportunity to use each processor’s strengths, namely an AI-centric custom-designed NPU, along with the CPU and GPU. For example, each excels at different tasks: the CPU for sequential control and immediacy, the GPU for streaming parallel data, and the NPU for core AI workloads with scalar, vector, and tensor math.

The Qualcomm AI Engine, featured in our Snapdragon platforms and many of our other products, is at the core of our on-device AI and heterogenous computing advantage. With the CPU, GPU, NPU and Qualcomm Sensing Hub all working together and the result of many years of full-stack AI optimization, the Qualcomm AI Engine provides best-in-class on-device AI performance at extremely low power to support use cases today and in the future.

By using an appropriate processor, heterogeneous computing maximizes application performance, thermal efficiency, and battery life to enable new and enhanced generative AI experiences.