--- title: "Inference‑first architecture built for hyperscale" url: "https://www.qualcomm.com/data-center/expertise/ai-accelerators" description: "Dragonfly delivers rack-scale data center AI inference at up to 8× better tokens/watt vs. GPU, powered by High Bandwidth Compute." --- AI Accelerators # Inference‑first architecture built for hyperscale Qualcomm Technologies is redefining data center infrastructure with rack-scale inference solutions optimized for the agentic AI era. While legacy architectures were built for raw compute speed, the AI era demands a focus on extreme energy efficiency, real-time responsiveness, and total cost of ownership to maximize tokens per watt and tokens per dollar. [Contact sales](https://www.qualcomm.com/support/contact/forms/data-center) ![](https://s7d1.scene7.com/is/image/dmqualcommprod/ai-accelerator-hero-desktop?fmt=webp-alpha&qlt=85) ## Innovative architecture for exponential effective bandwidth Qualcomm® High Bandwidth Compute (HBC) delivers a generational leap in effective memory bandwidth, addressing the inference decode bottleneck and enabling token generation at lower power, latency, and cost. - **18x** — Effective memory bandwidth (AI250 with HBC Gen 1 vs AI200) - **54x** — Effective memory bandwidth (AI300 with HBC Gen 2 vs AI200) ### Qualcomm High Bandwidth Compute By fusing memory and compute dies, HBC processes extreme memory bandwidth within the package and skips energy-intensive data movement to SoC for operations with low arithmetic intensity. HBC eliminates the HBM tax and enables industry-leading tokens-per-watt and tokens-per-dollar. ![Diagram of QCM memory architecture, showing computation happening within memory, and data moving to the SoC only when necessary.](https://s7d1.scene7.com/is/image/dmqualcommprod/accelerator-hbc-diagram?$QC_Responsive$&fmt=png-alpha&wid=700) ### Compared to HBM The traditional HBM approach transmits high bandwidth data through the entire path from DRAM arrays to the SoC, a model that frequently hits the "memory wall". This architecture relies on expensive 2.5D packaging and results in higher costs per gigabyte, more power consumed, and less effective memory bandwidth compared to HBC. ![Diagram of HBM memory architecture, showing data moving to the SoC for every operation.](https://s7d1.scene7.com/is/image/dmqualcommprod/Inference-HBM-Comparison?$QC_Responsive$&fmt=png-alpha) ## Key ingredients for rack-scale AI inference ### Near-memory computing HBC has a multi-year roadmap to deliver faster, more power-efficient, and more scalable processing at lower TCO compared to HBM. ### Purpose-built neural processing The [Qualcomm Hexagon™ NPU](https://www.qualcomm.com/processors/hexagon) technology delivers industry-leading performance and energy efficiency to process the most demanding AI tasks. ### Unrivaled compute density Our custom [Qualcomm Oryon™ CPU](https://www.qualcomm.com/processors/oryon) technology, purpose-built for servers, delivers industry-leading power efficiency, performance, and rack density. ### High-speed connectivity & interconnects A comprehensive portfolio of advanced connectivity solutions optimized for scale‑up, scale‑out, and scale-across. ### Advanced nodes Leading-edge process at 3nm and below for high performance and low power. ### Leading performance-per-watt Our holistic systems approach and co-design across AI, compute and connectivity lead to enable rack-level power efficiency. ## Bring innovative efficiency to AI performance Our holistic full-stack architecture for AI racks leverages decades of expertise in power efficiency, performance optimization, complex systems design, and memory management to deliver scalable AI infrastructure that optimizes total cost of ownership (TCO) at hyperscale. ### Orchestration software Coordinates multi-stage AI inference workflows and aligns them to the right infrastructure resources via the [Qualcomm Dragonfly AI Infrastructure Management Suite (AI MS).](https://www.qualcomm.com/data-center/solutions/ai-infrastructure-management-suite) ### Connectivity The "connectivity tissue" provided by Qualcomm Dragonfly high-speed connectivity protocols. ### Core infrastructure Our efficient and custom-designed AI inference solutions are based on our differentiated IP, including Hexagon NPU and Qualcomm Oryon CPU technology scaled for data center workloads, along with HBC to address the “memory wall.” ![](https://s7d1.scene7.com/is/image/dmqualcommprod/Inference-Architecture-Diagram?$QC_Responsive$&fmt=png-alpha) ## AI performance engineered for hyperscaler infrastructure Inference for hyperscalers is bottlenecked by memory bandwidth, memory capacity and data movement energy costs. Our architecture optimizes for token generation efficiency, prioritizing low latency consistency, power efficiency, and unit economics over theoretical peak performance. - **Tokens / (second * watt)** — up to **8x** — better than GPU based systems¹ - **Memory bandwidth / watt** — **6x** — higher than HBM based systems² - **Memory capacity / watt** — **200x** — better than SRAM based systems³ - **Performance / watt** — **>2x** — better than the latest server CPU systems⁴ - **Effective memory bandwidth** — **54x** — higher per card (AI300 with HBC Gen 2 vs AI200) ¹ With Qualcomm HBC; Qualcomm estimates compared to contemporary GPU-based architectures on decode performance for select models. ² With Qualcomm HBC; Qualcomm estimates compared to competing published product specifications normalized at card-level. ³ With Qualcomm HBC; Qualcomm estimates compared to competing published product specifications normalized at rack-level. ⁴ With Qualcomm Dragonfly C1000 CPU; Qualcomm estimates compared to existing product benchmarks for server CPU competitive offerings based on specs.​ AI Accelerator Products ## An innovative roadmap backed by long‑term investment | Product | Description | Specification 1 | Specification 2 | |---|---|---|---| | Qualcomm Cloud AI 100 Ultra | Inference accelerator optimized for generative AI and LLM workloads. | **548 GB/s** — Memory bandwidth per card | **128 GB LPDDR4X** — Memory capacity per card | | Qualcomm Dragonfly AI200 | Rack‑scale inference platform with superior memory capacity and low TCO. | **414 TB/s** — Memory bandwidth per rack | **38.6 TB LPDDR5X** — Memory capacity per rack | | Qualcomm Dragonfly AI250 | Introduces Qualcomm High Bandwidth Compute for disaggregated inference workloads. | **7.4 PB/s** — Effective memory bandwidth per rack with HBC Gen 1 | **38.6 TB** — Memory capacity per rack | | Qualcomm Dragonfly AI300 | 3rd generation rack-level AI inference platform with increased bandwidth and deployment options. | **3x AI250** — Effective memory bandwidth per rack with HBC Gen 2 | **3x AI250** — Memory capacity per rack | - [Learn more](https://www.qualcomm.com/data-center/products/cloud-ai-100-ultra) (Qualcomm Cloud AI 100 Ultra) - [Learn more](https://www.qualcomm.com/data-center/products/qualcomm-dragonfly-ai200) (Qualcomm Dragonfly AI200) - [Learn more](https://www.qualcomm.com/data-center/products/qualcomm-dragonfly-ai250) (Qualcomm Dragonfly AI250) - [Learn more](https://www.qualcomm.com/data-center/products/qualcomm-dragonfly-ai300) (Qualcomm Dragonfly AI300) ![](https://s7d1.scene7.com/is/image/dmqualcommprod/AIMS-BACKGROUND-DESKTOP?fmt=webp-alpha&qlt=85) Qualcomm Dragonfly AI Infrastructure Management Suite ### Orchestrate and scale efficiency The suite provides provisioning, monitoring, orchestration and fault handling across rack‑scale deployments. Together, hardware, connectivity and software form the foundation of a cohesive data center platform approach — one designed to scale with customers as AI workloads evolve. [Learn more](https://www.qualcomm.com/data-center/solutions/ai-infrastructure-management-suite) ## More software tools ### Qualcomm® AI Inference Suite A comprehensive set of ready-to-use AI applications, agents, tools, and libraries for developing and deploying AI inference on premises or via cloud deployments. [Learn more](https://www.qualcomm.com/developer/software/qualcomm-ai-inference-suite) ### Qualcomm® Cloud AI SDK Optimize trained deep learning models for high-performance inference on Qualcomm Dragonfly AI accelerators. [Learn more](https://www.qualcomm.com/developer/software/cloud-ai-sdk) ### Contact Sales We have an extensive technology portfolio designed to enable products across industries. Our sales team is ready to provide you with tailored assistance. [Contact us](https://www.qualcomm.com/support/contact/forms/data-center) ![](https://s7d1.scene7.com/is/image/dmqualcommprod/Contact-Pattern?fmt=webp-alpha&qlt=85) ## Latest news - [Your guide to the mobile experiences shaping what's next at Snapdragon Summit 2026](https://www.qualcomm.com/news/onq/2026/09/mobile-experiences-snapdragon-summit-2026) — OnQ Blog — Sep 24, 2026 | 5 min read - [Agentic commerce: Enabling agents to understand, anticipate and act](https://www.qualcomm.com/news/onq/2026/09/agentic-commerce-ai-agents-mastercard) — OnQ Blog — Sep 24, 2026 | 7 min read - [Why hearables could become the most natural interface for Personal AI](https://www.qualcomm.com/news/onq/2026/09/personal-ai-hearables-snapdragon-sound) — OnQ Blog — Sep 24, 2026 | 4 min read - [Qualcomm Announces Renewal of Global Patent License Agreement with Apple](https://www.qualcomm.com/news/releases/2026/09/qualcomm-announces-renewal-of-global-patent-license-agreement-wi) — Press Release — Sep 24, 2026 - [Inside Snapdragon Summit 2026: Agentic AI PCs, Googlebooks and Linux are here](https://www.qualcomm.com/news/onq/2026/09/snapdragon-summit-agentic-ai-pcs-linux) — OnQ Blog — Sep 23, 2026 | 7 min read - [Left of the timeline: A new era of proactive healthcare through Personal AI](https://www.qualcomm.com/news/onq/2026/09/personal-ai-proactive-healthcare) — OnQ Blog — Sep 23, 2026 | 4 min read - [Snapdragon for the agentic age](https://www.qualcomm.com/news/onq/2026/09/snapdragon-for-agentic-age) — OnQ Blog — Sep 22, 2026 | 5 min read - [A day in the life with personal AI agents: Intelligence that moves with you](https://www.qualcomm.com/news/onq/2026/09/personal-ai-agents-day-in-life) — OnQ Blog — Sep 16, 2026 | 8 min read - [AI Infra Summit 2026: Building infrastructure for the agentic AI era](https://www.qualcomm.com/news/onq/2026/09/qualcomm-dragonfly-ai-infra-summit-2026) — OnQ Blog — Sep 14, 2026 | 7 min read - [Who owns 6G IP: The enduring value of foundational wireless innovation](https://www.qualcomm.com/news/onq/2026/09/6g-ip-foundational-wireless-innovation) — OnQ Blog — Sep 10, 2026 | 3 min read