Building on our technology collaboration with AWS, the Qualcomm Cloud AI 100 launch marked the first major milestone in the company’s joint efforts with the general availability of new Amazon Elastic Compute Cloud (Amazon EC2) DL2q instances. The Amazon EC2 DL2q instances serve as the first instances to bring the Qualcomm artificial intelligence (AI) solution to the cloud.

With its flexible and scalable multi-core architecture, the Cloud AI 100 accelerator supports a wide range of use-cases spanning:

• Generative AI and Large Language Models (LLMs): Covering productivity and creativity use cases with support to models with up to 16B parameters on single card and 8x that in one DL2q instance, and
• Classic AI: Including natural language processing and Computer vision.

At this year’s AWS re:Invent 2023 we recently demonstrated diverse applications employing AWS EC DL2q powered by Cloud AI 100:

• An AI chatbot using the Llama2 7B parameter LLM model.
• Text-to-image generation using the Stable Diffusion model.
• Transcribing multiple audio streams simultaneously using Whisper Lite model.
• Translating multiple languages using Opus transformer-based mode.

“Working with AWS is empowering us to build on our established industry leadership in high-performance, low-power deep learning inference acceleration technology,” said Nakul Duggal, SVP & GM, Automotive & Cloud Computing at Qualcomm Technologies, Inc. “Our work to date demonstrates the strong potential in integrating cloud technologies into software development and deployment cycles.

A cost-effective AI revolution

The Amazon EC2 DL2q instance enables EC2 customers to run inference on a broad range of models with best-in-class performance-per-total cost of ownership (TCO). For example:

• Up to 50% better price-performance for (DL inference models — compared to current-generation graphics processing unit (GPU)-based Amazon EC2 instances.
• More than three times reduction in Inference cards with CV-based security, leading to much lower-cost system solution.
• Enabling two and a half smaller models such Deci.ai models optimized on Cloud AI100.

The DL2q instance features the Qualcomm AI Stack which delivers a consistent developer experience across Qualcomm AI in the cloud and other Qualcomm products.

The same Qualcomm AI Stack and base AI technology runs on the DL2q instances and Qualcomm edge devices, enabling customers to enjoy a consistent developer experience, with a unified application programming interface (API) across their:

• cloud,
• automotive,
• PC,
• extended reality, and
• smartphone development environments.

Customers can use the AWS Deep Learning AMI (DLAMI), which comes prepackaged with Qualcomm’s Software Development Kits (SDK) and popular machine learning frameworks, such as PyTorch and TensorFlow.

For more information, please visit Qualcomm Cloud AI100.
 

To get started using DL2q instances, visit: