Generative AI/LLMs
Get started with Amazon EC2 DL2q instances powered by Qualcomm AI 100 Standard accelerators.
What is
Cloud AI SDK
Cloud AI SDKs enable developers to optimize trained deep learning models for high-performance inference on Qualcomm® Cloud AI accelerators.
The Cloud AI SDKs provide workflows to optimize the models for best performance, provides runtime for execution and supports integration with ONNX Runtime and Triton Inference Server for deployment.
Cloud Providers
Server Providers
Software architecture
Application (Apps) SDK
The Apps SDK is used to convert models and prepare runtime binaries for Cloud AI platforms. It contains model development tools, a sophisticated parallelizing graph compiler, performance and integration tools, and code samples. Apps SDK is supported on x86-64 Linux-based systems.
Platform SDK
The Platform SDK provides driver support for Cloud AI acclerators, APIs and tools for executing and debugging model binaries, and tools for card health, monitoring and telemetry. Platform SDK consists of a kernel driver, userspace runtime with APIs and language bindings, and card firmware. Platform SDK is supported on x86-64 and ARM64 hosts.
Use cases
Build your models to support any AI use case
Natural Language Processing
Object Detection
Semantic Segmentation
Traffic Detection
Quality Control
Character Recognition
Search Engines
Benefits
Flexible toolchain to meet the growing inference needs of cloud data centers.
Running multiple complex AI models
High performance Generative AI, Natural Language Processing, and Computer Vision models.
Optimizing performance
Optimizing performance of the models per application requirements (throughput, accuracy and latency) through various quantization techniques.
Cross platforms apps development
Development of inference applications through support for multiple OS and Docker containers.
Deployment at-scale
Deployment of inference applications at scale with support for Triton inference server.
Power efficiency
Sets a higher bar for power efficient inference processing delivering the highest performance/watt.
Get started with
AI model recipes
Tutorial Videos
Stable Diffusion Notebook
This demo showcases Stable Diffusion running on Qualcomm Cloud AI 100. The demo highlights how AI cores are designed for AI inference acceleration, and how app developers can use python integration to quickly build apps that can generate detailed images from text prompts.
Stable Diffusion Demo
This video will walk you through a Stable Diffusion Notebook on Qualcomm Cloud AI 100. First, we’ll introduce you to Stable Diffusion and identify the resource intensive processes that needs to run on the optimized hardware. You will also learn how you can use high level python API’s in the Notebook. Finally, you will see the comparison between the Image generation on CPU vs the Qualcomm Cloud AI 100.
CodeGen Notebook
This video will walk you through a Codegen Notebook on Qualcomm Cloud AI 100. Familiarize yourself with Codegen text-to-code model and learn how you can use high level python API’s with the Codegen Notebook.
Cloud AI SDK platform in the cloud.
Cloud AI SDK on-premises
Latest news
Snapdragon and Qualcomm branded products are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
