Deploy AI Models on Snapdragon X Elite with Qualcomm AI Hub
Sign up for Developer monthly newsletter
Join thousands of developers around the globe who receive latest news and updates from our monthly curated newsletter.
Sign up
Come for support, stay for the community
Get support from experts, connect with like-minded developers, and access exclusive virtual events.
Join Developer DiscordUsing Qualcomm AI Hub, in just minutes, developers can convert any model to an ONNX model that can run on the NPUs (neural processing units) in Snapdragon® platforms.
You can either leverage a model from our collection, Qualcomm AI Hub Models, that we’ve tested across various devices or, if you already have a pretrained model for your use case, you can bring your own model (BYOM) and leverage the Qualcomm AI Hub Platform.
The Qualcomm AI Hub optimizes and compiles your model, providing a downloadable target model to bundle into your app and run on your desired edge device. Through Qualcomm AI Hub you can test your model ahead of deploying by running inference on real devices to see key performance metrics (latency, memory, compute unit) and confirm numerical accuracy with your dataset.
1. Convert your model to ONNX Runtime
Choose a Pre-Optimized Model
Browse and download 175+ popular pre-optimized ONNX models from Qualcomm AI Hub Models, or export from PyTorch to ONNX yourself via Qualcomm AI Hub Models GitHub Repository:
# Install AI Hub Models
pip install qai_hub_models[whisper_base_en]
# Compile Whisper Base to ONNX and profile on a Snapdragon X Elite laptop
python -m qai_hub_models.models.whisper_base_en.export --target-runtime onnx --device "Snapdragon X Elite CRD”
Bring Your Own Model
Use Qualcomm AI Hub API to convert your own PyTorch model to ONNX Runtime.
1. Install requirements:
pip install torch torchvision numpy qai-hub2. Export your model:
import numpy as np
import torch
import torchvision
import qai_hub as hub
# Using pre-trained MobileNet
torch_model = torchvision.models.mobilenet_v2(pretrained=True)
torch_model.eval()
# Trace model
model_input_spec = {"image": ((1, 3, 224, 224), "float32")}
sample_inputs = [torch.rand(model_input_spec["image"][0], dtype=torch.float32)]
traced_model = torch.jit.trace(torch_model.to("cpu"), sample_inputs)
# Export trace to ONNX
compile_job = hub.submit_compile_job(
model=traced_model,
device=hub.Device("Snapdragon X Elite CRD"),
input_specs=model_input_spec,
options="--target_runtime onnx --output_names classes"
)
# Download compiled model to disk
compile_job.get_target_model().download("mobilenet_v2")
# Profile model on a cloud-hosted Snapdragon X Elite Device
profile_job = hub.submit_profile_job(
model=compile_job.get_target_model(),
device=hub.Device("Snapdragon X Elite CRD"),
)
# Run inference on model on a cloud-hosted Snapdragon X Elite Device
inference_job = hub.submit_inference_job(
model=compile_job.get_target_model(),
device=hub.Device("Snapdragon X Elite CRD"),
inputs={"image": [np.asarray(sample_inputs[0])]}
)
# Download on-device inference outputs
# {“classes”: [output_numpy_array]}
outputs = inference_job.download_output_data()
2. Execute an ONNX Model on Qualcomm® Hexagon™ DSP
AI Hub provides several pre-written demos that target Hexagon DSP on Windows.
Browse Python Demos
Python demos are available for:
Browse Native Demos
Native (C++) demos are available for several model types:
Run your Model in Python on Qualcomm Hexagon DSP
On your Windows on Snapdragon device, follow the steps outlined in our example notebook.
Your turn
With the above steps, you compiled your model for Snapdragon X Elite and ran it on device in just minutes. For next steps, browse our website for more models to try, or join our Slack community to ask questions and chat with others building AI experiences on Snapdragon platforms.
