Ollama serves as a simple yet powerful runtime that brings LLM models to local environments. Its compatibility with Windows on Snapdragon (WoS) makes it ideal for developers building AI-powered applications for mobile and edge devices.
This project is designed to:
- Demonstrate setting up Ollama locally on a Snapdragon-powered Windows device.
- Run DeepSeek-R1 models entirely on the CPU.
- Integrate Ollama with the AnythingLLM application using Docker, ensuring the local service is correctly configured and operational.
- The entire setup is executed on Qualcomm® Device Cloud (DC) & on Windows on Snapdragon (WOS) Laptops validating its feasibility for developers working on Snapdragon and Qualcomm® platforms hardware.
This hands-on step by step instructions give developers a complete pipeline—from model setup to application-level integration—showcasing the power of running large language models locally on efficient platforms.
Materials Required / Parts List / Tools
|
GitHub links |
https://github.com/Mintplex-Labs/anything-llm?tab=readme-ov-file |
|
Qualcomm Device Cloud |
|
|
Developer Blog |
Blog: Ollama tutorial. |
Build / Assembly Instructions
1. Setup (10 minutes)
- Hardware
- An Ubuntu/Windows Host system – on Qualcomm Device Cloud.
OR
- Windows laptop with below specifications – on Device.
- Machine: Any Windows on Snapdragon Support device
- Chip: Snapdragon X-Elite
- OS: Windows 11
- Memory: 8GB or higher
- Software and library installation
- Docker Desktop for Windows –ARM https://docs.docker.com/desktop/setup/install/windows-install/
- WSL
- Development environment setup
- Qualcomm DC account https://qdc.qualcomm.com/
- Install Anything LLM https://github.com/Mintplex-Labs/anything-llm?tab=readme-ov-file
- Download & Install Ollama https://ollama.com
- Access to an Ollama running locally or remotely.
2. Deployment options
Users have two options for implementing this project:
- Using Qualcomm® Device Cloud (DC):
- Users opting for Qualcomm DC must First complete the Qualcomm DC setup for Snapdragon X-Elite (section 4). After setting up, proceed to Build the Application (section 5).
- Using Real Device with Snapdragon X-Elite:
- Users with a physical windows device powered by Snapdragon X-Elite can directly begin from the Build the Application.
Project Walkthrough
Qualcomm DC setup for laptop with Snapdragon X-Elite
- Visit https://qdc.qualcomm.com/ and sign up for an account to create a new session.
- In the home page click on New Interactive Session. We recommend using Remote Desktop instead of web browser to interact with Compute devices.
3. In Select a Platform section, select Compute option and click Next
4. In Select Device Section, select Snapdragon X Elite and fill the mandatory fields to Unlock your free 1000 minutes and click on Next.
After utilizing the initial 1000 free minutes, developers have the option to request additional free minutes.
5. In Device Configuration check the toggle button to install SNPE and configure another Qualcomm
build and click on Next.
6. In Test Configuration fill in the mandatory fields in Session details like Session name according to your use case. Can choose Maximum minutes per device is 420 minutes (7 hours). Choose the mode of operation out of three modes and follow the appropriate guide provided for each mode. For this use case we have selected screen mirroring only.
7. There is a provision of Package Provider to upload assets in zip format. Zip will be extracted and pushed to C:\Temp\file\ before the session is started. Once you are done, it will enable Create Session tab, click on it and continue.
8. The process will take some time to open the window, and you are ready to access cloud device Snapdragon X-Elite.
Build the Application (20 minutes)
1. On device with Snapdragon X-Elite install Docker from the link provided in the prerequisite section refer to the below images as reference.
wsl –update
2. Once Docker and wsl have been installed. Pull the latest image from docker.
docker pull mintplexlabs/anythingllm
3. Mount the storage locally
# Run this in powershell terminal
$env:STORAGE_LOCATION="$HOME\Documents\anythingllm"; `
If(!(Test-Path $env:STORAGE_LOCATION)) {New-Item $env:STORAGE_LOCATION -ItemType Directory}; `
If(!(Test-Path "$env:STORAGE_LOCATION\.env")) {New-Item "$env:STORAGE_LOCATION\.env" -ItemType File}; `
4. Run AnythingLLM in Docker (Keep this running in one terminal)
docker run -d -p 3001:3001 `
--cap-add SYS_ADMIN `
-v "$env:STORAGE_LOCATION`:/app/server/storage" `
-v "$env:STORAGE_LOCATION\.env:/app/server/.env" `
-e STORAGE_DIR="/app/server/storage" `
mintplexlabs/anythingllm;
5. Visit the ollama website.
- Open your browser and go to https://ollama.com.
- Download & install. Click the Download button, select the Windows installer and follow the prompts to complete the installation.
Run your model locally. Open a command prompt or terminal and enter the following command:
ollama run <model_name>Select any Deepseek model
ollama run deepseek-r1:1.5b
ollama run deepseek-r1:7b
ollama run deepseek-r1:8b
ollama run llama3.2
Similarly, you can run other models like Phi-3, Mistral, and Gemma using similar commands - which are available in the Ollama model registry.
List of models available on https://ollama.com/library
Usage
1. To check if Ollama is running, visit http://localhost:11434 in your browser. If Ollama is running on the host machine and you're connecting from within a Docker container (e.g., AnythingLLM), use http://host.docker.internal:11434 as the connection URL in the AnythingLLM settings.
2. Visit http://localhost:3001 in your browser & you are now using AnythingLLM!
- In LLM preference – select Ollama
- In Ollama base URL – enter the URL http://host.docker.internal:11434 where Ollama is running.
- Complete the user setup.
- Create your First Workspace – by providing your workspace name.
- Click on your workspace name and start your conversation.
Snapdragon and Qualcomm branded products are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
