Edge AI developer hackathon in Bengaluru: and the winners are...

Every minute counts in a 24-hour hackathon. The GameSense team points to a half-dozen well thought-out steps in their development process:

1. Designing the flow and architecture – They sketched out a system architecture and planned the data flow: a VR cricket game triggers events → events are processed → an LLM generates live commentary → output is converted to speech/audio.
2. Selecting the AI model and text-to-speech (TTS) engine – They turned to the subject matter experts from Qualcomm Technologies to learn which models in the Qualcomm AI Hub can run on the Hexagon NPU. They also had questions about generating NPU-compatible versions of the binaries for the models. Finally, they selected LLaMa 3.2 3B for the SLM and PyTTS for the audio generation.
3. Building the data flow pipeline – To ensure the proper connectivity and data flow among components, they built a simple app in VR without graphics and used it to communicate via the Flask middleware. That passed the data to the AI backend, which generated the audio and handed it off via Flask to the VR.
4. Developing the VR front end in Unity – Once the entire data pipeline was ready, the team started developing the core VR game, the essence of the immersive experience.
5. Incorporating real-time streaming – In parallel, they optimized the data flow to stream data at all layers and ensure nearly real-time performance.
6. Integrating and testing – They integrated all the different components, then tested it extensively and fixed bugs.

Easy parts and hard parts

In their front-end development, the team found that the easiest part was developing the core game mechanics: setting up the entire scene in Unity3D, integrating all components and deploying to VR. Unity made it straightforward to build the environment, connect input controls and handle interactions like bat movement and game event triggers. In back-end development, the easiest part was setting up the Flask middleware.

The part that took the most time and work on the front end was simulating real-world physics, especially getting the bat-ball interaction – force, angle, timing – right. Integrating the gameplay with the scoring system required fine-tuning Unity’s physics engine and adjusting multiple parameters to mimic real cricket kinematics.

On the back end, it took a lot of work to achieve a streaming response with the C++ chat application using the Qualcomm GENIE model. But the judges were impressed, noting that the team used the flow to great effect.

People’s Choice Award: EdgeFit Coach – “We won’t have your data, but we will have your back.”

Remote work, online classes and extended computer usage have led to widespread poor posture, a leading cause of chronic back/neck pain, fatigue and musculoskeletal disorders. But most people lack real-time feedback or awareness when they begin to slouch or sit incorrectly. Even with tools like apps and wearables, it’s hard to muster the consistency and motivation to improve habits.

EdgeFit Coach provides personalized coaching and correction so users can improve their posture while working. It uses edge AI to ensure real-time feedback without delays caused by round trips to/from the cloud. Its cloud-independence means that users benefit even in low- and no-internet environments (e.g., in rural or mobile settings) without the need for specialized hardware. The app selectively uses LLMs for personalized coaching (motivational quotes, analysis), executing the compute-intensive operations on the device.

Most of all, EdgeFit Coach processes all sensitive visuals like posture data and health analytics locally; hence the motto, “We won’t have your data, but we will have your back.”

Team members include Akshat Gosain, Kalash Poddar, Ayush Pareek and Parv Jain from the K.K. Birla Goa Campus of BITS Pilani.

Overview

The Copilot+ PC powered by the Snapdragon X Series processor is ideal for this project. Most prominently, EdgeFit Coach runs its ONNX-based model for pose estimation on the processor’s Hexagon NPU, resulting in 2 to 3x faster inference compared to CPU. The lower power requirement from running on NPU means the app can deliver non-intrusive, persistent posture correction in the background without degrading battery life unduly.

The processor’s image signal processor (ISP) and hardware-accelerated computer vision resulted in faster frame acquisition for quicker posture recognition.

For landmark detection, the hackathon team based the data pipeline on the MediaPipe BlazePose model, converted to ONNX format and optimized for edge inference. That gave them real-time video analysis at more than 13 frames per second with response latency of less than 100 ms — entirely on the device.

They built the coaching with a lightweight, FastAPI back end and LLM integration for surprisingly context-aware motivational quotes, analysis of six useful posture metrics and interactive chat served over WebSockets.

Software and hardware

Language:

• Python – Core application logic, API back end, pose processing, analytics and LLM integration

Environments and frameworks:

• FastAPI + Uvicorn – High-performance REST APIs with async support for back end
• Streamlit – Front-end dashboard and user interaction
• WebSockets library – Real-time motivational quote streaming
• OpenCV + MediaPipe – Real-time pose tracking and image processing
• ONNX runtime – Optimized model inference
• Custom llm_handler.py – Using HTTP requests to local LLM
• Streamlit charts and JSON-based analytics pipeline – For data presentation
• Win10toast – Desktop alerts

Development tools:

• Visual Studio Code – Primary IDE
• Git + GitHub – Version control
• Edge AI Dev Terminal – On the Copilot+ PC for testing NPU inference
• ·         ONNX Optimizer – For converting and simplifying the BlazePose model

Additional hardware:

None. The entire system ran on standard, webcam-equipped laptops, proving that posture monitoring can be done using just software, with inference on CPU/NPU.

Models implemented

For pose estimation, the team used MediaPipe BlazePose (converted to ONNX) with optimizations like operator fusion, float16 quantization and graph simplification. To estimate the distance from user to camera, they used MiDaS-v2.

For context-aware motivational feedback and query response, they integrated an LLM as AI coach and chatbot based on Meta LLaMA 3.2B with a Sarvam-M local back end. The LLM required no fine-tuning, since prompt engineering and context chaining ensured relevance.

They used Qualcomm AI Hub as their source for the BlazePose and MiDaS-v2 models, which integrated easily with their FastAPI + ONNX setup and ran natively on the Hexagon NPU.

Choosing models from the Qualcomm AI Hub helped shorten development and deployment time, and the models imposed little load on the CPU, even with continuous, real-time posture detection and analysis.

Compute cores used

EdgeFit Coach distributed workloads between CPU and NPU.

The CPU ran the FastAPI back end, LLM coaching/feedback and WebSocket server for streaming. It classified and analyzed posture data, served the Streamlit dashboard and handled system-level tasks and desktop notifications.

The NPU performed pose estimation inference using the ONNX-optimized BlazePose model from the Qualcomm AI Hub. It handled all real-time landmark detection from webcam frames at 13 FPS and up.

EdgeFit Coach ran entirely on the edge device, ensuring privacy, low latency and power-efficiency.

Development flow

The team worked out the following steps during the hackathon:

1. Define the problem and design the user flow – Once they had identified posture monitoring as their prime use case, they mapped out the user flow of webcam capture → pose detection → feedback → analytics.
2. Select and optimize models – They chose MediaPipe BlazePose for body landmark detection – specifically, the model in the Qualcomm AI Hub that was quantized and ONNX-optimized for the Hexagon NPU.
3. Develop an edge-centric pipeline – Using Python, they built a pipeline to capture webcam frames (OpenCV), run pose inference (NPU), classify posture and exercises, and stream real-time data (WebSocket) and dashboard metrics (FastAPI). They designed all modules to run on the device, without relying on external resources.
4. Integrate LLM for coaching – They created a modular LLM handler (ask_llm) to power AI-based posture analysis and motivational chat. The handler had the flexibility to use small models locally or route selectively to lightweight APIs when needed.
5. Create front-end and analytics interface – They built a Streamlit dashboard to present metrics like posture trends, slouch frequency and consistency score, and to deliver real-time feedback with motivational quotes and posture reports.
6. Test on Copilot+ PC – Finally, they ran EdgeFit Coach on a Copilot+ PC powered by the Snapdragon X Series processor. Tests showed end-to-end performance of pose detection at more than 13 FPS, response times of less than 100 ms, smooth UI and minimal CPU usage.

Qualcomm Technologies’ experts helped most when it came to model optimization and deployment on NPU. The experts assisted the team in selecting the right ONNX format, enabling NPU acceleration, benchmarking performance and debugging inference behavior.

Easy parts and hard parts

The team found it relatively easy to build the application logic and API system. Using Python, FastAPI and Streamlit, they were able to quickly spin up a back end, create real-time APIs for posture data and chat and put together a working front-end dashboard. Also, once they had the optimized model from Qualcomm AI Hub, they found it easy to plug it into the ONNX runtime with no additional reconfiguration.

The most difficult part lay in optimizing for heterogeneous computing (CPU vs. NPU). The difficulty wasn’t in writing code but in understanding how to make full use of the Snapdragon architecture. The team put considerable effort into making sure the model was actually using the NPU (not CPU fallback), profiling performance bottlenecks, and balancing async processing, UI refresh and inference threading.

They also found it tricky to coordinate the LLM chat with the real-time UI; for example, to keep the AI chat responsive without blocking pose detection or dashboard updates. It became a balancing act among the asynchronous WebSocket updates, the chat history management and the rendering of the front end.

Next steps

Those are the two winners from our Bengaluru Edge AI Developer Hackathon. Of course, we judges voted for GameSense because of the remarkable pipeline from the gameplay all the way to the VR headset without leaving the Copilot+ PC. (And because we like cricket.) Then, we think EdgeFit Coach won People’s Choice because most of us developers know that our posture could use some help.

We want you to bring your developer friends to one of our upcoming Developer events and show us what you’ve got. Read through our FAQ, put together your team and sign up for the next hackathon near you.

Meanwhile, look through the Awesome Qualcomm Developer Projects repo to see how you can build edge AI into your own products. You’ll find projects like GameSense and EdgeFit Coach, and we’ll be posting results and projects from more hackathons there soon.

Connect with us on Discord and be sure to sign up for the Qualcomm Developer newsletter to be notified of our upcoming developer events across the Unites States, Asia and Europe.

Read about Edge AI Developer Hackathon in Paris for more inspiring stories.