On this Page
Introduction
License
Community
Get the Qualcomm® newsletter straight to your inbox.
Qualcomm AI Research presents the Qualcomm Exercise Video Dataset (QEVD) which explores human-AI interaction in the challenging real-world domain of fitness coaching – a task which intrinsically requires monitoring live user activity and providing timely feedback. Our dataset includes corrective feedbacks to address potential user mistakes and steer them towards successful workout completion.
Our dataset contains 474+ hours of videos and includes the following:
- short-clip videos (∼5 seconds in length) annotated with 1M+ question-answer pairs
- short-clip videos (∼5 seconds in length) annotated with 650k+ live feedbacks including corrective feedbacks
- long-range videos (>3 minutes in length) annotated with 7.5k+ live feedbacks including corrective feedbacks
Our Qualcomm Exercise Video Dataset contains three subsets:
- QEVD-Fit-300k (fine-grained short videos)
- The QEVD FIT 300K dataset includes 289k video clips between 2 and 10 seconds long of humans performing pre-defined exercises including specific variations of each. The dataset also includes 14k video clips of everyday human activities relevant in a fitness context. The dataset is designed for training machine learning models for fine-grained human motion understanding in the fitness domain.
- They cover 148 different exercises and their variations including: varied pacing, performing common mistakes, and modified form.
- Approx. 10 variations were collected per exercise.
- QEVD-Fit-Coach
- The QEVD FIT-COACH Dataset includes 149 workout videos of humans performing a structured workout consisting of 4-6 exercises.
- Each video is ~3.5 minutes long and is annotated with feedbacks from the perspective of a fitness coach.
- This dataset also includes additional annotations for the QEVD-FIT-300K dataset including:
- feedbacks from a fitness coach perspective
- high level questions (e.g., What exercise is the user doing?)
- fine-grained questions (e.g., Is the user moving their arms?). The dataset is designed for training machine learning models for interactive fitness coaching.
- QEVD-Fit-Coach-Benchmark
- The QEVD FIT-COACH Benchmark includes 74 workout videos of humans performing a structured workout consisting of 4-6 exercises.
- Note: These videos contain distinct participants from the QEVD-FIT-Coach Dataset.
- Each video is ~3.5 minutes long and is annotated with feedbacks from the perspective of a fitness coach.
- The benchmark is designed for testing machine learning models for interactive fitness coaching.
Classes
- QEVD-FIT-300K Classes
Example of exercise class and fine-grained attributes
| Exercise | Fine-grained Attributes |
| Arms moving too much | arms flapping too much flapping heels only height=1 height=2 height=3 height=4 height=5 no jump no obvious issue no wrist circles not moving side to side speed=0.75 rps speed=1.00 rps speed=1.25 rps speed=1.50 rps speed=1.75 rps speed=2.0 rps |
Example of General actions
| General Actions | |
|---|---|
| bobbing head (imagine there is music) | neck-warm-up (without hands) |
| boxing bounce-steps | nodding head to say yes (long) |
| catching your breach (crouching) | nodding head to say yes (short) |
| catching your breath (hand on knees) |
open and drink from a |
| catching your breath (hands behind head) | picking up the camera |
| catching your breath (leaning on something) | plank preparation |
| catching your breath (walking around) | pretending to towel off sweat (without using a towel) |
| changing the webcam view while lying down | scratching arm |
| clapping hands (long) | scratching back of the head |
| clapping hands (short) | shaking head to say no (long) |
| coming closer to the webcam | shaking head to say no (short) |
| crouching | shoulder swipe |
| drinking something from a bottle | shoulder warm-up |
| falling over | shrugging (long) |
| feet apart | shrugging (short) |
| fist bump (hold) | sitting down |
| fist bump (preparation and hold) | sitting on a chair |
| first bump (quick) | small kicks while waiting |
| fixing hair (long, both hands) | standing up |
| fixing hair (long, one hand) | step feet together |
| fixing hair (short, one hand) | stretching arms |
| give up gesture | thumb down (hold) |
| going down on knees | thumb down (preparation and hold) |
| grabbing a bottle (bottle visible from the start) | thumb down (quick) |
| grabbing a towel (towel visible from the start) | thumb up (hold) |
| grabbing an off-screen bottle | thumb up (preparation and hold) |
| grabbing an off-screen towel | thumb up (quick) |
| high five (hold) | using towel to remove sweat |
| high five (preparation and hold) | walking towards the webcam |
| high five (quick) | waving (hold) |
| jump feet together | waving (preparation and hold) |
| keeping hands in pockets | waving (quick) |
| leaving plank position | wiping face sweat on shirt |
| lying down after push-up | yawning (covering mouth with hand) |
| lying down in random position | yawning (long) |
| neck warm-up (with hands) | yawning (short) |
Dataset Citation Instructions
The dataset is intended for research purposes only. Please cite our paper if you use this dataset in your research:
Panchal, S., Bhattacharyya, A., Berger, G., Mercier, A., Böhm, C., Dietrichkeit, F., Pourreza, R., Li, X., Maden, P., Lee, M., Todorovich, M., Bax, I., Memisevic, R. (2024) Live Fitness Coaching as a Testbed for Situated Interaction
Dataset license
This dataset is intended for research purposes only and to support and contribute to the graph research community. The quality of the configuration space design and the collected execution times may be suboptimal and should not be considered as reference performances of the target device but rather as representative of the problem at hand for research purposes.
Qualcomm AI Research
At Qualcomm AI Research, we are advancing AI to make its core capabilities – perception, reasoning, and action – ubiquitous across devices. Our mission is to make breakthroughs in fundamental AI research and scale them across industries. By bringing together some of the best minds in the field, we’re pushing the boundaries of what’s possible and shaping the future of AI.
Find out more about Qualcomm AI Research.
For any questions or technical support, please contact us at [email protected]
Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc.
Connect with our communities
Stay ahead of the curve
Receive the latest updates, exclusive offers, and valuable insights delivered through the Qualcomm newsletter straight to your inbox.
Stay ahead of the curve
Receive the latest updates, exclusive offers, and valuable insights delivered through the Qualcomm newsletter straight to your inbox.
