The following are sample video frames showing the detection and classification of letters drawn by humans:

The AirLetters dataset has over 161,000 of video-label pairs, covering:

• 36 primary gesture classes (26 Latin letters and 10 digits)
• Two contrasting classes: “doing nothing” and “doing other things”

The data is provided in the following directory structure:

• /airletters
  • /videos: Video files containing the participants' motions
  • test.csv: Test data
  • train.csv: Training data
  • val.csv: Validation data

Each CSV contains the following columns:

• id: Unique data ID
• filename: Name of the video file.
• label: Description of the drawn letter/behavior in the video
• worker_id: ID of the individual who recorded the video
• video_duration: Length of video (in seconds)

A custom platform integrated with crowd-sourcing providers was used to collect our dataset. This platform enabled us to recruit participants from diverse gender, geographical, and ethnic backgrounds and to provide the required instruction and recording functionality.
 

Participants were asked to record themselves performing all 36 gestures in front of their camera. Detailed visual and textual instructions were provided to ensure clear hand visibility, high video quality, and precise gesture execution. Supplementary example videos were provided to demonstrate correct gestures and address text instructions' limitations.
 

After reviewing the guidelines, participants prepared for recording with the help of a countdown timer. Recordings averaged around three seconds, after which participants could review and re-record if necessary. Each participant could make up to three submissions. To encourage scene variability, participants could interrupt recording and resume later.

All submissions were reviewed by human operators to verify accuracy. Participants with mostly correct submissions but minor errors were allowed to make corrections and resubmit. This approach ensured the high quality and consistency of the dataset. Finally, all videos were resized to a width of 640 pixels, maintaining the aspect ratio.

The AirLetters dataset is available for research purposes.

Qualcomm AirLetters Dataset License Agreement - Research Use

If you use this dataset, please cite:

@inproceedings{dagliairletters,
    title={AirLetters: An Open Video Dataset of Characters Drawn in the Air},
    author={Dagli, Rishit and Berger, Guillaume and Materzynska, Joanna and Bax, Ingo and Memisevic, Roland},
    booktitle={European Conference on Computer Vision Workshops},
    year={2024}
}

At Qualcomm AI Research, we are advancing AI to make its core capabilities – perception, reasoning, and action – ubiquitous across devices. Our mission is to make breakthroughs in fundamental AI research and scale them across industries. By bringing together some of the best minds in the field, we’re pushing the boundaries of what’s possible and shaping the future of AI.
 

Qualcomm AI Research continues to invest in and support deep-learning research in computer vision. The publication of the AirLetters dataset for use by the AI research community is one of our many initiatives.
 

Find out more about Qualcomm AI Research.
 

For any questions or technical support, please contact us at research.datasets@qti.qualcomm.com
 

Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc.