OnQ Blog

Developers: New projects for video capture/playback and AI on QCS610

Jun 2, 2021

Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.

Start and stop video recording through an Azure IoT hub.

Send the status of your camera recordings to the AWS IoT console.

Use GStreamer plug-ins to record [email protected], [email protected] and 1080p video streams via TCP.

Detect a scene and use a pretrained AI model to classify it into different scenes.

Those are just a few of the recent projects we’ve posted to QDN to introduce you to the Qualcomm QCS610 SoC. The QCS610 is an application processor designed for the high performance and power efficiency needed to run artificial intelligence, video and audio at the network edge. It’s available for prototyping and testing in several QCS610 development kits and SoMs.

If you’re building applications for industrial IoT, enterprise security, home IP cameras, dash cams, body cams, smart displays or videoconferencing, these four projects are a good way to get started. They revolve around the TurboX C610 development board from Thundercomm, and they include a list of required equipment, additional resources, project dependencies and prerequisites, along with step-by-step instructions.

1. Video recording and playback over TCP using GStreamer

The GStreamer plug-in is at the heart of video capture and playback in this project. GStreamer is a framework for creating streaming media, and GStreamer plug-ins from Qualcomm Technologies, Inc. (QTI) provide access to video and camera subsystems using our multimedia framework. This project uses elements of the QTI plug-in for capturing 4K and 1080p video frames on the TurboX C610 board. Once you’ve built the application, use it in three different cases: recording 4k resolution video at 30fps, recording [email protected] video and streaming [email protected] video over TCP. See more details on the GStreamer video project.

2. Video capture and image classification

This project uses QTI GStreamer plug-in in another way: not only to capture video but also to run inference on the TurboX C610 board. You convert a TensorFlow or Keras model into either TensorFlow Lite (TFLite) format or, for the Qualcomm Neural Processing Engine for AI, into DLC format. Then you build and run the application to capture video from the camera feed and use the model to perform image classification on ten different indoor/outdoor activities. Use the qtioverlay plug-in to display output metadata from the model on a video stream. Take a look at the video capture and image classification project.

3. Managing camera recording through an Azure IoT hub

This project integrates the Azure IoT SDK source into the TurboX C610 board. You then build an application to turn the board’s camera on and off and post the status of camera recording to an Azure IoT hub. Once the development board receives the command, it starts or stops streaming the video. It also sends the camera status to the Azure IoT hub and the status message displays in an event monitoring terminal. A player like VLC lets you watch the live stream through a new terminal in the host system. Have a look at the Azure IoT hub video project.

4. Post the status of camera recordings to the AWS IoT console

This project shows you how to build an application to post the status of camera recordings to the AWS IoT console. It uses the same TurboX C610 board and integrates the C++/Python AWS IoT SDK source. Use the application you build to save video in MP4 format at either 4K or 1080p resolution, or stream it over TCP. Whenever the camera application starts or stops capturing, it connects to AWS IoT Core and publishes the recording format, device details and current camera status in JSON. You can watch the live stream with a player like VLC on the host system. Here are full details on the AWS IoT video project.

Next step

Those QCS610 projects are just a few of the dozens of projects we’ve published. If you’re looking for a way to evaluate and tinker with hardware or software tools built around our technologies, visit our Projects page. You can vet projects by software tools, operating system, cloud service, skill level and area of focus.

Dig in and build a better mousetrap (or Chicken Sound Box). Once you’ve done it, let us know how you did it. We’ll gladly publish your project too.

Qualcomm QCS610 and Qualcomm Neural Processing Engine are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

 

Engage with us on

and

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries ("Qualcomm"). Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.

Ramya Kanthi Polisetti

Applications Engineer, Qualcomm Technologies

©2021 Qualcomm Technologies, Inc. and/or its affiliated companies.

References to "Qualcomm" may mean Qualcomm Incorporated, or subsidiaries or business units within the Qualcomm corporate structure, as applicable.

Qualcomm Incorporated includes Qualcomm's licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm's engineering, research and development functions, and substantially all of its products and services businesses. Qualcomm products referenced on this page are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

Materials that are as of a specific date, including but not limited to press releases, presentations, blog posts and webcasts, may have been superseded by subsequent events or disclosures.

Nothing in these materials is an offer to sell any of the components or devices referenced herein.