Developers: Build enhanced video conference experiences with the Qualcomm QCS8250
This post originally appeared on Qualcomm Developer Network on March 7, 2023.
Advances in video conference technologies let users collaborate across geographies, time zones, and even pandemics. But great experiences happen when every attendee feels immersed in the meeting, and their voices are heard. Today’s average technology often lacks that immersive experience, and issues like disruptive background noise or limited camera views, can make video conferences feel disjointed and impersonal.
In collaboration with Taiwan-based AmTRAN Technology Co, Qualcomm Technologies set out to change that by building a cutting-edge Qualcomm Video Collaboration Platform and Reference Design powered by the Qualcomm QCS8250 application processor.
In this blog post, we learn more about how QCS8250 helped bring about this reference design.
Re-engaging Video Conference Experiences
Online meeting participants can often have less-than-engaging experiences because of a need for richer video and audio capture. For instance, capturing behaviors, like facial expressions and vocal nuances, contributes to the natural feeling of a meeting.
So, we worked with AmTRAN to show the industry how much better video conferences could be. The result: the Video Collaboration Platform and Reference Design, which demonstrates the ability of the QCS8250 to power compute-intensive camera and edge-Artificial Intelligence (AI) applications.
Running on Android, this platform combines a rich set of peripherals with Machine Learning (ML) algorithms to deliver enhanced, director-like video productions of meetings – all in real-time. Through the reference design, developers can see how the premium features of the QCS8250 could work and also have a ready-to-use platform to evaluate and quickly develop their own video conference devices.
To accomplish all of this, a suite of video collaboration AI features, including group framing, speaker tracking, and AI-based echo cancellation noise suppression (ECNS).
Powering these features is the onboard Qualcomm Hexagon DSP AI accelerator that runs the device’s video director, which tracks people in the room from three different cameras (ultrawide, wide, and telephoto). Together, these cameras cover everybody in a meeting room, even those furthest away from the camera. The reference design can automatically zoom in or out on participants and crop areas where nobody is sitting. Its crop and zoom (aka group framing) feature then centers the stream around those present at the meeting. Settings such as tracking speed and zooming can also be adjusted.
The rendered video is very clear, thanks to three 4K ultra-HD HDMI outputs, each able to display different content such as a view of the primary speaker, the presentation content, and a gallery view of all meeting participants.
On the audio side, speaker framing can zoom in as someone speaks. Multiple microphones work in concert to detect where a person is relative to the camera, and onboard algorithms determine the audio’s direction of arrival. When the speaker stops talking, on-board AI will automatically zoom out to capture the optimal view of all the meeting room participants. Additional ML-powered audio processing features include background noise suppression and echo cancellation so those on the other end of the meeting can clearly hear everyone in the meeting room.
To look the part, the enclosure’s design represents a finished video collaboration bar that can be wall mounted or placed at the end of a conference table. Internally, the QCS8250 is hosted on a system-on-module, which can be easily replaced with a different unit should developers wish to use a different application processor.
Additional Considerations and Use Cases
In addition to technical details, developers should consider certification with unified communications providers like Microsoft Teams and Zoom. To create an optimal product, developers should endeavor to exceed standard-level certification KPIs. Pre-qualification testing has been performed on the reference design to help speed up the time needed by developers to submit their own devices for certification.
In the end though, it’s all about the possibilities and opportunities the platform unlocks. The technologies demonstrated in the reference design can also be applied to use cases spanning enterprise to consumer. For example:
- Adding video chat capabilities to TVs
- Using multiple cameras for 360-degree security views
- Redirecting cameras to the source of a sudden loud noise
- Evaluating the poses of participants in a remote yoga class
Moving Forward
The platform and reference design provide a solid foundation upon which OEMs can differentiate their products and gain a faster time-to-market thanks to its mainline board support package. In addition, its underlying technologies allow for improved hybrid meetings, where everyone – remote or in-person – feels more involved in the video meeting. Customers and ISVs can build on the reference design to add their own value.
We expect remote and on-premise experiences to converge as video conferencing technologies evolve. We’d love to see your creativity in creating better user experiences with the Qualcomm Video Collaboration Reference Design.
Ready to Get Started?
For additional information, check out the following resources:
- QualcommVideo Collaboration Reference Design
- AmTRAN Reference Design (see the Support tab for the user manual)
