OnQ Blog

Bringing AI at the Edge to smart cameras on the IoT

Jul 7, 2020

Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.

Artificial intelligence, edge computing, and high resolution are already in the smart cameras of premium-tier devices in the home and enterprise. From morning to night, we depend on AI-driven activities around us thanks to smart cameras powering devices as varied as robots and connected cars.

But what about devices outside of these premium price points? How do you build those kinds of features and performance into cameras that meet the needs of the Internet of Things and future market segments?

The Qualcomm Vision Intelligence Platform spans a full range of system-on-chips (SoCs) designed for running compute-intensive workloads at the edge of the network. The platform is engineered with an eye to IoT devices across consumer and enterprise segments. Chips on the Qualcomm Vision Intelligence Platform allow designers of IoT devices to keep workloads like computer vision, image processing, machine learning, and security on the device. AI on the edge frees manufacturers to focus on IoT innovation without worrying about user privacy and network latency.

Premium camera technology and AI at the edge for more camera segments

We recently launched the Qualcomm QCS610  and Qualcomm QCS410, SoCs for AI inference at the edge with improved performance compared to previous generations.

These latest additions to the Qualcomm Vision Intelligence Platform make advanced AI available in segments where it was previously only available at a premium. That means supporting commonly used surveillance cameras, dashboard cameras, and video conferencing devices with applications like object classification, segmentation, license plate recognition, face recognition, body detection, and people counting. The chips are designed to offer the potential for new product ideas based on the power of AI inference at the edge.

The QCS610 and QCS410 follow Qualcomm Technologies’ paradigm of combining hardware and software for heterogeneous computing. They run the Qualcomm AI Engine, built on the hardware of the Qualcomm® Hexagon DSP Vector Processor and the Qualcomm Adreno GPU. For connectivity, the QCS610 and QCS410 include integrated Wi-Fi (802.11a/b/g/n/ac) and Bluetooth 5.x radios, along with interfaces to support 5G.

Software completes the heterogeneous paradigm. The Qualcomm Neural Processing SDK for AI is designed to provide tools for model conversion and execution, plus APIs for matching computing core (CPU/GPU/DSP) to the desired performance profile. TensorFlow Lite has released a Hexagon delegate software that can accelerate inferencing on the hundreds of millions of Android devices with Hexagon DSPs.

The result is greater efficiency and more computing power that is designed to enable AI inferencing at the network edge. Keeping the workloads on the device also resolves the cloud computing challenges of privacy, latency, connection, and cost.

Use cases for AI at the edge

With the QCS610 and QCS410, manufacturers can design new devices and improve existing ones with AI and edge computing. Consider these use cases:

Map projection

Pilot AI has developed a suite of algorithms that move AI inference workloads from the cloud to the edge device. That provides a private, secure and fast way to make decisions close to the source of the data.

The use case is becoming more relevant in places like retail stores, factories, buildings, and offices. With the QCS610 and QCS410, Pilot AI can follow movement and aggregate inputs from multiple smart cameras in a 3D space onto a 2D map, as shown below.

One timely application of this technology is in addressing the current coronavirus pandemic and any public health crisis yet to come. AI cameras and wireless have a role to play in measuring social distancing and managing footfall, which remain imperative for public and private entities.

By mapping out traffic, the managers of an office, retail store or manufacturing plant can determine how close people are to one another and provide social distancing alerts. That is especially useful in areas like cafeterias, hallways or doorways, where people naturally gather. The ability to trace the path of a person with suspected elevated body temperature can help manage affected areas. Analytics generated by Pilot AI can be used for sending real-time alerts and, in the longer term, for assisting companies in rearranging floor plans based on where people tend to congregate.

Dual-facing AI dash cam

AI dash cams help companies improve the safety, security and visibility of their fleets. Dashboard-mounted cameras use edge processing for real-time event and object detection, with the goal of reducing vehicle accident rates.

A dual-facing dash cam uses AI and edge computing to analyze driver behavior (paying attention, dozing off, taking eyes off the road) and street conditions in real time, lowering the risk of accidents.

AI dash cams can also detect events and send alerts in case the vehicle operator is driving distractedly, running a red light or rolling through a stop sign. As part of a platform for fleet management and driver safety, the dash cam records and stores footage around events such as sudden accelerations, unexpected braking and collisions. The device can automatically upload footage to the cloud for later viewing.

COVID recovery

As cities and entire countries ease out of lockdown into a cautious reopening, it becomes important to monitor at points of entry to schools, offices and other venues. To meet that need, companies are developing body temperature and facial recognition scanners, like OneScreen’s GoSafe, that use the Qualcomm AI Engine.

In this use case for smart buildings, devices are designed to screen people who enter a building and control access. If visitors have an elevated temperature or if they’re not wearing a mask, GoSafe can detect it. The device can notify a human monitor to prevent them from entering or, if connected to a building security system, it can automatically grant/deny access.

Advanced AI in a wide range of devices

Moving AI workloads from the cloud to IoT devices is the next step in computing at the network edge. When ordinary, camera-driven devices deliver more inferences per second, innovative applications are bound to proliferate. 

The Qualcomm Vision Intelligence Platform is the ideal point of departure to develop and prototype smart cameras for AI at the edge. Current ecosystem partners include Altek, ArroweInfochips, LantronixThundercomm, and more. Watch for Qualcomm Technologies to continually develop breakthroughs like the Qualcomm Vision Intelligence Platform that enable our customers to deliver the most innovative and compelling products and services possible.

 

Qualcomm QCS610, Qualcomm QCS410, Qualcomm Hexagon, Qualcomm Adreno, Qualcomm Neural Processing Engine, Qualcomm AI Engine and Qualcomm Vision Intelligence Platform are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

 

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries ("Qualcomm"). Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.

Winnie Bekmanis

Staff Manager, Product Marketing, Qualcomm Technologies

Related News

©2020 Qualcomm Technologies, Inc. and/or its affiliated companies.

References to "Qualcomm" may mean Qualcomm Incorporated, or subsidiaries or business units within the Qualcomm corporate structure, as applicable.

Qualcomm Incorporated includes Qualcomm's licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm's engineering, research and development functions, and substantially all of its products and services businesses. Qualcomm products referenced on this page are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

Materials that are as of a specific date, including but not limited to press releases, presentations, blog posts and webcasts, may have been superseded by subsequent events or disclosures.

Nothing in these materials is an offer to sell any of the components or devices referenced herein.