Qualcomm Snapdragon powering new mobile experiences on Snap [video]

Snapchat needs no introduction. It has become a truly global phenomenon thanks to its 218 million daily users creating over 3.5 billion Snaps per day, making it arguably the most widely-used social camera experience in the world.

The device camera is essential to the Snapchat experience, and the lenses filter creates a fun and engaging experience with friends and family. These lenses see and track your face, then apply various special effects called “style transferring.” Using this effect, users can see their faces morphed in real time on screen. The lenses filter is based on artificial Intelligence, and the inferencing performance of “neural nets” play a critical role in how smooth the video experience is in preview mode. In order for the video to qualify as smooth video (without lag), the frame rate should be at least 30 frames per second (FPS). That wasn’t always the case for this particular Snapchat filter, as the frame rate wasn’t optimal processing on the CPU.

Qualcomm Technologies debuted the Qualcomm Snapdragon 865 Mobile Platform at its annual Tech Summit in Hawaii in December 2019, and with it a new Qualcomm AI Engine featuring a beefed-up Qualcomm Hexagon digital signal processor. By accelerating the AI neural networks on the Hexagon processor, video capture sped up four times faster compared to processing on the CPU and 2.2 times faster compared to the GPU. This means this experience was able to hit “smooth video” at 30 FPS.  The science behind all of this is the Snapchat application accessing the Hexagon processor with a tool called Hexagon NN Direct, which allows Snapchat’s runtime and framework to directly access our specialized Qualcomm Technologies library and accelerate the Snapchat filters on the Hexagon hardware. You can see the difference in the video below. On the left, the video is being run on a CPU-based filter and on the right, it is being accelerated on Hexagon. Notice the smoothness of the filter that can critically enhance the user experience.

This is an early and compelling collaboration that Qualcomm Technologies has participated in with another leading tech company to help drive next-generation artificial intelligence experiences with the Snapdragon 865 mobile platform.