Talk is cheap. If you’ve been following my series of blogs about mobile heterogeneous computing, you’ll see I’ve talked about the importance of it for enabling breakthrough experiences and how Qualcomm Technologies offers a comprehensive solution. Rather than continue to just talk about it, I wanted show some real-world examples of heterogeneous computing in action.

Qualcomm Technologies looked at this issue in a recent webinar, in which third-party companies illustrated how they were able to deliver breakthrough mobile experiences at low power by taking advantage of heterogeneous computing on the Snapdragon processor. The three companies that presented were Pelican Imaging, MuseAmi, and ArcSoft. Besides explaining the application, each company emphasized how it took advantage of the compute capabilities of the programmable processing engines (the CPU, GPU, and DSP) on Snapdragon processors. (The following is based on their respective presentations.)

Pelican Imaging: Computational camera with depth-enabled imaging

Pelican Imaging spoke about all the great experiences possible with depth-enabled imaging by using an array camera. Their array camera generates 16 low resolution images and combines them into a super resolution image along with a depth map by performing complex computation. Having the depth map enables many interesting experiences such as image refocus, selective filtering and image segmentation, and the unique capability of measuring the distance to any object in the photo or video.

The imaging algorithms Pelican Imaging developed to enable these experiences specifically run on the CPU, GPU and DSP. By using these diverse processing engines, Pelican Imaging claimed up to 100x performance improvement and up to 10x power improvement for key algorithms. By taking advantage of the heterogeneous computing capability of Snapdragon, the desired computational imaging experiences can be achieved while fitting within the performance, power, and thermal constraints of mobile devices.

MuseAmi: Computer vision and audio analysis

MuseAmi talked about the software they’ve created that can see and hear, the way human beings see and hear. MuseAmi’s technology uses machine learning and digital signal processing to create software that detects, analyzes, and categorizes both audio and images. The MusicPal application allows a person to snap a photo of notated music and then playback that music on their choice of instruments.

In addition, the application can act as a tutor by providing real-time evaluation of a student’s playback accuracy vs. the golden reference. The MusicPal application is optimized by using the CPU, GPU, and DSP for these complex algorithms. MuseAmi expects that further optimization within the GPU and CPU on the Snapdragon processor would result in even faster on-device processing, possibly 8x faster.

ArcSoft: Image processing

ArcSoft talked about the powerful image processing enabled by their software algorithms. ArcSoft develops many different sophisticated imaging algorithms to provide a better camera experience, such as face recognition, high dynamic range, and improved image quality. NightHawk™ is ArcSoft’s low-light video capture technology that greatly improves taking video in poor lighting conditions. By only using the CPU, NightHawk™ would not be able to run in real time since the frame rate would be too low for a good user experience, not to mention that the power consumption would be too high. By using the CPU, GPU, DSP and ISP, ArcSoft claims that the NightHawk™ application is able to run in real time with more than 30% power savings.

Be sure to check out the webinar for many more details about the applications, how the diverse processing engines are being used, and the actual benefits in terms of performance and power.

Want to learn more? Look for future blogs and webinars to learn about Qualcomm’s view on mobile heterogeneous computing.