Qualcomm Cloud AI 100 showcases leadership in power efficiency with the latest MLPerf v2.1 results

Qualcomm Cloud AI 100 has once again demonstrated industry leadership in performance-to-power efficiency from edge to cloud in the latest MLCommons™, MLPerf™ v2.1 Inference testing results.

MLCommons is an open engineering consortium that focuses on creating machine learning benchmarks for training and inference testing platforms. The tests involve AI inference in data center environments in both offline and server query-driven scenarios, and in Edge servers and devices in offline, single-stream, and multi-stream scenarios.

For this latest round, we expanded our results based on the Qualcomm Cloud AI 100 to over 200 with submissions coming from Dell, HPE, Lenovo, Inventec, and Thundercomm. Some of the new platforms include HPE Edgeline EL8000 - e920d, Dell PowerEdge 7515, and Lenovo ThinkSystem SE350. For the first time, PCIe HHHL-Standard (350 TOPs) accelerator-based submissions were also made in addition to PCIe HHHL-Pro (400 TOPs) accelerators. We also introduced three new edge device platforms based on Snapdragon technology with Qualcomm Cloud AI 100 accelerators. These include Gloria High-End with 200 TOPs, Thundercomm TurboX EB6, and Inventec Heimdallr – each with a 70 TOPs accelerator.  

With the increased size and complexity of AI and ML workloads, it becomes critical that the solutions that support those continue to offer a better value proposition. Inference-per-second-per-watt (I/S/W) has emerged as the most important benchmark to provide the best value-to-service for providers and end users. Qualcomm has reinforced our leadership in power efficiency with our MLPerf™ v2.1 submission.

Throughput power is also critical to accelerate AI deployment. The 2U server platform with 18x Qualcomm Cloud AI 100 accelerators achieved the ResNet-50 offline peak performance of 428K inference per second (418K in V2.0). 

The MLPerf™ v2.1 introduced a new RetinaNet object detector replacing previous object detectors SSD-ResNet34 and SSD-MNv1. We had made RetinaNet submissions in Open Div as our SW release cadence could not align with MLPerf submission dates. In addition to Qualcomm, industry leaders like Dell, HPE, and Lenovo have also done RetinaNet benchmark submissions. The Qualcomm Cloud AI 100 based RetinaNet submissions demonstrate our leadership in power efficiency (inference/watt).

The MLPerf™ v2.1 submissions showcases Qualcomm’s breadth of inference applications for both Edge and Datacenter categories, while demonstrating a significant advantage over all competitors in key metrics such as Inf/Sec and inf/sec/watt. The Qualcomm Cloud AI 100 provides a unique blend of high computational performance, low latency and low power utilization, and is well suited for a broad range of applications ranging from Edge to Cloud.