Co-written with Ross Ning, Meta.

In our previous blog post, we explored the Adreno Motion Engine’s support of Depth-from-Stereo (DFS), highlighting how it generates disparity and depth maps from stereo image pairs. We discussed the various applications of DFS, including mixed-reality XR devices, and emphasized the strengths of the Qualcomm Adreno GPU in delivering high-performance, low-latency DFS solutions.

For a detailed overview of DFS technology and its benefits, you can refer to our earlier post here.

In this follow-on article, we delve into a specific customer use case of using Qualcomm Adreno GPU to improve judder in Meta Horizon Hyperscape to showcase the practical implementation and impact of DFS in real-world scenarios.

Hyperscape

Since our last update, we have been collaborating with Meta to integrate the Adreno Motion Engine’s DFS solution into Meta Horizon Hyperscape.  Hyperscape gives people tools to scan physical spaces, then uploads and renders the photorealistic digital replicas in Horizon.

One of the critical challenges in cloud streaming, especially for experiencing immersive worlds created with Hyperscape, is the occurrence of judder. Judder presents as a stuttering or jerky motion and causes discomfort and breaks the sense of immersion.

This is problematic in mixed reality (MR) environments, where smooth and natural motion is essential for maintaining a convincing and comfortable experience.

To address this, DFS can be utilized to perform positional-timewarp, a technique that reprojects frames based on the user's latest head position and orientation. By generating depth maps in real-time, DFS allows Hyperscape to adjust the rendered images just before they are displayed, ensuring that the visuals align more accurately with the user's movements.

This process effectively reduces judder and provides a smoother, more immersive experience.

What makes the Adreno Motion Engine DFS solution particularly effective in this context is its low latency and high performance.

The Adreno GPU's optimized DFS implementation can generate depth maps in well under 1 millisecond, allowing for just-in-time processing on the client side. This means that even when there is a delay in the streaming frames, the DFS can quickly and efficiently perform the necessary calculations to reproject the images accurately.

This capability is crucial for maintaining a high-quality user experience in cloud streaming applications, where any delay can impact the perceived smoothness and responsiveness of the content.

Takeaway

We are continuously developing and improving the Adreno Motion Engine and adding new capabilities like our Depth-from-Stereo (DFS) technology. Our work with Meta on the Hyperscape technology showcases how low latency DFS can be used to reduce judder and provide a smoother user experience.

Meta is gradually rolling out the update over the coming month.  We encourage you to check out the Hyperscape demo and see the difference for yourself!

Be sure to join our Developer Discord to connect with peer developers and get exclusive access to teams building the products.