I heart this hack: How I won the MIT Reality Hack with Snapdragon Spaces

Earlier this year, I visited the bustling city of Boston to join the world's most talented developers (devs) who gathered at Massachusetts Institute of Technology (MIT) for the famous MIT Reality Hack. The moment I stepped inside the hall, I knew it was going to be an exciting weekend: The event was abuzz with participants like me who were ready to explore the possibilities of extended reality (XR). My curiosity peaked — “Who will I meet and what will it be like to work with them?” I wondered.

As an aspiring developer, I was eager to showcase my skills and create something extraordinary using the latest cutting-edge technologies. Little did I know that my journey would intertwine with Snapdragon Spaces, an extended reality (XR) platform developed by Qualcomm Technologies, Inc.

The importance of hackathons for devs

Freedom: From a developer’s perspective, hackathons are very important events because it gives us a chance to think and innovate freely out of the daily work routine.

Challenge: Bonus points if the hackathon has access to the latest technologies (even if they’re still in beta). This introduces a whole new set of challenges that makes the hackathon more enjoyable.

Prototype: For devs, the fruit of hackathons is the working prototype we get at the end of it. It represents a result of continued hard work during the hackathon, and it will forever live on that developer’s GitHub account as part of their portfolio, giving them priceless exposure.

Collaboration: Hackathons depend on team work; collaborating with mentors and other hackers from different backgrounds. This gives us developers a chance to grow our network by engaging with people from different industries, as well as different levels of expertise.

As a developer with more than five years of experience in the field of XR, I’ve always had my eyes on MIT Reality Hack and I felt fortunate that as a first-time attendee, I got to be part of its “presence” edition, which meant I got to work with Snapdragon Spaces platform and collaborate with the team behind it.

What is Snapdragon Spaces XR Developer Platfrom?

Warning, geeky content ahead!

Snapdragon Spaces is a platform for developing XR experiences on Qualcomm Technologies' mobile hardware that leverages OpenXR application programming interfaces. The software development kit (SDK) — on Unity’s side — is built upon ARFoundation, leveraging the same APIs used when building for other mobile augmented reality (AR) platforms.

Snapdragon Spaces targets Android devices, which means a developer will export the experience into an .APK to run on an Android device, just like developing a normal mobile application (app).

Why developers work with Snapdragon Spaces

I chose to work with Snapdragon Spaces SDK for this build because it was the most stable among the MIT Reality Hack sponsor platforms. In fact, the whole team agreed that this platform would be ideal for developing our MR experience. Let's explore the key features that drove this decision.

Snapdragon Spaces key features:

• Positional tracking
  • 6 DOF (six degrees of freedom): Tracks the user’s movement and positioning in a three-dimensional space with six degrees of freedom.
  • Anchors: Tracks virtual objects with fixed reference points in (three dimensional) 3D space (also known as anchors).
  • Hit testing: Raycasting and detecting collision against virtual objects in the 3D space.

• Interactions
  • Gaze: Allow users to interact with virtual objects by gazing at them, for example, highlighting an object when the user gazes at it.
  • Companion controller: Allow the user to interact by a touch pad via phone screen as a companion controller.
  • Hand tracking: Allow users to interact via hand gestures naturally with virtual objects.

• Environmental understanding
  • Plane detection: The ability to detect a flat surface in the real world, for example, finding a surface on the ground to place a virtual object on it.
  • Image tracking: The ability to recognize and detect an image in the real world, for example, placing a virtual object on top of an image.

Snapdragon Spaces has a unique hardware setup. The hardware kit consists of a mobile device connected to AR glasses via display cable, and the interconnections between them is handled by the Snapdragon Spaces services app. Here’s a closer look at what’s included in the Snapdragon Spaces hardware kit (as of January 2023):

• Motorola Edge+ smartphone: The mobile device to which the app will be deployed and run, also can be used as a touch pad controller. The main feature that makes this smartphone stand out is the Snapdragon 8 Gen 1 processor (view the full specs here).
• Lenovo ThinkReality A3 smart glasses: See-through AR glasses — where all the visualization happens. At the time of the hackathon, the SDK didn’t yet support streaming, so we couldn't stream the experience through the 8 RGB camera.
• Snapdragon Spaces Services app: This is where the magic happens. This app should be started before you run your experience in order to convert it from a normal three-dimensional (3D) app that works on a two-dimensional (2D) screen, into an MR app that's fully visualized on Lenovo ThinkReality A3 mentioned previously.

Limitations

Despite its power, like any XR platform, Snapdragon Spaces has its limitations. Some of those include:

• A narrow Field of View (FOV).
• Testing battery drain: The glasses drain the phone battery, so make sure to disconnect it if not in use.
• Debugging issues: Device debugging is difficult due to the hardware setup; it was hard to debug the app on the headset.

Developing for Snapdragon Spaces

During the hackathon I used Unity game engine to develop the MR experience, leveraging these tools and resources: 

• Unity 2020.3.35f1+,
• Android build support,
• OpenXR plugin,
• Snapdragon Spaces SDK, and
• Resources.

Challenges and setbacks

Hackathons can present many challenges for developers from platform limitations to bugs, to non-technical challenges such as:

• Setting up Snapdragon Spaces: The onboarding process and limited documentation in hand made it slower to set up the project.
• Navigating Android build environment issues.
• Last-moment updates to the Services app.
• SDK issues and bugs that got resolved on site by the Snapdragon Spaces team — we nicknamed these solutions in code “Hack for the Hack.”

The reward of hard work

After coding, testing, solving bugs and polishing for two-and-a half days straight without rest, my hard work finally paid off when I won “Best Use of Snapdragon Spaces.” I would've never accomplished that without the help of the Snapdragon Spaces team. Our mentors — and of course my team at the Hackathon Stone Soup — along with the beautiful 3D designs and the brilliant idea of our project, is what made this an award-winning build.

As I reflect on this journey from MIT Reality Hack to collaborating with Snapdragon Spaces, I can't help but feel joy and a sense of accomplishment. I had the chance to work on a brand-new tech which is Snapdragon Spaces, and I had the opportunity to team up with Marques, Nik, Simian and Dolce from whom I’ve learned a lot. During the hack I had the chance to collaborate with an XR influencer, Director of Product Management for Qualcomm Technologies Steve Lukas and his amazing team at Snapdragon Spaces. Lastly, I was able to go back home with a working prototype, a prize and lots of memories. 

I am grateful to have had this experience and for its impact on my portfolio and career as a developer to this day. As an XR developer, the entire experience confirmed I chose the right path more than five years ago. Today, I will continue to push forward with the same passion and the experience I brought from MIT Reality Hack. And I hope this inspires you too — to anticipate what's next in XR, to continue to push the boundaries, to learn, and to continue to shape the future with XR.