February 08, 2023

A new way to decorate your workspace using augmented reality (AR). A twist on a Geocaching game. An immersive experience that puts users inside the music. Four Mizzou Engineers participated in MIT’s Reality Hack last month, an international hackathon focused on extended reality (XR, an umbrella term for virtual reality, augmented reality and mixed reality), developing novel ways to implement emerging technologies.

Stuart Aldrich, a senior in computer science, Erika Zhou, a senior in computer science minoring in information technology, Jacob Woods, a second-year information technology major, and Weiyu Feng, BS CS ’21 who’s now a master’s student at the University of California-Berkeley, competed in the hackathon, hosted by Massachusetts Institute of Technology in Cambridge. Aldrich, Zhou and Woods are members of the University of Missouri Virtual Reality Organization (MUVR), while Feng is a former MUVR officer.

Feng’s team won Best Use of Spatial Audio for Up in the Air, a proposed app that turns your office space into a fantasy setting. A user would wear AR glasses to overlay virtual objects onto physical space. The white walls surrounding you could become a sky with moving clouds, an ocean view or other whimsical environments. You could navigate the web from a virtual pop-up box next to your monitor. Email message notification could appear as virtual components matching your chosen theme.

“We think that this could change the workplace from a plain one to a beautiful fantasy-style environment,” Feng said. “Another point of it was functionality; how we interact with our phones, tablets and laptops, connecting all devices together to boost efficiency and bringing AR into this device family.”

Aldrich and Zhou were on a different team that built AR Geocaching, a game in AR using Oculus Quest 2’s passthrough feature.

Essentially, the team combined geocaching — which challenges gamers to locate hidden objects using GPS — and the children’s scavenger hunt that relies on verbal “hot” or “cold” clues, Zhou said. With Bluetooth signals transmitted to the headset, users could get an approximate idea of how close they are to the hidden object. They would also see an augmented temperature gauge on top of their physical space indicating whether they are close to the object, i.e. “hot” or “cold.”

While they developed a game for entertainment, the combination of technologies could have application in industry, too.

“In talking with the judges and thinking about ideas, one thing I thought would be neat would be to use signal strength combined with QR codes to find items in a store,” Aldrich said. “Employees would know roughly how far they were from specific objects based on signal strength, then a QR code would give them exact locations.”

The app could also be expanded with more AR effects to enhance the game features, Zhou said.

Woods was on a separate team that developed Amadeus, an interactive application that teaches students about waveforms using a repurposed Guitar Hero controller and microcontrollers. Looking through VR glasses, users would be immersed in sound particles, allowing them to visualize the transformation of waveforms.

“One of the mentors brought in old toys, and we thought it would be interesting if we could make microcontrollers interface with the guitar and work with the Unity program via Bluetooth,” Woods said. “We made it so you could strum Guitar Hero keys in virtual reality to synthesize music you were playing through the guitar.”

Future Plans

While none of the participants plan to further build on their specific projects, they agreed Reality Hack gave them unique opportunities to explore advanced and emerging technologies too cost-prohibitive to use otherwise. It also highlighted gaps in existing technologies, Feng said.

“I feel like it gave a more comprehensive look at industry and where technology could be improved in the future,” he said.

Woods gained a deeper appreciation for Open Source Software (OSS) at the event. OSS is software with source code that multiple contributors inspect, modify and improve.

“I have a newfound respect for the OSS community,” he said. “Open source projects make huge impacts in the world that probably wouldn’t have been economically viable otherwise.”

Aldrich and Zhou agreed that they learned more about project management — how to better prioritize tasks, ask team members for help and reach out to professionals for advice.

 “Talking to sponsors and mentors — for me that was one of the really cool things about being at the hackathon,” Zhou said. “I really learned a lot talking with them. Mentors were there to help if we had problems. This was a fantastic experience to meet so many talented people from around the world, network with sponsors, try out cutting edge devices and build cool projects.”

The Reality Hack experience also encouraged MUVR to consider starting two projects building open-source XR hardware.

“It was inspiring for us to think about what we can do as an organization to further extended reality at Mizzou,” Zhou said.

Gain the skills needed to work with advanced and emerging technology. Apply to the University of Missouri College of Engineering!