February 12, 2024

Mizzou Engineering’s Shane McKelvey and his team earned a first-place award at MIT’s Reality Hack in January for their development of a novel treatment that uses virtual reality to help young patients undergo electro-stimulation therapy.

McKelvey, a junior computer science major, and fellow project participants from across the country won for Creative Inputs/Outputs in the Hardware category.

“The biggest prize was the friends we made along the way,” McKelvey said. “The stand-out thing for me has to be the other people who were there and the connections we made. Who you know is as important as what you know, and Mizzou helps us do both. Classes and research can teach you the what, and these types of opportunities to travel and network helps you meet people.”

McKelvey was one of four current students and two Mizzou Engineering alumni who participated in MIT Reality Hack, which has become one of the largest extended reality (XR) hackathons in the country since it began seven years ago. Fang Wang, associate teaching professor in information technology, provides mentorship for participants, all of whom participate in undergraduate research in Wang’s VR lab. That experience prepared them for success at the hackathon.

Wang also advises the MU Virtual Reality Organization (MUVR), a student organization focused on XR. All six participants are either serving or have served as officers.

While the projects gave them the ability to work with industry-level technology on real-world projects, attendees agreed meeting others was the highlight of the event.

“The main take-away for me was that there are so many like-minded people you can meet who come from different places and different backgrounds with different experiences,” said Livia Fiebelkorn, a junior information technology (IT) major. “It was really a great experience to talk to people and hear different perspectives.”

In addition to meeting students from schools around the nation—including some from the Intercollegiate XR association—the event also brings in professionals in the XR community, including leaders in the field, said Erika Zhou, who graduated with a degree in computer science and minor in IT in December,

“The hacking and projects were cool, but community is a huge part of Reality Hack,” she said. “It was a great networking opportunity.”

Mike Sturm, a junior studying IT, volunteered at the event, helping organizers move equipment and directing other participants. The experience gave him the opportunity to see a variety of different projects.

“I saw how the skills and things we learn in school have real-world applications,” he said. “You may be told to make a program for class, and you think it’s a one-off, but now you’re able to connect it to actual projects. It’s a total summation of what you’ve learned.”

The projects

McKelvey’s team created FlexVR Wellness, an entire ecosystem that allows therapists to use an augmented reality (AR) glasses to interact with a patient remotely while the patient is fully immersed in a VR environment. The team used the Unity game engine, the Normcore multiplayer framework and Snapdragon Spaces to interface the various devices. Ultimately, the system would allow therapists to deliver interactive distance-based treatment.

“The therapist would be able to place things in the VR customized space, making it more calming for a patient in real time,” he said. “With the AR glasses, a therapist would see the patient’s VR setting like it’s on a table top and be able to pick up and move objects like game pieces.”

The goal is to deliver electro-stimulation therapy—which can be uncomfortable—to children in a relaxed setting. Ultimately, McKelvey hopes the system can gamify treatment, allowing patients to “play” while treatment is administered.

Zhou and her team developed the WEDGE, a multi-faceted XR controller that includes two buttons and LEDs, which allow those outside of the VR setting to receive information about what’s happening within the simulation.

“One of the cool things about it is that a person using it can use the buttons to interact with the VR experience, and those outside of VR can see the LEDs blinking, which provides another way of interaction,” Zhou said. “One example is an educational setting. A person facilitating it can use the LEDs to know how the person is doing within the VR setting. A facilitator can use regular VR hand controllers to change the VR experience with that LED feedback.”

Albert Zhou, a senior in IT, and his team created EmpoweredAR, which leverages the ability of XREAL eyewear to generate meshes of the physical environment that, when combined with distance tracking, provides wearers with an auditory map. The idea is to give those with vision impairments a better sense of their environments beyond traditional canes.

Mizzou Engineering alumnus Weiyu Feng also participated in a project, Collage MR. The mixed reality app lets users click on an object to generate a related experience. The idea is to allow photos, videos and 3D scenes to connect so users can essentially “visit” their memories.

McKelvey thanked the Organization Resource Group (ORG) at Mizzou for helping fund the trip.

“Meeting people has been really valuable to me,” he said. “We’re making connections, and I’m seeing now why Mizzou provides these opportunities—I see what it does for everyone involved. It’s cool to see what I can do for the University when they invest in me.”

Be part of a university that invests in you! Apply today.