January 12, 2024

Augmented and virtual reality are changing the way universities can offer remote and online courses. These technologies allow for course materials to be presented to students in a more engaging and interactive way. However, right now, there’s a disconnect between the “wow” factor and what students actually learn using these technologies.

“We understand from our educational experience that offering impressive visual enhancements does not enhance ultimate learning outcomes,” said Jung Hyup Kim, associate professor in industrial and systems engineering. “Students are required to invest time in interacting with educational materials, grasping the concepts, and learning from their mistakes. However, this type of feedback system is still lacking in AR environments .”

Kim has funding from the National Science Foundation for ongoing investigation into best practices using augmented reality, specifically for engineering education. While a lot of studies are looking into this topic, Kim’s work is unique in that he’s determining how metacognition—or one’s thinking about their own thinking—directly relates to performance and workload.

“In a classroom, when we inquire if students have any questions and nobody responds, we can interpret their facial expressions to assess their understanding of the material,” he said. “We can then revisit certain details. However, in a virtual setting, this type of feedback is not available..”

One area that’s been debated is the usefulness of having virtual instructors absent of human faculty. Ideally, interactive virtual course materials can stand on their own without any sort of teaching guidance. However, in his latest study, Kim and collaborators found that the benefits of virtual instructors directly correlate to student metacognitive status.

“Overall, when we looked at the entire group of students, there were no significant differences in learning performance between classes that had virtual instructors and those that did not have virtual instructors,” he said.

However, when split into groups of overconfident students and those who were underconfident, virtual instructors were shown to significantly improve the performance of the latter group.

Essentially, students who are overconfident report feeling that they grasp course concepts. Yet, when tested, these students fail to demonstrate that they truly understand and can apply those ideas.

On the flip side, students with underconfident express challenges in comprehending the materials, yet show a higher level of understanding than they initially thought. 

“When separated, the underconfident students whom virtual instructors taught achieved significantly better results than those who did not have virtual instructors,” Kim said. “So we found that virtual instructors, offering step-by-step instruction, is beneficial for students with underconfident in AR learning environments.”

For students who are overconfident, there’s a relationship between workload and performance. When students think they understand concepts, but struggle to demonstrate that knowledge, it significantly increased stress levels.

Understanding these subtle differences between how students learn can allow those developing augmented and virtual courses to integrate features based on student cognition and performance. Ideally, Kim said, students would alternative from being identified as overconfident and underconfident, and would receive virtual instruction accordingly.

The next step is to use motion capture and eye tracking data to gain even deeper insights into a student’s cognitive learning processes during augmented or virtual courses.

“We can observe the movement of their eyes to discern their focus, which will indicate whether they are watching without actually learning,” Kim said. “Based on that data, we can provide different types of feedback such as a prompt to maintain attention or motivation to complete a learning module.”

Ultimately, the goal is to have a solid understanding of how students learn in virtual settings to ensure these technologies are efficient moving forward.

“Industries are increasingly looking to retrain workers and cannot afford to wait a year, leading them to direct employees towards virtual classes alternatives,” Kim said. “But the learning outcomes may differ from what they anticipate or require. We need to provide develop improved methods to maximize the benefits of AR/VR learning.”

Kim’s study was published in the Proceeding of the Human Factors and Ergonomics Society Annual Meeting.

Learn more about Kim’s lab and research here.