The design and evaluation of a mixed reality approach to interactively blend dynamic models with corresponding physical phenomena.

摘要

People understand how to interact with the objects in the world around them (e.g., an ATM machine, a car) but most people do not understand how these objects operate internally. Moreover, even with an abstract knowledge (e.g., a schematic) of an object, it still may be difficult to apply this knowledge in the context of the real object. To address this challenge, this work presents an interactive approach called a mixed simulator that superimposes abstract visualizations over the corresponding real objects and improves overall understanding of the objects in the surrounding world.;For example, to address this challenge in anesthesia education we engineered the Augmented Anesthesia Machine (AAM)--a mixed simulator that superimposes a dynamic, abstract model of a generic anesthesia machine over a real anesthesia machine. Moreover, the machine and dynamic model are synchronized, enabling students to interact with the model through their physical interaction with the machine, such as turning knobs. Students can then visualize how their physical interactions affect the internal workings (e.g., gas flow dynamics) of the machine, effectively affording students abstract 'x-ray vision'.;We evaluated the mixed simulator approach in a formal study that investigated the educational benefits specific to mixed simulators. The study compared mixed simulators to several other types of currently used training simulators. Overall we found that mixed simulators compensated for low spatial cognition and more effectively helped users to transfer their abstract knowledge into real world scenarios.;To extend mixed simulators, we engineered a novel immersive visualization approach that enabled students and educators to aggregate, filter, visualize, and review massive amounts of previous student interaction data. An informal study suggested that this immersive approach for after action review (AAR) may give students and educators insight into the elusive thought processes and misconceptions of students.;Finally, we generalized mixed simulators in both software and theoretical frameworks for a more effective design and implementation process. The theoretical framework enables engineers to classify and design mixed simulators based on the educational needs of an application. The software framework supports engineers with a code infrastructure and an authoring tool for efficient implementation of mixed simulators.