Annotating the Real World with Knowledge-Based Graphics on a See-through Head-Mounted Display

摘要

We describe an experimental, knowledge-based, virtual-world system that uses amonocular see-through head-mounted display to overlay graphics on the user's view of the real world. In a simple equipment maintenance domain that we have developed, the overlaid graphics include 3D representations of actual physical objects, textual annotations and callouts, and virtual metaobjects, such as arrows. A set of 3D position and orientation sensors monitor the user's head and several of the objects in the world. Our system includes a knowledge-based graphics component that determines what information to present, a display-list-based display server that represents the models to be displayed, and a set of sensor servers that track the user and selected objects. The sensor servers directly modify the object transformations and viewing specification in the display list. The knowledge-based graphics component also receives the sensor data and uses it to redesign the information being presented. knowledge-based graphics, virtual worlds, headmounted displays, heads-up displays, augmented reality.