Role of computer vision in augmented virtual reality

摘要

An important issue in augmented virtual reality is making the virtual world sensitive to the current state of the surrounding real world as the user interacts with it - changing gaze, manipulating an object, etc. For providing the right virtual stimulus at the right position and time, the system needs some sensor to interpret the surrounding scene. Computer vision holds great potential in providing the necessary interpretation of the scene. We present the preliminary design of a computer vision-based augmented reality system for helping a human in assembling an industrial part from its components. The context of assembly helps in keeping the computer vision task simple by exploiting the geometric model of the assembly components for recognition and pose estimation. The augmentation stimuli include labeling of objects in the scene, helping with sequencing using an assembly planner, visualization of assembly at different stages, handling errors by the human operator, etc. Such a system would have potential applications in assembling complex parts, maintenance, and education. We will present an overview of the design of the system and discuss some of the issues involved in computer vision-based augmented reality.