Interacting with spatially augmented reality

摘要

We present the notion of projector-based spatially augmented reality (SAR), and explore how it can be used as an effective user interface to enable users to easily and naturally interact with their real physical environment and the virtual environment. In SAR, the user's physical environment is illuminated with images projected from the projectors. We then describe a framework that can easily incorporate different types of interactions on a continuum of display surfaces and input devices. More specifically, in the framework, we first establish an understanding of the 3D relationship between the user, the projectors and the display surfaces. Then, with the help of some new calibration and rendering techniques, only a simple procedure is needed to effectively illuminate the display surfaces. We describe various underlying techniques and discuss the results in the context of three different applications.