The interaction with screen using the projection head mounted display with an infrared camera

摘要

The aim of this study is to propose a interaction style with a new device and show the example application. A small projection head mounted display with infrared camera and light source is developed. This HMD project the image on a hand-held retro-reflective screen. The infrared camera equipped on this HMD is watching the image of the screen and estimate the location (position and orientation) of the screen relative to the user's head. The shape of the user's finger on the screen is also recognized samely. With these information about the screen location and fingertip position, the user can interact with the virtual object on the screen.