Document search support by making physical documents transparent in projection-based mixed reality

摘要

This paper presents Limpid Desk that supports document search on a physical desktop by making the upper layer of a document stack transparent in a projection-based mixed reality environment. A user can visually access a lower-layer document without physically removing the upper documents. This is accomplished by superimposition of cover textures of lower-layer documents on the upper documents by projected imagery. This paper introduces a method of generating projection images that make physical documents transparent. Furthermore, a touch sensing method based on thermal image processing is proposed for the system’s input interface. Areas touched by a user on physical documents can be detected without any user-worn or handheld devices. This interface allows a user to select a stack to be made transparent by a simple touch gesture. Three document search support techniques are realized using the system. User studies are conducted, and the results show the effectiveness of the proposed techniques.