A robust projector-camera interactive display system based on finger touch control by fusing finger and its shadow

摘要

A robust projector-camera interactive display system consisting of one camera and one projector is proposed, which enables users to interact with computer by touching on arbitrary surfaces with bare hand. The system utilizes the camera to recover 3D information of fingertip for touch detection. Firstly, predicted image generated from geometric and photometric calibration matrix is used to segment hand and its shadow. Secondly, fusion degree of the hand and its shadow is detected using linear-scanning method. Finally, a square tracking mask is used to determine the touch action on the projection surface. Experimental results show that the proposed system is robust to different lighting conditions. The touch detection accuracy is 93.0% and the average processing time of each frame is 36 ms.