Visual Multi-touch Air Interface for Barehanded Users by Skeleton Models of Hand Regions

摘要

In this paper, we propose a visual multi-touch air interface. The proposed air interface is based on the image processing and does not require any additional equipment except a built-in camera. The implemented device provides a barehanded interface which copes with the multi-touch operation. The proposed device is easy to apply to the real-time systems because of its low computational load and is cheaper than the existing methods using glove data or 3-dimensional data because any additional equipment is not required. To improve the robustness of the extraction of hand regions under various circumstances, we propose an image processing algorithm based on the HCbCr color model, the fuzzy color filter, and the labeling. In addition, to improve the accuracy of the recognition of hand gestures, we propose a motion recognition algorithm based on the geometric feature points, the skeleton model, and the Kalman filter. Finally, the experiments show that the proposed device is applicable to remote controllers for video games, smart TVs and any computer applications.