Development of an Intuitive Interface for PC Mouse Operation Based on Both Arms Gesture

摘要

This paper describes an intuitive interface to operate the mouse based on both arm gestures. The Kinect sensor provided the skeleton information of the user’s posture, which was used for detection of joint positions of upper half of body. From the obtained joint positions, the motion of the operator was recognized to reflect a mouse operation. Here, we constructed the system which could choose two displays, scroll and move the position of the mouse cursor by changing input modesaccording to the length of the extended right hand. The user could decide the position of the cursor with the right hand, and click with the left hand. In addition, we made the wearable device on the wrist for sensory feedback to user. In the test operationof the developed system, the user could successfully operate the mouse cursor by using both arms gesture,as sameas the general mouse and the touch pad device.