Hand Gesture Interaction Using Neural Network and Data-Glove in Virtual Reality

摘要

We developed a driver simulator based on virtual reality technique to train the drivers of artillery Self-Propelled Gun ( SPG) . The gestures and positions of the hand in real world, which are gotten from the datatglove and motion tracking device, are used to drive the virtual hand in virtual environment. A DataGlove with 18 sensors is employed to perform the driving tasks such as pressing switches, manipulating steering wheel, changing gears, etc. To accomplish these tasks, some hand gestures must be defined from the DataGlove sensors data. This paper uses BP neural network to recognize the hand patterns which exist in the raw sensor data of the DataGlove. A pattern set of 300 hand gestures is used to train and test the neural network. To validate the efficiency and practicability of the presented methods, we apply the hand gesture based interaction technique to the virtual driving training of SPG. The users can manipulate the virtual objects and components in virtual environment as if they are in real world.