An augmented reality situational training system by recognition of markers and hands of trainee

摘要

The present invention relates to a situation training system that allows a developmentally disabled person to repeat the training safely and without restriction of the surrounding environment by constructing an augmented reality space in which a marker is placed in a real training environment and objects that are difficult to use in a real space are replaced with virtual objects. As an augmented reality situation training system using a marker and trainer hand recognition according to the present invention, a mobile camera for capturing an image of reality in the direction of the eye of the trainer; A fixed camera installed at a fixed position and capturing an image of a reality of a training area of a trainer; A camera input unit for inputting images captured by the mobile camera and the fixed camera, and a marker matrix including position coordinates and rotation information are detected by detecting all markers included in the fixed camera image and the mobile camera image inputted by the camera input unit. A marker detection unit to generate, a hand recognition unit which extracts the position information of the hand end of the trainer and the hand axis direction information from at least one of a fixed camera image and a mobile camera image inputted by the camera input unit, and three-dimensional model data Using the model data storage to store and the marker matrix transmitted from the marker detector, the visibility of a specific marker, the distance between the marker and the marker, and the distance between the viewpoint of the moving camera and the marker and the hand endpoint position information transmitted from the hand recognition unit And hand axis direction information are combined to generate the behavior information of the trainer In addition, scene information provided to the trainer by applying the behavior information of the trainer to the scene information according to the predefined scenario and the situation control unit for determining whether to transfer to the next scene, and the mobile camera image input by the camera input unit as a background image An image matching unit for extracting and matching a corresponding 3D model from a model data storage unit at a marker position using a marker matrix transferred from the marker detection unit, and an HMD output unit for outputting an image matched by the image matching unit; A training controller; And a head mounted display which is integrally coupled with the mobile camera and displays an image output by the HMD output unit.