Integration Interface Method and System based on Eye tracking and Gesture recognition for Wearable Augmented Reality Device

摘要

An integrated interface method and system for integrating an eye tracking system and a hand gesture recognition system, and finally integrating a wearable AR device application, eye tracking, and hand gesture recognition system are provided. An integrated interface method according to an embodiment of the present invention includes: calculating, by the integrated interface system, a position of a user's gaze using the user's eyes and anterior image; moving the cursor and selecting an object, by the integrated interface system, based on the result of calculating the gaze position; Recognizing, by the integrated interface system, a hand gesture from the user's hand gesture data; and selecting and manipulating, by the integrated interface system, an object based on the hand gesture recognition result, wherein, in the selecting and manipulating step, a command for enlarging, reducing, or rotating the virtual object is performed according to the hand gesture recognition result. do. Accordingly, it is possible to move a cursor and select and manipulate an object based on information about the user's both pupil positions and hand gesture recognition information in real time, thereby providing convenience to the user of the wearable AR device.