Recognition of human hand activity to convey series of numerical digits in a natural way using hand shape tree structure

摘要

This paper presents a human hand activity recognition system to recognize the activity of conveying a series of numerical digits in a video. In this work user may use one hand (left or right) or both hands to represent numerical digits ranging from zero to nine. This work uses hand shape features from each frame extracted from a video are fingertips, palm line, palm center, opened finger tips, closed finger tips and presence of thumb. Recognition of number of fingers used to convey numerical digits are recognized by representing the opened fingertips as nodes in a Hand Shape Tree (HST). The tall fingertip represents the root node of HST and other fingertips as its left or right nodes. To determine the fingertip as folded or opened fingertip, draw two right angled triangles with common base using root, its adjacent node and the palm line point as vertices, compare the perimeter of these two triangles to determine the fingertip is from the folded or opened finger. Now count the opened nodes and the presence of thumb if any. This work also recognizes whether the hand activity is from right hander or left hander in practice by using the HST. The system first detects the hand(s) by using YCBR color space and uses our FT algorithm to determine fingertips based on 2D coordinate geometry system.