Hand gesture recognition framework for recognizing sign gestures and handling movement epenthesis using Level Building nested dynamic programming approach

摘要

In this research paper, two crucial problems in continuous sign language recognition from unaided video sequences are considered. At the feature level, the problem of hand segmentation and grouping is considered and at the sentence level, the movement epenthesis problem is considered. A framework that can handle both of these problems based on an enhanced, nested version of the dynamic programming approach is constructed. To handle movement epenthesis problem, a nested version of a dynamic programming framework, called Level Building is used to simultaneously segment and to match signs from continuous sign language sentences. This approach is then coupled with a trigram grammar model to optimally segment and label sign language sentences. This approach will show improvement over past approaches in terms of the frame labeling rate and also our approach shows the flexibility when handling a changing context. The proposed approach is novel since it does not need explicit any models for movement epenthesis.