Real-time 3D Sign Language Avatar Animation from DTV Closed Captions

摘要

Whereas closed captions are provided for almost the whole broadcast time for deaf people, in South Korea sign language broadcasting covers only 5% of the time. By translating the closed captions into sign language it is possible to provide three-dimensional (3D) sign language interpretation for more of the broadcast time. We propose a real-time avatar-based sign language broadcasting system for all kind of TV programmes. We analyzed the last three years' of weather forecast scripts, and an open-domain corpus of about 1.2 million words from KBS, so that we could find the frequency of each word and determine the priority of sign word motion capture. We used the Korean wordnet, KorLex, to build the sign language synonym dictionary and disambiguate the words with multiple meanings, for better translation performance. Optically-captured 3D sign language motions are used for the avatar to present motion-blended sign language interpretation. We implemented the real-time sign language interpretation system by extending the sign language dictionary, translation module, and motion database of our previous on-demand weather forecast sign language interpretation system.