Using Computer Vision to Control a Reactive Computer Graphics Character in a Theater Play

摘要

It/I is a two-character theater play where the human character I is taunted and played by a vision-based, autonomous computerized character -called It - which controls computer-graphics, sound, and stage lights. Unlike previous immersive interactive systems, the computer vision system recognizes the human character's actions by considering not only tracking and gestural information and the character's internal variables but also the context provided by the current situation in the story. This paper focuses on a methodology to represent and recognize the human and computer characters' actions that is based on interval scripts, a paradigm that uses Allen's temporal primitives to describe the relationships among the different actions and reactions. The system was tested in six public performances held at the MIT Media Laboratory in 1997, when the computer graphics character ran automatically during the 30-minute duration of the play.