New distributed virtual TV set architecture for a synergistic operation of sensors and improved interaction between real and virtual worlds

摘要

A virtual TV set is a studio that is able to combine recorded actors and objects with computer generated virtual environments in real time. In order to achieve this combination seamlessly, in an ideal configuration, several elements such as cameras, objects and people should be tracked so that all their actions on the stage have a corresponding effect in the virtual world. However, in the actual professional virtual TV sets, the tracking possibilities are quite limited because of the hardware and software architecture used, which has not had a major evolution since the first prototypes presented in the nineties. This traditional architecture uses to be rigid, including just one monolithic tracking system and low levels of interactivity. In this paper, a new distributed, flexible and scalable hardware and software architecture that allows the inclusion of multiple kinds of devices in parallel is introduced. It breaks with the traditional structure of the virtual TV sets, opening the technology to an easier inclusion of new devices without the need of updating the proprietary software of the set, thus facilitating its future evolution. The design, implementation and test of this architecture, through the adaptation of a traditional virtual TV set, is presented. The tests are developed through the inclusion of modern devices (in our case Optitrack infrared cameras, Microsoft Kinect V2 and Leap Motion) that, through a synergistic operation, allow the system to solve some traditional drawbacks of this technology such as free and multiple object and camera tracking, presenter natural interaction and automatic distance keying.