Immersive video, including video hypermosaicing to generate from multiple video views of a scene a three-dimensional video mosaic from which diverse virtual video scene images are synthesized, including panoramic, scene interactive and stereoscopic images

摘要

Immersive video, or television, images of a real-world scene are synthesized, including on demand and/or in real time, as are linked to any of a particular perspective on the scene, or an object or event in the scene. Synthesis is in accordance with user-specified parameters of presentation, including presentations that are any of panoramic, magnified, stereoscopic, or possessed of motional parallax. The image synthesis is based on computerized video processing--called "hypermosaicing"--of multiple video perspectives on the scene. In hypermosaicing a knowledge database contains information about the scene; for example scene geometry, shapes and behaviors of objects in the scene, and/or internal and/or external camera calibration models. Multiple video cameras each at a different spatial location produce multiple two- dimensional video images of the scene. A viewer/user specifies viewing criterion (ia) at a viewer interface. A computer, typically one or more engineering work station class computers or better, includes in software and/or hardware (i) a video data analyzer for detecting and for tracking scene objects and their locations, (ii) an environmental model builder combining multiple scene images to build a 3D dynamic model recording scene objects and their instant spatial locations, (iii) a viewer criterion interpreter, and (iv) a visualizer for generating from the 3D model in accordance with the viewing criterion one or more selectively synthesized 2D video image(s) of the scene.