Real-time compositing of live recording-based and computer graphics-based media streams

摘要

The present invention relates to providing composite media streams based on a live recording of a real scene and a synchronous virtual scene. A method for providing composite media streams comprises recording a real scene using a real camera; creating a three-dimensional representation of the real scene based on the recording; synchronizing a virtual camera inside a virtual scene with the real camera; generating a representation of the virtual scene using the virtual camera; compositing at least the three-dimensional representation of the real scene with the representation of the virtual scene to generate a media stream; and providing the media stream at the real camera. A camera device, a processing pipeline for compositing media streams, and a system are defined.