A client-server architecture for real-time view-dependent streaming of free-viewpoint video

摘要

Streaming of live captured 3D data from multi-camera set-ups or RGB-Depth cameras has many applications in entertainment and for peer-to-peer communication. Emerging technologies of Head-mounted displays (HMDs) equipped with positional tracking allow users now a Holodeck-experience. Live captured content for these devices needs to conform with both, requirements for 3D-TV or 3D-telecommunication systems, namely narrow-baseline high-quality 3D capture and more wide-baseline oriented free-viewpoint video (FVV). This contribution presents a server-client architecture that fits these requirements by combining features of view-dependent rendering with stereoscopic set-ups by optimizing multi-camera information for the viewpoint requested by a client rendering system, based on a games engine. We demonstrate our system with a HMD VR system and an auto-stereoscopic display. Our system achieves low latencies that are required for VR applications. Our results are lower than comparable networked FVV systems.