A multiplicative multi-linear model for inter-camera prediction in free view 3D systems

摘要

Recordings from multiple cameras in the context of 3D television are transmitted using a simulcast encoding structure or a multi-view encoding structure. Multi-view Video Coding (MVC) extends H.264/AVC standard and exploits the large amount of inter-view statistical dependencies by combined temporal and inter-view prediction in 3D systems. We propose herein an alternative object oriented video coding scheme for multi-view video that introduces an intermediate step of estimating highly correlated sets of image parameters per Croup-Of-Pictures (GOP). A structure that we call a Multi-view Video Plane (MVP) is defined. It holds the cross-correlation coefficients between views. The proposed method may be applied upon groups of orthonormal motion vector parameters corresponding to different views as well. No specific knowledge of the extrinsic or the intrinsic parameters of the recording system is required. Object planes associated with a certain view are approximated as multi-linear components of an image that are cross-correlated with object planes in other views in a tensor-like fashion. The order of the tensor equals the number of multiple views. The correlation coefficients of the tensor subspace projections as well as the updates of the multi-linear components, i.e. object-planes (MVPs) and sets of motion vector parameters (that we call Motion Prediction video-Objects or MPOs), are quantized and transmitted in the MPEG stream. Residual object planes are encoded using conventional MPEG algorithms. Numerical results suggest an improvement compared with current encoding approaches with a moderate increase in computational burden.