Self-Reproducing Video Frame Interpolation

摘要

Frame interpolation has recently witnessed success by convolutional neural networks, that are learned from end to end to minimizing the reconstruction loss of dropped frames. This paper introduces a novel self-reproducing mechanism, that the real (given) frames could in turn be interpolated from the interpolated ones, to further substantially improve the consistency and performance of video frame interpolation. Such a consistency constraint accounts for the inherent symmetry between existing and interpolated frames in a video sequence, providing a strong form of self-supervision. We then build a pyramid-like architecture, under which existing interpolation models can plug-and-play as building blocks. Extensive experiments validate its state-of-the-art performance, on both high resolution videos in the wild and public benchmarks.