Convolutional Neural Network Based Inter-Frame Enhancement for 360-Degree Video Streaming

摘要

360-degree video has attracted more and more attention in recent years. However, it is a highly challenging task to transmit the high-resolution video within the limited bandwidth. In this paper, we first propose to unequally compress the cubemaps in each frame of the 360-degree video to reduce the total bitrate of the transmitted data. Specifically, a Group of Pictures (GOP) is used as a unit to alternately transmit different versions of the video. Each version consists of 3 high-quality cubemaps and 3 low-quality cubemaps. Then, the convolutional neural network (CNN) is introduced to enhance the low-quality cubemaps with the high-quality cubemaps by exploring the inter-frame similarities. It is shown in the experiment that a single CNN model can be used for various videos. The experimental results also show that the proposed method has an excellent quality enhancement compared with the benchmark in terms of PSNR, especially for videos with slow motion.