Accelerated Vision-based Pose Estimation utilizing IMU sensor data for Inside-out Tracking in Virtual Reality

摘要

A method of using IMU sensor data of a VR HMD in a cloud VR video encoding and decoding process is disclosed. IMU data can be used as additional information to improve video encoding speed in cloud VR by reducing motion search overhead in the inter prediction process. The use of IMU data for the video encoding process has four advantages. The first is to select an inter-screen reference picture, the second is to determine a prediction method (inter-screen prediction or intra-screen prediction), and the third is to determine a motion search at the starting point. In addition, the IMU can be used to improve the decoding process and time warp at the client end. Video encoding and decoding acceleration based on IMU data can improve the service quality and performance of the entire cloud VR.