PROPANE: A Progressive Panorama Streaming Protocol to Support Interactive 3D Virtual Environment Exploration on Graphics-Constrained Devices

摘要

Image-Based Rendering (IBR) has become widely known by its relatively low requirements for generating new scenes based on a sequence of reference images. This characteristic of IBR shows a remarkable potential impact in rendering complex 3D virtual environments on graphics-constrained devices, such as head-mounted displays, set-top boxes, media streaming devices, and so on. If well exploited, IBR coupled with remote rendering would enable the exploration of complex virtual environments on these devices. However, remote rendering requires the transmission of a large volume of images. In addition, existing solutions consider limited and/or deterministic navigation schemes as a means of decreasing the volume of streamed data. This article proposes the PROgressive PANorama StrEaming protocol (PROPANE) to offer users a smoother virtual navigation experience by prestreaming the imagery data required to generate new views as the user wanders within a 3D environment. PROPANE is based on a very simple yet effective trigonometry model and uses a strafe (lateral movement) technique to minimize the delay between image updates at the client end. This article introduces the concept of key partial panoramas, namely panorama segments that cover movements in any direction by simply strafing from an appropriate key partial panorama and streaming the amount of lost pixels. Therefore, PROPANE can provide a constrained device with sufficient imagery data to cover a future user's viewpoints, thereby minimizing the impact of transmission delay and jitter. PROPANE has been implemented and compared to two baseline remote rendering schemes. The evaluation results show that the proposed technique outperforms the selected and closely related existing schemes by minimizing the response time while not limiting the user to predefined paths as opposed to previous protocols.