Volumetric Media Streaming for Augmented Reality

摘要

Volumetric media, popularly known as holograms, need to be delivered to users using both on-demand and live streaming, for new augmented reality (AR) and virtual reality (VR) experiences. As in video streaming, hologram streaming must support network adaptivity and fast startup, but must also moderate large bandwidths, multiple simultaneously streaming objects, and frequent user interaction, which requires low delay. In this paper, we introduce the first system designed specifically for streaming volumetric media. The system reduces bandwidth by introducing 3D tiles, and culling them or reducing their level of detail depending on their relation to the user's view frustum and distance to the user. To allocate bits among different tiles across multiple objects, we introduce a simple greedy yet provably optimal algorithm for rate-utility optimization, whose utility measures is based not only on the underlying quality of the representation, but on the level of detail relative to the user's viewpoint and device resolution. Simulation results show that the proposed algorithm provides superior quality compared to existing video-streaming approaches adapted to hologram streaming, in terms of utility and user experience over variable, throughput-constrained networks.