Multi-User Adaptive Video Delivery Over Wireless Networks: A Physical Layer Resource-Aware Deep Reinforcement Learning Approach

摘要

In this paper, we investigate the adaptive video delivery for multiple users over time-varying and mutually interfering multi-cell wireless networks. The key research challenge is to jointly design the physical-layer resource allocation scheme and application-layer rate adaptation logic, such that the users' long-term fair quality of experience (QoE) can be maximized. Due to the timescale mismatch between these two layers and the asynchrony of user requests, however, it is difficult to directly model the cross-layer stochastic control problem by using a reinforcement learning framework. To address this difficulty, we propose a novel two-level decision framework where an optimization-based beamforming scheme (performed at the base stations) and a deep reinforcement learning (DRL)-based rate adaptation scheme (performed at the user terminals) are, respectively, developed, such that a highly complex long-term multi-user QoE fairness problem is decomposed into some relatively simple problems and solved effectively. Our strategy represents a significant departure from the existing schemes with consideration of either a short-term multi-user QoE maximization or a long-term single-user point-to-point QoE maximization. Extensive simulations demonstrate that the proposed cross-layer design is effective and promising.