Learning-Based Computing Task Offloading for Autonomous Driving: A Load Balancing Perspective

摘要

In this paper, we investigate a computing task offloading problem in a cloud-based autonomous vehicular network (C-AVN), from the perspective of long-term network wide computation load balancing. To capture the task computation load dynamics over time, we describe the problem as an Markov decision process (MDP) with constraints. Specifically, the objective is to minimize the expectation of a long-term total cost for imbalanced base station (BS) computation load and task offloading decision switching, with per-slot computation capacity and offloading latency constraints. To deal with the unknown state transition probability and large state-action spaces, a multi-agent deep Q-learning (MA-DQL) module is designed, in which all the agents cooperatively learn a joint optimal task offloading policy by training individual deep Q-network (DQN) parameters based on local observations. To stabilize the learning performance, a fingerprint-based method is adopted to describe the observation of each agent by including an abstraction of every other agent’s updated state and policy. Simulation results show the effectiveness of the proposed task offloading framework in achieving long-term computation load balancing with controlled offloading switching times and per-slot QoS guarantee.