Reinforcement Learning Based Continuous-Time On-line Spacecraft Dynamics Control: Case Study of NASA SPHERES Spacecraft

摘要

In this paper, we propose a reinforcement learning based framework to generate the optimal control policy for continuous-time SPHERES spacecraft model in real-time with only partial knowledge or without any knowledge of the underlying dynamics. This framework allows the spacecraft to learn the control strategy for the rendezvous problems from the past experience, current plant dynamics and unknown environments, which is extremely crucial for spacecraft maneuver operations in International Space Station (ISS) and constitute the novel approach in the paper. The on-line adaptive critic based and Q-learning based algorithms are developed for the partially model-free and model-free SPHERES spacecraft model to solve the corresponding optimal control problems. The effectiveness and functionality of presented schemes are verified/validated through the simulation results.