Improving the safety of reinforcement learning models

摘要

An infrastructure is provided for improving the safety of autonomous systems. An autonomous vehicle management system (AVMS) controls one or more autonomous functions or operations performed by a vehicle or machine such that the autonomous operations are performed in a safe manner. An infrastructure is provided for using reinforcement learning (RL) AI models in a safe manner. The infrastructure implements a safety indicator (e.g., Safety Sanity Index (SSI)) that provides an indication of the safety performance of an RL model. The SSI value provides an indication of a degree of safety given an observed state of the vehicle and one or more safety considerations for the vehicle. Additionally, the infrastructure provides interruptible commands that can be used to override an action selected by an RL agent when the safety indicator (or a sequence of safety indicator values) indicates an unsafe condition.