On deep reinforcement learning security for Industrial Internet of Things

摘要

The Industrial Internet of Things (IIoT), also known as Industry 4.0, empowers manufacturing and production processes by leveraging automation and Internet of Things (IoT) technologies. In IIoT, the information communication technologies enabled by IoT could greatly improve the efficiency and timeliness of information exchanges between both vertical and horizontal system integrations. Likewise, machine learning algorithms, particularly Deep Reinforcement Learning (DRL), are viable for assisting in automated control of complex IIoT systems, with the support of distributed edge computing infrastructure. Despite noticeable performance improvements, the security threats brought by massive interconnections in IoT and the vulnerabilities of deep neural networks used in DRL must be thoroughly investigated and mitigated before widespread deployment. Thus, in this paper we first design a DRL-based controller that could be deployed at edge computing server to enable automated control in an IIoT context. We then investigate malicious behaviors of adversaries with two attacks: (i) function-based attacks that can be launched during training phase and (ii) performance-based attacks that can be launched after training phase, to study the security impacts of vulnerable DRL-based controllers. From the adversary's perspective, maximum entropy Inverse Reinforcement Learning (IRL) is used to approximate a reward function through observation of system trajectories under the control of trained DRL-based controllers. The approximated reward function is then used to launch attacks by the adversary against the Deep Q Network (DQN)-based controller. Via simulation, we evaluate the impacts of our two investigated attacks, finding that attacks are increasingly successful with increasing accuracy of the control model. Furthermore, we discuss some tradeoffs between control performance and security performance of DRL-based IIoT controllers, and outline several future research directions to secure machine learning use in IIoT systems.