Run-to-Run Control of Chemical Mechanical Polishing Process Based on Deep Reinforcement Learning

摘要

The chemical mechanical polishing (CMP) process usually suffers from drift and shift in the Run-to-Run material removal process due to the wear and replacement of the polishing pad, lacking of in-suit measurements of the product quality of interest and other environment variations. This paper proposed a deep reinforcement learning (DRL)-based run-to-run (R2R) controller for the CMP process. Firstly, deep reinforcement learning is effectively utilized as a training algorithm of the R2R controller, which is a model-free controller to take a decision with infinite horizon information and thus improves the control performance; Secondly, a novel policy network is embeded to the DRL model, which divides the network into linear and nonlinear part explicitly to improve the prediction performance of the R2R controller on process changes. Finally, a special reward function is proposed to improve the training of the R2R controller, which trades off between target tracing and fluctuations of production parameters. The effectiveness of the proposed controller is validated on a CMP process. The testing results illustrate that the DRL-based R2R controller can precisely trace the desired target of material removal rate (MRR) and is very effective to control various process variations online.