Reinforcement Learning for Production Ramp-Up: A Q-Batch Learning Approach

摘要

The ramp-up process is a significant bottleneck during the development of manufacturing systems. The effort and time required to ramp-up a system is largely dependent on the effectiveness of the human decision making process to select the most promising action and improve the system. Although existing work has identified significant factors influencing ramp-up performance, little has been done to support the actual process. This work approaches ramp-up as sequence of technical changes which aim to get a manufacturing system to a desirable performance in the fastest time. A reinforcement learning approach is proposed to support decisions during ramp-up. The aim is to capture the dynamics between an operator and the system and support time reduction of the process. A batch learning approach has been identified as promising since it matches the practical aspect of decision making during ramp-up. It is combined with a Q-learning algorithm which provides theoretical foundation of optimum convergence. The learning approach has been demonstrated on a highly automated production station during its ramp-up and the generated policy was shown to have significant impact on the ramp-up time reduction.