Learning while preventing mechanical failure due to random motions

摘要

In this thesis one of the negative effects of learning from scratch on the durability of LEO is analysed. LEO is one of the bipedal walking robots of the TU Delft Robotics Institute. It uses Reinforcement learning to learn a stable and energy efficient walking gait. LEO’s learning algorithm causes its gears to fail faster during the initial learning phase than during the optimisation phase. One of the reasons for the low mean time between failure (MTBF) is the learning algorithm itself. The learning algorithm initially causes random motions which in turn cause high stresses in the gears and mechanical failure. The MTBF due to these motions can be predicted. This MTBF can be increased by adapting the learning algorithm in various ways. We investigated 5 algorithms that increase the MTBF and compared them to SARSA(?) learning. In general, increasing the MTBF decreases the learning performance. Three of the investigated algorithms are unable to increase the MTBF while keeping their learning performance approximately equal to SARSA(?). Two algorithms are able to do this: the PADA algorithm and the low-pass filter algorithm. In case of LEO, the MTBF can be increased by a factor of 108 compared to SARSA(?) learning. This indicates that in some cases, failures due to random motions can be prevented without decreasing the learning performance.