Q-Learning Applied to Genetic Algorithm-Fuzzy Approach for On-Line Control in Autonomous Agents

摘要

This paper proposes an adaptive approach based on a state-action space search for learning and planning an optimal policy with direct reinforcement learning. The optimization of the fuzzy rule-based system is performed by a combination of genetic algorithms and Q-learning, whereby an agent-based predicting machine with desired performance is achieved. The heuristic search method is constructed with the advantage of finding the optimal solution without evaluating the entire state-action space, which gains substantial computational savings for large state-action space. Genetic algorithms are used to generate and select the appropriate small number of subsets of fuzzy If-Then rules and learning of the action-value function (or credit assigned to fuzzy rules) is performed with Q-learning scheme. Selected rules are considered as strings (or individuals) and genetic operations such as selection, crossover, and mutation are applied to them. The experienced trajectory is achieved by learning the action-value function toward the optimal value. The evolution forms an explicit algorithm, which makes decisions that bias the proposed result. The designed evolved autonomous agents are capable of learning and establishing primarily action-effect relations without any prior knowledge either for control problems encountering large number of stimuli. For illustrating the validity of the described technique in control applications, the approach is evaluated on the acrobot task.