基于函数逼近的冗余值迭代算法

摘要

针对值迭代算法存在算法收敛不稳定及收敛速度慢的问题,文中提出改进的基于函数逼近的冗余值迭代算法.结合值迭代算法与贝尔曼冗余值迭代算法,引入权重因子,构建值函数参数更新向量.同时从理论上证明,利用此更新向量更新值函数参数可以保证算法收敛,解决值迭代算法收敛不稳定的问题.此外,算法引入遗忘因子,加快权重向量的更新速率和算法收敛速度.在Grid World问题上的实验表明,文中算法收敛性能较好,具有较好的鲁棒性.%Aiming at the problem of unstable and slow convergence of traditional value iteration algorithm, an improved residual value iteration algorithm based on function approximation is proposed.The traditional value iteration algorithm and the value iteration algorithm with Bellman residual are combined.Weight factors are introduced and new rules are constructed to update value function parameter vector.Theoretically, the new parameter vector can guarantee the convergence of the algorithm and solve the unstable convergence problem in the traditional value iteration algorithm.Moreover, the forgotten factor is introduced to speed up the convergence of the algorithm.The experimental results of Grid World problem show that the proposed algorithm has good performance and robustness.