A Robust Self-Constructing Normalized Gaussian Network for Online Machine Learning

摘要

In this thesis, I aim to improve the robustness and applicability of Normalized Gaussian networks (NGnet) in the context of online machine learning tasks. A challenging problem in online machine learning is limited domain knowledge provoked by restricted prior knowledge, while additional domain information are obtained only sequentially over time. The limited domain knowledge makes the application of artificial neural networks (ANN) more difficult, where major challenges are negative interference and selection of an accurate model complexity. I consider these challenges for the NGnet. NGnets belong to a group of ANNs with a certain grade of robustness against negative interference due to the local properties of their network architecture. Yet, further improvements of robustness are possible in regard to its learning algorithm and model complexity selection. A recently proposed learning algorithm with localized forgetting provides robustness against negative interference, but can be further improved as it is not applicable over the full numerical range of an implied discount factor. Also, dynamic model selection was yet to be considered. In this thesis, I revise the learning algorithm with localized forgetting and adapt dynamic model selection to it in a self-constructing manner. In addition, I propose localization of some of the model selection mechanisms for improved robustness and add a new merge manipulation to deal with model redundancies. The effectiveness of the proposed method is compared in several experiments with earlier NGnet learning approaches. The proposed method possesses robust and favorable performance for the tested learning tasks, showing it to be a robust alternative when applied to online learning tasks with proneness to negative interference.