Fault tolerance in feedforward artificial neural networks.

摘要

Traditional computers architectures are extremely sensitive to even the most trivial and isolated hardware failures. Inspired by the impressive information processing capabilities of human brains, many researchers study artificial neural networks (ANN's) as an alternative to traditional symbolic processing algorithms. While the style of computation of ANN's is functionally more similar to the brains than are traditional computers, they are not as fault tolerant as is popularly assumed. In this thesis we discuss how ANN's can become fault tolerant and we investigate methods for automatically improving the fault tolerance of ANN's.; In the context of classification tasks, we explore an algorithm that, during training, randomly and temporarily introduces the types of faults that one might expect to occur. We have found this to be a simple yet powerful technique for reliably achieving fault tolerance in ANN's for a variety of tasks, including the recognition of handwritten characters. One benefit of the new method is that it can readily handle a variety of different faults such as stuck-at-max faults as well as double and triple faults. Furthermore, our technique can actually improve an ANN's ability to generalize and properly respond to new data.; For analog function approximation tasks, a more exact output is required and complete fault tolerance is harder to achieve. We present a technique that is able to improve fault tolerance by limiting the maximal contribution of each unit in the network to a small fraction of the total output signal. To achieve a large localized output signal, several Gaussian units are moved into the same location in the input domain and summed together. Since the contribution of each unit is small and equal in magnitude, there is only a modest error under any possible failure mode. We also investigate a technique that utilizes multiple networks each calculating the same function and then uses the combination of their outputs to determine the overall network output. This technique is quite fault tolerant even on difficult tasks such as sunspot prediction.