Information measure of knowledge extracted from neurons as a tool for analyzing Boolean learning in artificial neural networks

摘要

Neural network research depends on convergence and learning characteristics traditionally derived from error measures. Recent studies have attempted more direct extraction of knowledge from a network, but they require control of the training process. We show how Boolean information may be extracted and measured efficiently from a neuron's internal representation. The information measure is compared with training error by observing twelve-input three-layer networks during multiple training runs. The experiment indicates a natural termination point for training by backpropagation.