DESIGN AND TRAINING OF BINARY NEURONS AND BINARY NEURAL NETWORKS WITH ERROR CORRECTING CODES

摘要

A data processing system having a binary neural network architecture for receiving a binary network input and in dependence on the network input propagating signals via a plurality of binary processing nodes, in accordance with respective binary weights, to form a network output, the data processing system being configured to train each node of the plurality of binary processing nodes by implementing the node function as an error correcting code (e.g. an r-th-order Reed-Muller code such as the 1-st order Reed-Muller code or cosets of the lst-order Reed-Muller code) function to identify a set of binary weights by channel decoding (e.g. fast Walsh-Hadamard transform algorithm) which minimize, for a given input to that node, any error between the node's output when formed in accordance with the node's current binary weights and a preferred output from the node and to update the weights of that node to be the identified set of binary weights. This training is performed without storing and/or using any higher arithmetic precision weights or other components.