Quadratic Hadamard Memories I: Adaptive Stochastic Content-Addressable Memory.

摘要

A novel associative memory is discussed which overcomes the early saturation problem of Hopfield memories, without resorting to dilute state vectors or nonlocal learning rules. The memory uses a Bidirectional Linear Transformer (BLT) which transforms the bipolar input vector x into a vector u, which is a linear combination of Hadamard vectors. The matrix of the BLT is of Hebbian form, equal to the sum of outer products of stored vectors q to infinity and Hadamard vectors h to infinity. The Hadamard vectors are considered to serve as labels for the stored vectors. The BLT is followed by a Dominant Label Selector (DLS), which finds the dominant Hadamard component in the linear combination u, and returns the associated Hadamard vector to the BLT, to be processed in the BLT backstroke. This report deals with the DLS, which may be seen as an associative memory which stores N orthogonal bipolar vectors, the Hadamard vectors. A DLS architecture has been found which gives perfect associative recall of these stored vectors. The method involves a quadratic activation which, on account of a group property of Hadamard vectors, requires no more physical connections than a fully connected Hopfield memory of the same dimension. (kr)