Information Storage Capacity of Connectionist Systems: The Linear Associator.

摘要

The information-storage capacity of hetero-associative memory systems is addressed. The associator can be treated as an M-ary symmetric channel when M associations are stored. The maximum number of associations storable is bounded asymptotically by N/2 where N is the number of connection weights. Storage efficiency is bounded by M/N so that it never exceeds 1/2. Information capacity degrades as inter-vector correlations increase and also when classification tasks are performed. The correlation effect is most pronounced in high-dimensional systems storing a large number of associations. Keywords: Artificial intelligence, Connectionism, Information theory.