Four Capacity Models for Coarse-Coded Symbol Memories.

摘要

Coarse-coded symbol memories have appeared in several neutral network symbol processing models. In order to determine how these models would scale, one must first have some understanding of the mathematics of coarse-coded representations. We define the general structure of coarse-coded symbol memories, and discuss their strengths and weaknesses. Memory schemes can be characterized by their memory size, symbol-set size and capacity. We derive mathematical relationships between these parameters for various memory schemes, using both analysis and numerical methods. Finally, we compare the predicted capacity of one of the schemes with actual measurements of the coarse-coded working memory of distributed connectionist production system (DCPS), Touretzky and Hinton's distributed connectionist production system. Keywords: Connectionist models; Distributed representations; Memory devices; Coarse coding; Artificial intelligence; Symbol processing. (jg)