Resonator Networks, 1: An Efficient Solution for Factoring High-Dimensional, Distributed Representations of Data Structures

摘要

The ability to encode and manipulate data structures with distributedneural representations could qualitatively enhance the capabilities of traditionalneural networks by supporting rule-based symbolic reasoning,a central property of cognition. Here we show how this may be accomplishedwithin the framework of Vector Symbolic Architectures (VSAs)(Plate, 1991; Gayler, 1998; Kanerva, 1996), whereby data structures areencoded by combining high-dimensional vectors with operations thattogether form an algebra on the space of distributed representations.In particular, we propose an efficient solution to a hard combinatorialsearch problem that arises when decoding elements of a VSA data structure:the factorization of products of multiple codevectors. Our proposedalgorithm, called a resonator network, is a new type of recurrent neuralnetwork that interleaves VSA multiplication operations and patterncompletion. We show in two examples-parsing of a tree-like data structure and parsing of a visual scene-how the factorization problemarises and how the resonator network can solve it. More broadly, resonatornetworks open the possibility of applying VSAs to myriad artificialintelligence problems in real-world domains. The companion articlein this issue (Kent, Frady, Sommer, & Olshausen, 2020) presents a rigorousanalysis and evaluation of the performance of resonator networks,showing it outperforms alternative approaches.