We present a new approach to the design of deep networks for natural languageprocessing (NLP), based on the general technique of Tensor ProductRepresentations (TPRs) for encoding and processing symbol structures indistributed neural networks. A network architecture --- the Tensor ProductGeneration Network (TPGN) --- is proposed which is capable in principle ofcarrying out TPR computation, but which uses unconstrained deep learning todesign its internal representations. Instantiated in a model for image-captiongeneration, TPGN outperforms LSTM baselines when evaluated on the COCO dataset.The TPR-capable structure enables interpretation of internal representationsand operations, which prove to contain considerable grammatical content. Ourcaption-generation model can be interpreted as generating sequences ofgrammatical categories and retrieving words by their categories from a planencoded as a distributed representation.
展开▼
