Discovering Faithful 'Wickelfeature' Representations in a Connectionist Network.

摘要

A challenging problem for connectionist models is the representation of varying-length sequences (e.g., the sequence of phonemes that compose a word). One representation that has been proposed involves encoding each sequence element with respect to its local context; this is known as a 'Wickelfeature' representation. Handcrafted Wickelfeature representations suffer from a number of limitations, as pointed out by Pinker and Prince (1988). However, these limitations can be avoided if the representation is constructed with a priori knowledge of the set of possible sequences. This paper proposes a specialized connectionist network architecture and learning algorithm for the discovery of faithful Wickelfeature representations -- ones that do not lose critical information about the sequence to be encoded. The architecture is applied to a simplified version of Rumelhart and MeClelland's (1986) verb past-tense model.