ABSTRACT
Systematicity, the ability to represent and process structurally related objects, is a significant and pervasive property of cognitive behaviour, and clearly evident in language. In the case of Connectionist models that learn from examples, systematicity is generalization over examples sharing a common structure. Although Connectionist models (e.g., the recurrent network and its variants) have demonstrated generalization over structured domains, there has not been a clear demonstration of strong systematicity (i.e., generalization across syntactic position). The tensor has been proposed as a way of representing structured objects, however, there has not been an effective learning mechanism (in the strongly systematic sense) to explain how these representations may be acquired. I address this issue through an analysis of tensor learning dynamics. These ideas are then implemented as the tensor-recurrent network which is shown to exhibit strong systematicity on a simple language task. Finally, it is suggested that the properties of the tensor-recurrent network that give rise to strong systematicity are analogous to the concepts of variables and types in the Classical paradigm.
