ABSTRACT

How might simple recurrent networks represent co-occurrence relationships such as those holding between a script setting (e.g., “clothing store”) and a script item (”shirt”) or those that specify the feature match between the gender of a pronoun and its antecedent? These issues were investigated by training a simple recurrent network to predict the successive items in various instantiations of a script. The network readily learned the script in that it performed flawlessly on the non-variable items and only activated the correct type of role filler in the variable slots. However, its ability to activate the target filler depended on the recency of the last script variable. The network's representation of the script can be viewed as a trajectory through multidimensional state space. Different versions of the script are represented as variations of the trajectory. This perspective suggests a new conception of how networks might represent a longdistance binding between two items. The binding must be seen as not existing between an antecedent and a target, but between a target item and the current global state.