ABSTRACT
In this chapter, we look at a popular class of dynamical systems that are based on simplified models of how the brain processes information. These systems fall under the heading of artificial neural networks called spatio-temporal connectionist networks (SCNs). SCNs are particularly interesting because they are capable of representing all dynamical systems and all models of computation. This means that they are as powerful (in terms of what they can do) as all other dynamical and computational systems. In addition, these systems also typically embody adaptive mechanisms. That means they are capable of changing over time to suit a particular task. In particular, the systems incorporate algorithms that allow them to be trained to match a dataset of example behavior. Furthermore, they are able to both interpolate and extrapolate. This means that they can perform well, not only on the data that was used to train the system, but also on other data that has never been seen. In short, regularization can serve to address data points beyond those available during the adaptation process and SCNs are thus able to generalize to novel situations.
