ABSTRACT
A minimal criterion for choosing a network topology is ‘denseness’. A network topology is dense if it contains networks that can come arbitrarily close to any functional relation between inputs x and outputs y. Within a chosen dense class of networks, the question is how large a network to choose. Here a minimal criterion is consistency. A method of choosing the size of the the network is consistent if, as the number of data or training examples grows large, all avoidable errors disappear. This means that the choices cannot overfit. The most widespread consistent methods of choice are variants of a statistical technique known as cross-validation.
