ABSTRACT
Recent neural network simulations have shown that averaging over the outputs of a population of neural network estimates can lead to improved network performance [3, 14, 19, 24, 27, 28, 26, 33, 39]. The perspective we will take in this paper is one of developing a firm theoretical basis for this observed phenomenon. This analysis will enable us to understand why averaging improves performance and to identify a wide class of common optimization tasks for which averaging can be used to improve performance.
