ABSTRACT
This chapter concentrates on inferring the building blocks which are essential for an adequate model of the data. It explores the possibility of the data deciding the model space. The chapter considers the data generation process similar to the Chinese Restaurant Process where the number of clusters is unknown. It examines a group of neural networks known as autoencoders. Auto encoders are introduced as very simple network architecture and illustrated with an extensive example showing how neural networks create a model space for the data. The chapter illustrates the capability of neural networks by training a two layer neural network with three hidden neurons. Back propagation only passes through the hidden neurons, whose latent variables are non-zero. The chapter suggests a technique of inferring a model space with variable dimension. The chapter concludes with a Bayesian treatment of inferring the model space, illustrated by the Indian Buffet Process.
