ABSTRACT
In this chapter, the authors describe an overview of the Minimum Description Length (MDL) principle for statistical inference proposed by Jorma Rissanen. The MDL principle follows the general philosophy for inductive inference put forth by algorithmic complexity theory. With the MDL principle, the authors restrict attention to a class of probabilistic models rather than all possible algorithms as in algorithmic complexity. In algorithmic complexity theory, the authors model a string of data by a smallest description of that data, or, in other words, the smallest program which outputs the data. However, it can be shown that no algorithm can find a smallest program which outputs the data. They consider MDL with the model class of Gaussian autoregressive processes. However, MDL was specifically designed to handle such nested model classes in a uniform manner. The authors also consider the maximum likelihood method of estimation.
