ABSTRACT
This chapter is concerned with various methods of estimation and their properties. The notions of mean square error and unbiasedness are discussed, and the Cramér–Rao inequality is established for smooth and stable families. Brief descriptions are given of the method of least absolute deviations, the method of least squares, and the method of moments. The Gauss–Markov theorem is stated and proved. However, the chapter focuses on the universality of maximum likelihood estimation, establishing a clear relation between model and method. Models are identified where the various methods above conform with maximum likelihood. In particular, it is shown that maximum likelihood estimation in regular exponential families lead to moment estimation with a canonical choice of moment functions.
