Probability Concepts in Least Squares
The intuitively reasonable principle of least squares was put forth in §1.2 and em-ployed as the starting point for all developments of Chapter 1. In the present chapter, several alternative paths are followed to essentially the same mathematical conclusions as Chapter 1. The primary function of the present chapter is to place the results of Chapter 1 upon a more rigorous (or at least a better understood) foundation. A number of new and computationally most useful extensions of the estimation results of Chapter 1 come from the developments shown herein. In particular, minimal variance estimation and maximum likelihood estimation will be explored, and a connection to the least squares problem will be shown. Using these estimation techniques, the elusive weight matrix will be rigorously identified as the inverse of the measurement-error covariance matrix, and some most important nonuniqueness properties developed in §2.8.1. Methods for rigorously accounting for a priori parameter estimates and their uncertainty will also be developed. Finally, many other useful concepts will be explored, including unbiased estimates and the Crame´r-Rao inequality; other advanced topics such as Bayesian estimation, analysis of covariance errors, and ridge estimation are introduced as well. These concepts are useful for the analysis of least squares estimation by incorporating probabilistic approaches.