ABSTRACT
The justification of the method of least squares requires no knowledge of the form of the distribution of the error vector apart from its mean and variance matrix, and the method can be applied without this further knowledge. The method of maximum likelihood is applicable mainly in situations where the true distribution on the sample space is known apart from the values of a finite number of unknown real parameters. The main justification of the method of maximum likelihood in terms of the criterion of minimum-variance unbiasedness is that it is possible to show that for large samples, maximum-likelihood estimators are nearly unbiased and have variances nearly equal to the Cramer–Rao lower bound. A similar adjustment is often possible in the large-sample theory for restricted maximum-likelihood estimates in the case where the information matrix is singular.
