ABSTRACT
Multicollinearity – the possibility that the n × k observation matrix has rank less than k – has been a topic of concern in econometrics ever since the publication of Frisch′s monograph (1934).1 Two approaches have already been discussed in the previous chapter (Sections 2.7 and 2.8). In this chapter attention will revolve around the singular-value decomposition of a matrix; in the case of an n × k observation matrix X, its singular values are the positive square roots of the eigenvalues of X′ X. If some of these are very small (and because of rounding error, a computer cannot easily distinguish between “small” and zero), classical methods of computing least-squares estimates (e.g., the Gauss-Seidel procedure) tend to be highly inaccurate. The method of computing the singular-value decomposition and replacing small singular values by zeros produces much more reliable results. Interestingly enough, statistical theory reaches a similar conclusion: replacing small singular values by zeros (which amounts to approximating X by an n × k matrix X(l) of reduced rank, l) leads to estimators with lower mean-square error. This theory is the subject of the present chapter.
