ABSTRACT

This implies that the vectors L and a1 are orthogonal. We now want to find a linear combination a'X that has maximum variance among all normalized linear combinations L'X, which are uncorrelated with Z1. Using Lagrange multipliers ?, v we want to find the a that maximizes

Since

the maximizing a must satisfy

Since from (10.10) and we get from (10.13),

Since ?1?0, we conclude that v=0, and therefore from (10.12) we conclude that ? and a must satisfy (10.3) and (10.4). Thus it follows that the coefficients of the second principal component of X are the elements of the normalized characteristic vector a2 of S, corresponding to its second largest characteristic root ?2. The second principal component of S is

This is continued to the rth (r<p) principal component Zr. For the (r+1)th principal component we want to find a linear combination a'X that has maximum variance among all normalized linear combinations L'X, which are uncorrelated

with Z1,…, Zr. So, with

To find a we need to maximize

where ?, v1,…, vr are Lagrange multipliers. Setting the vector of partial derivatives

the vector a that maximizes is given by

Since from this

and we conclude from (10.17) and (10.18) that if ?i?0,

that is, vi=0. If ?i=0, Sai==?iai=0, so that the factor L'Sai in (10.16) vanishes. This argument holds for i=1,…, r, so we conclude from (10.17) that the maximizing a [satisfying (10.4)] is the characteristic vector of S, orthogonal to ai, i=1,…, r, corresponding to its characteristic root ?. If ?r+1?0, taking ?=?r+1 and a for the normalized characteristic vector ar+1, corresponding to the (r+1)th largest characteristic root ?r+i, the (r+1)th principal component is given by

However, if ?r+i=0 and some ?i=0 for 1=i= r, then

does not imply that In such cases replacing ar+1 by a linear combination of ar+1 and the ai for which ?i=0, we can make the new ar+1 orthogonal to all ai, i=1,…, r. We continue in this way to the mth step such that at the (m+1)th step we cannot find a normalized vector a such that a'X is uncorrelated with all Z1,…, Zm. Since S is of dimension p×p, obviously m=p or m<p. We now show that m=p is the only solution. Assume m<p. There exist p-m normalized orthogonal vectors ßm+1,…, ßp such that

Write B=(ßm+1,…, ßp). Consider a root of det(B'SB-?I)=0 and the corresponding ß=(ßm+1,…, ßp)' satisfying