ABSTRACT
This chapter delves deeply into the linear algebra of PCA and related methods. Several types of mean centering are carefully investigated as projections in various linear spaces including row, column, and full matrix spaces. Many aspects of PCA are revealed through a straightforward singular value representation. A more classical Gaussian likelihood derivation is also considered, along with computational issues including lower rank exact representation of high-dimensional data. Singular value representation ideas also give insights into two-block, i.e. multi-view, analysis methods such as Partial Least Squares and Canonical Correlation Analysis and how the modes of variation generated by each relate to the other. Brief introduction is given to the more sophisticated two-block analysis method of (Angle-based) Joint and Individual Variation explained, which gives modes of variation some of which are joint between blocks, and others that are individual describing variation in one block that is not present in the other. The latter method relies on the relatively recent linear algebraic method of Principal Angle Analysis.
