ABSTRACT
It is expected that the reader of this chapter is familiar with the concept of the least squares solution of systems of linear equations. This chapter is the extension of Chapter 4, and is also a tutorial one. Consider the systems of linear equations
Ax = b
A is a real n by m matrix, b is a real n-vector and x is the solution m-vector. The residual vector for the system Ax = b, is given here by
r = b – Ax
It is known that A has an inverse A–1 if and only if A is a square nonsingular matrix. The solution of the system Ax = b, is given by x = A–1b. However, if A is a rectangular matrix, or a square singular matrix, Ax = b may have an approximate solution. A least squares solution x minimizes the L2 norm ||r||2 of the residual vector. The least squares solution is given by x = A+b, where A+ is known as the pseudo-inverse of A.
