Least Squares Methods

In all branches of engineering, various system processes are generally characterized by mathematical models. Controller design, optimization, fault detection, and many other advanced engineering techniques are based upon mathematical models of various system processes. The accuracy of the mathematical models directly affect the accuracy of the system design and/or control performance. As a consequence, there is a great demand for the development of advanced modeling algorithms that can adequately represent the system behavior. However, different system processes have their own unique characteristics which they do not share with other structurally different systems. Obviously the mathematical structure of engineering models are very diverse; they can be simple algebraic models, may involve differential, integral or difference equations or may be a hybrid of these. Further, many different factors, like intended use of the model, problem dimensionality, quality of the measurement data, offline or online learning, etc., can result in ad-hoc decisions leading to an inappropriate model architecture. For the simplest input-output relationship, the mapping from the state to the measurable quantities is approximated adequately by a linear algebraic equation:

Y¯ = a1x1 + a2x2 + ....+ anxn (1.1)

where Y and ai denote the measured variables and xi denotes the unknown parameters that characterize the system. So the problem reduces to the estimation of the true but unknown parameters (xi) from certain data measurements. When the approximation implicit in Eq. (1.1) is satisfactory, we have a linear algebraic estimation problem. The problem of linear parameter estimation arises in a variety of engineering and applied science disciplines such

as economics, physics, system realization, signal processing, control, parameter identification, etc. The unknown parameters of a system can be constant or time varying depending upon the system characteristics. For instance, the stability coefficients of the Boeing-747 for cruising flight are constant with respect to time while for the space shuttle these parameters change with time over the re-entry trajectory. All estimation algorithms fall into the category of either a Batch Estimator or a Sequential Estimator, depending upon the way in which observation data are processed. A batch estimator simultaneously processes all data in a single “batch” to estimate the optimum state vector while a sequential estimator is based upon a recursive algorithm, which updates the state vector as soon as each subset of new measurements arrive. Typically, the batch of measurements results in many more (m) equations of the form Eq. (1.1) than (n) the number of to-be-estimated parameters (xi). Due to their recursive nature, the sequential estimators are preferred for real time estimation problems but either can be used for static and dynamic estimation problems. The batch estimator results are generally more sensitive to model errors and may require some kind of post analysis if large model errors exist. In this chapter, the detailed formulation and analysis of classical batch and sequential least squares algorithms are presented.