ABSTRACT
In this chapter, we study how to derive a model from data, e.g. fitting a curve to a series of measurements. The method of least squares is widely used, and gives simple, often linear algorithms. However, it should be employed with care, as it leads to the hidden assumption that error terms are Gaussian with equal variance. We also discuss a less known alternative called 1 norm minimization, which implicitly assumes that error terms have a Laplace instead of Gaussian distribution. The resulting algorithms may be less simple, but are often tractable, as they correspond to convex (rather than linear) optimizations, and the method is more robust to outliers or wrong distributional assumptions.
