ABSTRACT
Regression developed from interpolation and is introduced as building a model of the process generating the data where the building blocks are chosen. Principal component regression ensures small correlation between building blocks, while Partial Least Squares emphasizes strong correlation with the data at the same time. The chapter describes the relationship between variables. Regression is related to curve fitting curve fitting, interpolation, and data prediction. The chapter shows that the variance in the mean squared test error increases significantly with increasing complexity that is the degree of polynomial. The MATLAB code generates the data, performs polynomial Ordinary Least Squares regression for different degrees of polynomials, and plots both mean squared training and test error together with their variances. The chapter concludes with taking a probabilistic viewpoint in form of Bayesian regression leading to Gaussian processes.
