ABSTRACT
Linear Regression, or Ordinary Least Squares (OLS) Regression, is one of the most widely used and established regression methods available and is a frequent choice for modelers because of its underlying simplicity and ease of interpretation. This chapter contains a step by step walkthrough of the ordinary least squares method using an example related to student test scores. Many foundational regression concepts are introduced such as residual errors, fit, and goodness of fit. The interpretation of linear models is discussed in detail, including coefficients, confidence intervals and coefficient significance. Model diagnostics are reviewed as a way to ensure that a linear regression model is an appropriate choice for the data. Simple extensions of linear regression to include interaction terms and polynomial terms are discussed as ways to improve model fit. Other important concepts related to model design are introduced, such as input variable selection, dummy variables, collinearity and multicollinearity.
