ABSTRACT
This chapter reviews four modeling and inference techniques that are used in the general third-variable analysis methods described in the book. (1) Bootstrapping is a generic resampling technique to estimate the uncertainty of a statistic from the data. Two examples are shown: estimate the standard error of inter-quartile range and bootstrap interval of the slope in the linear regression model. (2) Elastic net fits regularized generalized linear regression models with ridge and lasso penalties to avoid overfitting and improve prediction accuracy. The ridge, lasso and elastic net are illustrated through a simulation example. (3) Multiple additive regression trees (MART) is a popular gradient boosting algorithm that uses tree as base learner. The tree and MART algorithms, as well as the interpretation tools for MART, are introduced and illustrated with examples in R. (4) Generalized additive model (GAM) fits additive models with flexible smooth functions. The smoothing splines, a popular nonlinear smoother used in GAM, is introduced. The GAM is illustrated through a simulation example with model interpretation and inference, as well as graphical presentation of the GAM with component plots. The sample R codes are provided for all the topics covered in this chapter.
