ABSTRACT
In this chapter, methods for regression function estimation are considered. The regression function is introduced as a conditional expectation. Then, the relation between a two-dimensional kernel density estimation and the Nadaraya-Watson estimator is presented. The local regression idea is explained and the bias reduction for local linear estimators is derived. Classes of restricted estimators like Ridge and Lasso are introduced as methods for estimating an approximation of the regression function. The base spline estimators and natural splines estimators are explained as least squares estimators in an approximate linear regression model. The relationship between the general ridge estimators and smoothing splines is shown. The projection idea of wavelet smoothing and the wavelet estimators are presented. Finally, the bootstrap confidence band is applied to the estimator of the first derivative of the regression function.
Using the methods presented in this chapter on a climate data set on the bivalve Methuselah, it is demonstrated that it is possible to detect climate changes. The underlying theoretical background of the various methods are explained. All methods are demonstrated with a series of examples and plots. Some of the most important R codes are given. The chapter ends with a list of problems useful for written exams.
