ABSTRACT
This chapter offers a fly-by of classical methods for response surfaces, focusing on local linear modeling. Both steepest ascent and ridge analysis compartments share an intimate relationship with experimental design and benefit from a human-in-the-loop throughout stages of iterative refinement. That’s in stark contrast to Bayesian optimization methods which are rather more hands-off, intended for autonomous/automatic implementation. The method of steepest ascent involves a first-order model, sometimes fitted with interactions. Ridge analysis is steepest ascent applied to second-order models. Since second-order models are generally undertaken when the practitioner believes that s/he is quite near the region of the optimum, ridge analysis is typically entertained only in such settings. However, results from experiments may reveal a stationary point well outside the design region, contradicting that belief. Optimization and analysis of computer simulation experiments, nonlinear regression in spatial statistics and machine learning, are increasingly nonparametric.
