ABSTRACT
The objective of any kind of statistical analysis of given data is to consider a set of models for the process or phenomenon associated with the data and find the model that is best supported by it. Optimization problems arise in a wide array of fields, of course, and not just in statistical analysis. Statistical regression, being a subset of density estimation, also inherits parametric and non-parametric approaches. However, there is considerably more ambiguity in the literature on the distinction between the two as far as regression problems go. Non-parametric regression can also include the case of models that are parametric but have a large number of parameters that allow greater flexibility in capturing complexity in the data. The minimization problem illustrates the role of optimization in statistical regression. A different statistical analysis problem, that of hypotheses testing, is concerned with deciding which among two sets of models best explains the data.
