ABSTRACT
Basis functions are the fundamental building blocks for most approximation processes. There are infinitely many choices for basis functions such as polynomials, trigonometric functions, radial basis functions, etc. A central difficulty in learning input-output relationships directly from measurements lies in choosing appropriate basis functions and the choice of basis functions unfortunately depends on the characteristics of an unknown input-output map. The problem of choosing an appropriate basis function is difficult since one usually cannot say in advance how complex the input-output map will be, or specify its characteristics. Furthermore, one typically would not have the time or patience to search some handbook of known functions for a set that best represents what we want to study. Hence, we would like to choose building blocks which allow the adaptive construction of input-output maps efficiently and quickly. Fortunately, there is frequently a lack of uniqueness, in that many feasible choices exist for basis functions. So the challenge is not an impossible quest. Consider the Stone-Weierstrass theorem which gives one of the most remarkable results in the field of approximation theory: Qualitatively, there exists a sequence of polynomials that converge uniformly to any prescribed continuous function on a compact interval. This theorem was first stated by Weierstrass for polynomial approximations in 1-D spaces [7] and was later modified by Stone to generalize it for polynomial approximation in compact 2-D spaces [8-10]. For a general compact space, this theorem can be generalized to N dimensions as follows [11].
