ABSTRACT
The desire to predict the future and understand the past drives the search for laws that explain the behavior of observed phenomena; examples range from the irregularity in a heartbeat to the volatility of a currency exchange rate. If there are known underlying deterministic equations, in principle they can be solved to forecast the outcome of an experiment based on knowledge of the initial conditions. To make a forecast if the equations are not known, one must find both the rules and the state of the system. This chapter focuses on phenomena for which underlying equations are not given; the rules that govern the evolution must be inferred from regularities in the past. For example, the motion of a pendulum or the rhythm of the seasons carry within them the potential for predicting their future behavior from knowledge of their oscillations without requiring insight into the underlying mechanism. We will use the terms “understanding” and “learning” to refer to two complementary approaches taken to analyze an unfamiliar time series. Understanding is based on explicit mathematical insight into how systems behave, and learning is based on algorithms that can emulate the structure in a time series. More specifically, we use information theory to obtain some insights into the system, and we use neural networks to build models that emulate the system’s behavior. In both cases, the goal is to explain observations; the important related problem of using knowledge about a system for controlling it in order to produce some desired behavior is discussed in Chapter 11 in this volume by Narendra and Li.
