ABSTRACT
Often, intuition comes first when building an algorithm. “Couldn’t we make this in that way?” This leads to many so-called heuristics, which do not stand the test of time, when their theoretical properties are undesirable. Only if there are favorable theoretical properties for an algorithm, is there a very good chance that it will be used in practice for a longer period of time. At times the algorithms first have to stand the practice test, since deduction of theoretical properties generally needs time. This typically leads to two different situations. Some algorithms are not as much used as they should be as long as, e.g., their optimality properties are unclear - like the Partial Least Squares (PLS) method below. Other algorithms are used frequently since practice has uncovered the not yet proven theory - like in the case of the Expectation Maximization (EM) algorithm below. We include this chapter in the book as a motivation to take up the challenge to deduce theoretical properties, since only through them the practical properties of the heuristics become clear. In the following, we will study two meanwhile established algorithms regarding their theoretical properties:
– the PLS algorithm regarding its optimality properties and
– the EM algorithm regarding its convergence.
