ABSTRACT
In a broad sense, all iterative algorithms generate a sequence {xk} of vectors that describe the current state of the iterative process. The sequence may converge for any starting vector x0, or may converge only if the x0 is sufficiently close to a solution. The limit, when it exists, may depend on x0, and may, or may not, solve the original problem. Convergence to the limit may be slow and the algorithm may need to be accelerated. The algorithm may involve measured data. The limit may be sensitive to noise in the data and the algorithm may need to be regularized to lessen this sensitivity. The algorithmmay be quite general, applying to all problems in a broad class, or it may be tailored to the problem at hand. Each step of the algorithm may be costly, but only a few steps generally are needed to produce a suitable approximate answer, or, each step may be easily performed, but many such steps are needed. Although convergence of an algorithm is important, theoretically, sometimes in practice only a few iterative steps are used. In this chapter we consider several classes of operators that play important roles in optimization.
