ABSTRACT
In this chapter, we present an overview of some basic optimization concepts and algorithms for convex problems, with an emphasis on aspects of particular relevance to regularized estimators such as the lasso. At the algorithmic level, we focus primarily on first-order methods, since they are especially useful for large-scale optimization problems. We begin with an overview of some basic optimality theory for convex programs, and then move on to consider various types of iterative algorithms. Although we limit our focus mainly to convex problems, we do touch upon algorithms for biconvex problems later in the chapter.
