Sparse Matrix Factorization: Dictionary Learning and Beyond

In this chapter, we will focus on sparsity in the context of matrix factorization, and consider an approximationY ≈ AX of an observed matrix Y by a product of two unobserved matrices, A and X. Common data-analysis methods such as Principal Component Analysis (PCA) and similar techniques can be formulated as matrix factorization problems. This formulation is also central to a highly popular and promising research direction in signal processing and statistics known as dictionary learning, or sparse coding (Olshausen and Field, 1996), discussed in more detail below.