ABSTRACT
The key to inference in settings where the dimensionality of the parameter space is larger than the number of samples is to assume a parsimonious structure underlying the data generating process. While Bayesian hierarchical models offer a unified and coherent framework for structured modeling and inference, the properties of the posterior distribution and the exact role of the hierarchy have begun to be understood in the last few years. In this chapter, we review some of the recent theoretical works aiming to provide theoretical guarantees of discrete spike-and-slab priors. As one moves away from simple parametric models, understanding properties of a posterior distribution poses a stiff technical challenge and the prior choice (e.g., spike-and-slab) assumes a more fundamental role. Besides, full Bayesian computation using these hierarchical models often poses a significant computation bottleneck, necessitating the development of approximate Bayes methods such as variational inference.
