Spectral feature selection tries to select features that are consistent with the target concept via conducting [171]. In this chapter, we present several univariate formulations for spectral feature selection, and analyze the properties of the presented formulations based on the perturbation theory developed for symmetric linear systems [38]. We also show how to derive novel feature selection algorithms based on these formulations and study their performance. Spectral feature selection is a general framework for both supervised and unsupervised feature selection. The key for the technique to achieve this is that it uses a uniform way to depict the target concept in both learning contexts, which is the sample similarity matrix. Below, we start by showing how a sample similarity matrix can be used to depict a target concept.