This chapter reviews the data stream classification and novelty detection and discusses related works in semisupervised clustering which is an important component of authors’ data stream classification technique with limited labeled data. It describes the general approach to data stream classification and Ensemble classification, Novel class detection. The chapter explores Data stream classification with limited labeled data. Data stream classification is a challenging task because of several important properties of a data stream: infinite length, concept drift, concept evolution, and limited labeled data. The authors’ proposed novel class detection technique Enhanced Classifier for Data Streams with novel class Miner (ECSMiner) is related to both data stream classification and novelty detection. ECSMiner also applies the ensemble classification technique. Outliers are the byproduct of intermediate computation steps in ECSMiner algorithm. ECSMiner offers a more practical solution to the novel class detection problem, which has been proved empirically.