Extracting background from a video sequence is the very first stage in many applications related to video surveillance: vehicle traffic control, intruders’ detection, suspicious objects, etc. The most usual approach to segment moving objects is known as background subtraction. This technique consists of building a reference model which represents the static background of the scene during a certain period of time. Multiple factors and events may affect the scene, making this first background subtraction a non-trivial task: sudden and gradual illumination changes, presence of shadows, or background repetitive movements (such as waving trees), among many others.