ABSTRACT

This chapter reviews classical and popular deep learning algorithms and explains how they can be used for feature representation learning. It discusses how they are used for hierarchical and disentangled representation learning, and how they can be applied to various domains. The chapter introduces the basic building blocks of Convolutional Neural Network (CNN) and how CNNs can be used for feature learning. Word embedding and recurrent neural networks are the state-of-the-art deep learning models for natural language processing tasks. Restricted Boltzmann machine can be stacked and trained in a greedy manner to form socalled Deep Belief Networks (DBN). DBNs are graphical models which learn to extract a deep hierarchical representation of the training data. An autoencoder is a neural network trained to learn latent representation that is good at reconstructing its input. Recurrent neural networks are powerful concepts that allow the use of loops within the neural network architecture to model sequential data such as sentences and videos.