ABSTRACT
This chapter examines how Graphics Processing Units (GPUs) can be used in deep learning. Deep learning is an emerging machine intelligence algorithm based on artificial neural networks (ANNs). ANNs were proposed to be computational models of neurological systems; they were designed to "learn" performing a certain task by mimicking the way a brain learns. ANNs are formed by multiple layers of "neurons" connected to each other, which create a network that takes input data, processes the data in each layer, and creates an output in the final layer. Activation functions in a neuron are implemented to introduce non-linearity to the network. The ANNs, in which the connections between their neurons do not make a cycle, are called Feed-Forward Neural Networks. cuDNN is a library that provides support for implementing deep networks on GPUs. Although using a GPU instead of a CPU already provides significant speedup for training a network, using cuDNN improves that speedup further.
