ABSTRACT
Nowadays, with access to many open-source machine learning libraries like TensorFlow, it is quite easy to create a neural network without thinking about the underlying math. However, in order to fine-tune deep learning models or hyperparameters, along with right architecture selection, it is important to fully understand how neural networks and deep learning are working. Therefore, this chapter includes the basic mathematical concepts required to understand neural networks and deep learning in general where the concepts of linear algebra are especially important. It begins with the description of the steps for both forward and backward propagations including the underlying math for both concepts. Then, it discusses the various methods currently used to optimize deep learning models such as data scaling, network regularization and configuration, normalized initialization and more. [128 words]
