ABSTRACT

At this point in the book, you know how to train a neural network. Truth be told, though, there's some cognitive effort involved in having to remember the right execution order of steps like optimizer$zero_grad(), loss$backward(), and optimizer$step(). Also, in more complex scenarios than our running example, the list of things to actively remember gets longer.