Neural networks (NN) were originally developed as an attempt to emulate the human
brain. The brain has about 1.5×1010 neurons each with 10 to 104 connections called synapses. The speed of messages between neurons is about 100 m/sec which is much
slower than CPU speed. Given that our fastest reaction time is around 100 ms and
neuron computation time is 1-10 ms, then the number of steps must be less than 100.
This is inconceivably small for a sequential computation, even in machine code;
therefore, the brain must be computing in parallel.