Perceptrons and shallow neural network
How to create a shallow neural network in Mathematica
Exploring the mathematics for neural networks: a case study of 9 parameters
Some familiarity with multivariable calculus and linear algebra is needed to understand the exploration made here: Mathematica notebook and pdf