A Neural Network implemented from scratch based on the following tutorial: https://towardsdatascience.com/how-to-build-your-own-neural-network-from-scratch-in-python-68998a08e4f6
It is a 3-layer Neural Network. The input has 3 features, the first hidden layer has 4 nodes, the second has 3 nodes, and the output layer has 1 node.
There is an option to use Weight Decay regularization. You must set reg = True and define the reg_parameter which is the lambda value of the weight decay formula.
What is missing:
- Add bias
- Use it for a real dataset
- Implement a train function
- Maybe create a function to automatic add more layers
- Use a different activation function in the first two layers
- Plot the loss function curve