Project Overview
This project will aim to quantify the relationship between activation functions (mathematical functions used in pattern recognition in neural network algorithms) and other hyperparameters such as the number of hidden layers to minimize Model Overfit ( a phenomenon in which the AI algorithm completely memorizes training data - including any external noise and variance - without recognizing the underlying patterns behind the dataset (lack of effective pattern recognition).
This project will specifically explore image classification models, recording the impact the activation function used will have on levels of model overfit, while being stratified by the number of hidden layers.
Project Details
How do the use of different activation functions impact the presence of model overfit in neural networks when different numbers of hidden layers are used?
Question
?
How do the use of different activation functions impact the presence of model overfit in neural networks when the number of hidden layers used are altered?
Hypothesis
Since activation functions such as sigmoid and TanH have a vanishing gradient near the origin (vertical tangent), parametric functions such as reLU would be more effective in minimizing model overfit over the must number of model hidden layers.