The Beginnings
The beginning of my road to research began more than a year ago. Working as a consultant for a telecommunications company, I was first introduced to data analytics, process automation, and artificial intelligence. Working on automating customer support calls and other company processes, I witnessed firsthand the transformative power of AI and data transformation. Several months, I delved into the field of bioinformatics, learning the ins-and-outs of coding neural networks and image classification models.
Computer vision and its seemingly endless applications ultimately sparked my interest and passion for the field. While most everyone else was starstruck by the Generative AI boom, I saw computer vision proffer much more long-term impacts in a societal context. From the criminal justice system to the medical field, the applications of computer vision and neural network models were limitless. Following this passion, I created an image classification neural network model that diagnosed respiratory illnesses from input radiology scans. This project was largely the foundation and inspiration for my AP Research project.
A Code Snippet
Initial Research
Beginning AP Research, I knew that I wanted to build upon my knowledge on Python-based Neural Networks. I knew that were numerous parameters that went into coding Neural Networks and since this field is extremely novel, I wanted to expand upon the current research done on the subject - relatively little.
Researching the various parameters that are integrated in neural networks, I narrowed my research down to focusing on:
-
Hidden Layers
-
Number of Pooling and Convolutional Layers
-
-
Optimizers
-
Types of CNN Architecture
-
Activation Functions
-
Mathematical functions that activate the components of the neural network model to convert the input data into an end result (Classification).
-
I then shifted my focus to identifying what to explicitly measure in my project. The choices were initially narrowed to:
-
Model Accuracy
-
Training Speed
-
Model Overfit
Conducting thorough research on the scholarly conversation surrounding the topic, I found that most research focused specifically on model accuracy and how tuning the hyperparameters identified above would bolster or decrease accuracy. On the other hand, I determined that Model Overfit, despite being a significant problem, has been minimally researched.
Model Overfit refers to the phenomenon in which predictive models overtrain on input data (or in essence memorize input data) with no underlying pattern recognition.
In a social context, with computer vision models being increasingly incorporated into fields such as criminal justice, the potential that models overfit on inherently biased data presents numerous implications, such a discriminatory criminal profiling. Due to the tremendous implications attributed to this marginally researched topic, I chose to delve specifically into model overfit for my project.
Conversely, out of the research currently done on mitigating model overfit, the vast majority centers on deriving the causal relationship between the phenomenon and the types of CNN Architecture used. However, there was negligible research done behind the relationships between Model Overfit and the types of Activation Functions used - as well as the number of hidden layers used.
Therefore, I chose to center my AP Research project on recording how activation functions and the number of hidden layers used function together to impact levels of Model Overfit and designed an experimental procedures stratified on the number of hidden layers used, measuring the extent to which different activation functions impact levels of overfit.
These three articles laid the groundwork for my project
From Research, I determined the most common activation functions used as:
Sigmoid
TanH
ReLU
Using Python, I have started to graph the various activation functions I will be using throughout my Research Project using MatPlotLib. This will be instrumental to commence constructing the model I will use for my experiment.
Next Steps
As my research project has been approved, I am currently in contact Professor Tracy Dobbs at ACC as well as computer science faculty at Colorado School of Mines for mentorship and review of my overall project and its overarching potential. I will be presenting my elevator pitch to Professor Dobbs on December Fifth.
After presenting my research, I intend to begin constructing the actual model I will use for my project. This involves importing the various python packages necessary and importing the data needed for my project.
I will be using the MINST database for my input data. This database has widely been used in current research involving image classification neural network models.