Pierce developing an AI model and reviewing the output.

Author: Pierce Helton | Major: Computer Science | Semester: Summer 2022

During the Summer of 2022, I worked in the Computer Vision and Image Understanding lab of the CSCE department under Dr. Khoa Luu, professor of Computer Science and my research mentor. The focus of my project dealt with insects and how to measure their presence in nature. Insects and the tools used to manage them have a massive impact on crop management, public policy, and the economy as a whole, and having the information necessary to make important decisions is something that agriculturalists and scientists alike value. Our lab wanted to create an AI system capable of classifying and providing a real time count of insects to aid in the decision making process. To develop a system capable of performing these tasks, I worked on training AI models in classification and identification and fine tuning these models for deployment.

I first joined the lab during the Summer of 2021. At that time, I focused on data collection for the insect project and learning the basics of AI. I started with the insect project, as this allowed me to learn the most about the field while still being able to contribute to research. As my skills developed and my understanding of machine learning broadened, I began to perform more complex tasks like training AI and modifying existing projects. My research was submitted and published by the University of Arkansas Inquiry Journal, too. During Spring of 2022, Dr. Luu suggested that I apply for the research grant. I was enjoying the research and saw it as a valuable opportunity to learn more about the field, so I decided to apply.

One of the goals of the insect project was the development of a prototype system that could detect and identify insects. I had already completed the training of the detection model prior to receiving the grant; however, I still needed to train the classification model and deploy the two models onto our prototype hardware system. Training the classification model first required the creation of a dataset of detected insects produced by our detection model. This is because detection precedes classification in our bug system: an insect needs to be detected before it can be classified. After passing images from our custom bug dataset to the detection model, any incorrect or inaccurate samples were removed from the set of images. These images were given a label according to their species and used to train a Vision Transformer to perform identification. Modifications were made to an existing classification model to allow our custom data to be used. Upon completion of training, our model correctly identified the species of an insect 95% of the time. Although we were working with a family of insects, the Bombyliidae family, the results were a promising next step in the production of our functioning prototype and AI system. Once the classification model was complete, an additional program was created to allow images from our prototype to be detected and classified in real time. Each model needed to be converted from a development environment to one fit for deployment; reducing the computation time and allowing the models to work independently of each other is necessary for a fast and computationally efficient system. Separate threads were created for the models to increase throughput and efficiency. Next, we’d like to perform more field tests, identifying and fixing any issues found, and increase the number of insects that the classification model can recognize. After this, a production model can be developed to begin the deployment stage of the project. Additionally, we’d like to publish our research to AI and machine learning conferences such as Nature Machine Intelligence. Currently, these submissions are in development.

The project has taught me much in regards to software development and AI. During the research term, I learned more about how to work with machine learning models. Understanding the flow of information and the data types used are some of the more important concepts when working with AI, so getting more exposure to these ideas and skills was a valuable experience. In any field, encountering a problem can significantly slow progress, but working with a team and asking for help is crucial; sometimes another point of view is all you need. Most of the challenges I dealt with included errors in code, and my other lab members were always willing to help. Dr. Luu has been a great research mentor, too. He guides our lab, helping us with research and developing projects. He works with us to create AI solutions for various real world problems in any number of fields. I look forward to continuing working with Dr. Luu and the lab on this project in the future. Overall, having the opportunity to do practical work that I enjoy while helping others has been an exciting and rewarding experience.