Author: Carson Molder, Engineering, Computer Science, 2020.

Over the past semester, I have spent many hours developing a tool that uses artificial intelligence (AI) to draw new insights from images in expert domains such as medicine. Traditional AI systems look at images as a whole to determine what the contents are—such as analyzing images of pets to determine if they contain a cat or a dog. Such systems have proven useful in many domains that require advanced degrees or special expertise, such as diagnosing ailments from chest X-rays. [1] 

My research adapts a different approach to image analysis using artificial intelligence. Instead of looking at images as a whole, it would be interesting to look at small patches from these images to see what kinds of information can be extracted. For example, taking patches from a photograph of a tropical beach might help a computer to learn to discern different textures such as gritty sand, foamy waves of water and waxy palm leaves. Materials such as sand, water, foliage, glass, plastic and anything generally identifiable in the environment are known as natural materials. Other research has already shown that an AI can be trained to identify these materials accurately among a diverse set of images. [2] 

Last August, I began helping my advisor Dr. Justin Zhan start a new data science lab in the Computer Science and Engineering department. At that time we decided to explore the possibility of an AI system that can identify materials not in natural scenes, but in medical images like X-rays and MRIs. Such a system could, for example, help doctors diagnose brain tumors by showing where a tumor may exist on an MRI scan. Ideally, such a system could help doctors determine a diagnosis more quickly, freeing up time to see more patients. 

For the next three months, I spent extensive time learning about AI concepts and began writing some basic code to understand how they work. Once I felt confident in my ability to write AI code, I began working with Ben Lowe, a Ph.D. student, to develop a material recognition AI with the goal of identifying brain tumors. Starting in last December, I began working full-steam ahead on this project, spending many hours in the lab writing code that became a part of my final project. Numerous hours were spent configuring programs, modifying code and tweaking parameters to make my model as accurate as possible. 

When the COVID-19 pandemic hit Arkansas and the university had to close its labs, the Honors College Research Team Grant was invaluable to my research. Most AI systems use a computer’s graphics card since it can perform small calculations far more quickly than a CPU can. Unfortunately, my computer at home had an older graphics card that could not load my entire AI model without creating an error. Using the funds from my grant, I purchased a powerful, state-of-the-art graphics card that not only saved time but allowed me to make my model even more detailed. 

Near the end of April, we realized our model can be extended beyond medicine to many different fields where expert knowledge is required to annotate data. For example, our system could be used to identify rock types in a photograph by learning from images that were previously annotated by geologists. Our underlying model stayed the same, but we expanded the idea to a wider range of fields. At the time of writing this blog post, my paper about our material recognition AI is nearly complete and is currently being prepared for an application to be submitted in an IEEE research journal. 

This fall, I will be continuing my research in Dr. Zhan’s lab on a new project. I have yet to select a final topic for my honors thesis, but that will likely be determined over the summer. In recent weeks, Dr. Zhan, Ben and I have started working with Dr. Shilpa Iyer in the Biology department to create an AI that analyzes microscope images of mitochondria in human cells to determine potential diseases and conditions. This research will continue over the summer and into the fall semester. 

Overall, my research funded by the Honors College Research Team Grant has greatly enriched my view of computing and technology. My efforts to learn many complex, math-heavy topics within the field of artificial intelligence has helped me discover the beauty of computation—that rich, complicated systems can be created from solving equations and controlling the flow of electrons. It has turned the “black box” of machine intelligence into an understandable, intricate design where every part serves a purpose. 

Figures 

Some samples of how our material recognition AI evaluates brain tumor scans [3]. The leftmost column highlights where tumors actually are, and the two columns to the right demonstrate our system’s ability to identify these tumors. Although there is some noise in the system’s analysis, it works quite well at identifying tumors when they are present. 

References 

1. Irvin, Jeremy, et al. “CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison.” Association for the Advancement of Artificial Intelligence, Jan. 2019, Honolulu, Hawaii, 2019. 

2. Schwartz, Gabriel, and Ko Nishino. “Recognizing Material Properties from Images.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019. 

3. Cheng, Jun. “Brain tumor dataset.” Figshare, 2017. https://figshare.com/articles/brain_tumor_dataset/1512427/5