A day of AICV lab

Author: Taisei Hanyu | Major: Computer Science | Semester: Spring 2024

Hello, my name is Taisei Hanyu and I am majoring in Computer Science, especially Computer Vision. During the past semester at Electrical Engineering and Computer Science, I have been working in Artificial Intelligence and Computer Vision (AICV) lab under Dr. Ngan Le with one of her research assistants, Kashu Yamazaki. We studied about finding better ways to enable robots to understand and interact with the real world. In particular, our goal was to create 3D semantic reconstruction efficiently enough to be used in real-time applications. A practical example of our research is improving robot navigation using natural human language, transforming simple commands into complex tasks like instructing a robot to specifically clean under the living room chair.

I have long had a strong passion for incorporating cutting-edge AI technology into robots with physicality. Therefore, I joined Dr. Le’s AICV lab from my freshman year, where I spent time learning cutting-edge AI techniques and acquired many specialized skills. Last summer, inspired by the success of large language models such as ChatGPT, I became deeply interested in how software advancements are inherently linked to hardware capabilities. At that time, Dr. Le and Kashu were about to start a new research topic on 3D semantic scene reconstruction, which caught my interest. After various discussions, I decided to make this my research topic.

I have faced significant challenges in handling hardware throughout my research. This semester, much of my time has been devoted to implementing the software I developed last year onto an actual robot. At that time, there were no others in the lab working on robot research. Kashu and I had to figure out everything from how to power the robot on and off by ourselves. Finding even the user documentation for the older robots was a struggle. Moreover, we were using a TurtleRobot2 (similar to a Roomba) to conduct experiments with the proposed approach. However, we faced issues where the robot would incorrectly detect obstacles where there were none, and it could not navigate its route properly. Upon reviewing the program, we found that it assumed the floor to be ‘ideally flat’. In reality, the floor was quite uneven! What was invisible minor bumps to us felt like significant obstacles to the robot. It might sound strange, but having dealt with ideal simulators, I had completely overlooked the real-world unevenness.

Dr. Ngan Le and Kashu Yamazaki played invaluable roles in my research project. Dr. Le treated me with the same respect and expectations as other graduate students and provided valuable support through regular meetings where we discussed my progress and the direction of my research. Kashu was always willing to discuss implementation details and strategies with me. He shared best practices that were designed to anticipate future changes in implementation based on his experience. This knowledge proved to be extremely valuable and was a significant support in many aspects of my work.

Attending the International Conference on Robotics and Automation (ICRA 2024) was a highlight of my semester. The conference, known for its rigorous acceptance rates of 30% to 40%, provided a platform to present our work and engage in enriching discussions on future research directions in the integration of computer vision and robotics.

Thanks to the Honor College Research Grant, I was able to explore this interesting topic. The skills I developed in implementation and research methodology throughout the project will be invaluable for my continued studies in Fall 2024 and as I plan to pursue graduate studies in the same field.