Lab setup with UR robot.
Name: Taisei Hanyu | Major: Computer Science | Semester: Fall 2024
Hello, my name is Taisei Hanyu, and I am majoring in Computer Science, particularly Computer Vision. During the past semester at Electrical Engineering and Computer Science, I have been working in Artificial Intelligence and Computer Vision (AICV) lab under Dr. Ngan Le supervision. We studied about finding better ways to enable robots to understand and interact with the real world. Our primary goal was to build an efficient 3D semantic reconstruction system suited for real-time robotic manipulation. One practical application involves using natural language instructions to guide tasks such as cleaning a tabletop or preparing a meal, which can expand the capabilities of service robots in domestic settings.
Robot Learning is a rapidly growing field that connects machine learning and robotics, and it has always excited me to bring advanced AI techniques into physical systems. With that in mind, I joined Dr. Le’s AICV lab in my freshman year, gaining experience in cutting-edge AI methods and specialized skills. Motivated by the success of large language models like ChatGPT, I became interested in how software improvements align with hardware requirements. Around that time, Dr. Le began a new research project focused on 3D semantic scene reconstruction and robotic manipulation, an area that particularly caught my interest. After discussing potential directions, I decided to concentrate my research on this topic.
Last semester, I collaborated with Kashu Yamazaki, a research assistant in the AICV lab, and we published a paper at the 2024 International Conference on Robotics and Automation (ICRA) on efficient and queryable 3D representations, with me as the second author. Presenting our work at the poster session allowed me to introduce our research to leading experts and gain ideas into the field. This semester, I built on that foundation by extending our methods to support tabletop robotic manipulation. Since no one else in the lab was working on robotic hardware, I had to create and configure a platform from scratch. One of the most challenging aspects was developing a unified system for controlling robotic manipulators and grippers under varying conditions, including different protocols, libraries, and hardware implementations. My limited background in robotics made this especially demanding, but I tackled the problem by studying core theories and mechanisms instead of relying solely on trial-and-error approaches. This approach not only helped me design a robust software system in the end, but it also gave me a valuable starting point as a robotics researcher.
Dr. Ngan Le and Kashu Yamazaki made invaluable contributions to my research project. Dr. Le treated me with the same respect and expectations as her other graduate students, offering strong support through regular meetings where we discussed my progress and research direction. Kashu was always willing to discuss implementation details and share best practices that could adapt to future changes. Their guidance proved highly beneficial, helping me refine my work at every stage.
Finally, I would like to acknowledge the Honor College Research Grant for making it possible to explore this topic. The skills I have gained in implementation and research methodology will be valuable both in my studies during the 2025 academic year and as I plan to pursue graduate research in this field.