Shi Yin Hong
Author: Shi Yin Hong | Major: Computer Engineering & Computer Science | Semester: Spring 2024
I am Shi Yin Hong (Computer Engineering & Computer Science — College of Engineering). This semester, I have been working on my Emotional Support Conversations (ESC) project under the guidance of Dr. Susan Gauch. The goals of the ESC tasks are to reduce the user’s mental distress and elicit the user’s positive change in thoughts through natural interactive conversation.
Following, we propose a framework for interpretable ESC, termed Mind-to-Mind (Mind2), that endows an intuitive interpretability basis for the supportive dialogue generation of a large language model (LLM) with Theory-of-Mind (ToM). Our ESC framework utilizes prompt-based methodology with query expansion to integrate ToM-based discourse analysis into the process of supportive dialogue generation of LLM.
Mind2 is my second project working with Dr. Susan Gauch. Since Fall of 2023, I have been fortunate to continue contributing to NSF Data Analytics that are Robust and Trusted (DART) project under the guidance of Dr. Gauch. After closing my work on my previous DART project, Dr. Gauch guided me to solidify my research direction on Mind2 to focus on discourse analysis. Dr. Gauch’s constant encouragement and support have elevated my interest in natural language processing (NLP).
From part-of-speech-based exploratory experimental results and Dr. Gauch, I learned that the present semantic-based methods might not be the best approach to alleviate the ambiguity bottleneck in natural language generation on the ESC task. Hence, we decided to close the gap in ESC performance with a query expansion approach. To improve the interpretability of ESC dialogue generation, we perform ToM-based discourse analysis on the ESC dataset with prompt engineering. Specifically, we define a dynamic discourse context span that directs the LLM’s focus to perform discourse analysis as it analyzes conversations between the system and the user. Next, we prompt the LLM to extract context-specific ToM indicators concerning each speaker within the dynamic discourse context span from the ESC conversations. We then encapsulate the context-specific ToM knowledge into our psycholinguistic model to generate supportive language to accomplish the ESC task.
One challenge during the project was developing strategies to reduce the time-consuming manual data processing of intermediate experimental output. After directing the LLM extracted ToM knowledge, the ToM output needs to be transformed into the proper format before initiating model training. However, due to the instability of prompt-based LLM generation, not all utterances in the ESC dataset are guaranteed to be properly processed. Therefore, besides checking for the proper formatting of ToM knowledge extraction output, I also need to ensure not a single line of original utterance is skipped, repeated, or altered. This process requires manual effort.
I look forward to completing this project under the mentorship of Dr. Gauch. Without her patience and support, the process could not have been enjoyable. The next step for this closing project is to complete the draft of the paper. Overall, this research experience helps me understand the importance of perseverance and creativeness in research.