Applying Computational Grounded Theory to Analyze Student Misconceptions Using Chatbot Interaction Data in a Modern Physics Course

An AI-powered chatbot, the UTA Study Buddy Bot, was deployed in a university-level Modern Physics course to assist students through peer-like, interactive problem-solving conversations. Over the course of the semester, more than 10 million tokens of student-chatbot dialogue were collected, providing a rich corpus for analysis of student thinking. To investigate conceptual understanding and common misconceptions, a Computational Grounded Theory (CGT) approach was applied. This process involved (1) pattern detection through natural language processing and unsupervised clustering of sentence-level vector embeddings, (2) pattern refinement through human-guided interpretation of emergent themes related to student reasoning and learning challenges, and (3) pattern confirmation using supervised models to evaluate the generalizability of the identified categories across the dataset. Preliminary analysis revealed recurring misconceptions in topics such as relativistic momentum and quantum energy levels, along with trends in the structure and phrasing of student inquiries. The findings demonstrate the potential of CGT to extract scalable, theory-aligned insights from chatbot interaction data and inform the design of more adaptive, AI-driven educational tools in physics instruction.