Applying machine learning to understand student thinking about measurement in quantum and classical mechanics

The effect of instruction on student understanding of measurement uncertainty across quantum and classical contexts is not well understood. Research-based measures of student understanding, however, rely on open-ended survey questions, which are challenging to apply at scales necessary for evaluating instructional interventions. We are using machine learning tools to automate content analysis for open-ended survey questions as part of our ongoing work exploring student understanding of measurement uncertainty and how these ideas change as a result of course-level interventions. I discuss how we are applying both supervised and unsupervised algorithms to efficiently explore and analyze the various open-ended datasets from our survey. These datasets include student descriptions of the sources of uncertainty they expect in a given experimental scenario as well as how and why they expect distributions to change if more or better data are collected. I ground the analysis in uncertainty quantification methods in machine learning. Implications of this work include not only improving students' understanding of measurement uncertainty but also improving research methods to make trustworthy conclusions when using machine learning as a tool for analysis.