Encouragement of AI in Introductory Mechanics: Sentiment Analysis and FCI Impacts

The application of AI in education has great promise, though there is still much to understand about student perceptions on AI and its impacts on performance. A one-semester controlled study at the United States Military Academy¹ examined how students responded to unstructured encouragement to use AI chatbots in introductory mechanics. An experimental group (n=43) was encouraged to use AI as a study tool for physics while a control group (n=23) received no encouragement but had no restritions on usage. Surveys at weeks 5, 10, and 15 showed student sentiment towards AI as a useful study tool began higher in the experimental group yet both populations saw relative increases over the semester (58% control, 22% experimental). The most widely used bot was ChatGPT (79% of respondents). Likert data showed that AI was perceived to be more helpful than the textbook and student's sentiment towards the benefits of learning with AI increased significantly based on a two sample t-test (p=0.028). Self-reported AI usage increased in both groups over the semester but regression analysis showed no statistically significant effect on FCI gains (R²=0.0078, p=0.504), likely due to the unstructured nature of the intervention. Students reported the greatest barrier to AI use was academic integrity concerns (70% of responses) followed by a lack of trust in the AI output (49% of responses). This study confirms that students find AI tools useful, but more structured interventions are needed to assess its full impact on learning.