Automating motivational coding of CUWiP responses using natural language processing approaches

As a component of the evaluation of the Conferences for Undergraduate Women in Physics (CUWiP) we have been investigating motivating factors for women that choose to major in physics. Previous work, headed by Franklin, developed a hand-coding scheme for motivation based on theories of self-efficacy and expectancy value. This coding identified motivational factors women expressed for entering physics: Physiological/Emotional, Vicarious Experience, Mastery Experience, Intrinsic Value, Intrinsic Value (Astronomy), Social Intrinsic Value, Utility Value, Media Triggered Intrinsic Value, Event Triggered Intrinsic Value, Attainment Value, Social Persuasion. Franklin et al, also identified costs associated with majoring in physics: Cost (Emotional), Cost (Task Effort), Cost (Loss of Valued Alternatives), and Cost (Outside Effort). This coding was applied to a year's worth of CUWiP responses to a question asking the attendees their motivation for choosing physics. A total of 2125 responses were hand coded. We are in the process of developing a machine learning approach, using natural language processing, to identify the motivational codes of these responses from other years of CUWiP. Preliminary efforts have utilized word vectorizers and binary logistic regression methodologies found within the sklearn python package. Logistic regression application provides 80% or higher accuracy in identifying the codes.