Redesigning the Commonly Used Legacy Instruments: Using Evidence Centered Design to Develop Valid, Equitable, and Flexible Items Assessing Conceptual Understanding of Kinematics and Dynamics for Introductory Physics

Commonly used legacy instruments, such as the Force Concept Inventory (FCI) and the Force and Motion Conceptual Evaluation (FMCE), have shown some serious flaws including substantial psychometric problems which threatening their reliability and validity, and demographic biases which make them inaccurate for some underrepresented student populations. Even with these flaws, these instruments have been vital to the development of Physics Education Research (PER) as a discipline; however, while the teaching of physics has evolved, the formative assessments researchers, instructors and, more broadly, physics departments use to evaluate their classrooms have not. It is critical that these instruments be redesigned in order to continue to advance the teaching and learning of physics for all students. In this talk, we will discuss the project goals and development strategy of our new NSF award (#2235518) intended to design a set of valid, fair, and flexible tool assessing conceptual understanding of kinematics and dynamics for introductory physics courses. Constructed within Evidence Centered Design (ECD), the new assessment items will be grounded in learning and measurement theory, based on a construct-centered and community-based approach. The flexible instrument, the Kinematics and Dynamics Assessment (KDA) will include multiple scales measuring foundational topics within kinematics and dynamics organized into subscales which instructors can utilize to build their own classroom assessment. The library of items will be extensively validated to ensure superior psychometric properties and instrumental fairness for women, underrepresented minority students, and first-generation college students.