Achieving good fit to the Rasch Model through intentional item design: The case of the Quantum Computing Conceptual Survey

Compared to other item-response theory models for assessment validation, the Rasch model features a number of intrinsically desirable psychometric properties for assessment development within PER -- particularly the mathematical property of "specific objectivity" which helps guarantee metrological soundness of the instrument by mathematically separating item difficulty and student ability estimates. However, achieving acceptable fit to the Rasch model has sometimes been viewed as impractical in the PER community given the heavy constraints the model places on item development, motivating the adoption of more general IRT models that sacrifice these metrological properties. Drawing on 3 semesters of pilot data for the Quantum Computing Conceptual Survey, we demonstrate that it is indeed possible to achieve good fit to the Rasch model on a PER upper-division conceptual assessment through intentional item design, with implications for the development and validation of next-generation PER assessments moving forward.

This work was funded by the NSF GRFP and NSF grants nos. 2011958, 2012147, and 2143976.