An Overview and Comparison of Weighted Score Methods

Many studies of conceptual instruments, such as the Force Concept Inventory (FCI) and Force and Motion Conceptual Evaluation (FMCE), investigate weighted sum scores; however, most studies utilizing conceptual instruments use simple sum scores. In a simple sum score, all items are given the same weight, typically one, and item scores are summed together to form a total score. A comparison of different weighted sum and latent variable scoring methods with FCI and FMCE student responses was conducted to understand the impact of scoring on individual students and what these scores tell us about students' proficiencies. The simple sum score will be compared with latent ability scores from a range of unidimensional and multidimensional Item Response Theory (IRT) and Confirmatory Factor Analysis (CFA), and Cognitive Diagnostic Modeling (CDM) models. The use of predictive linear models to generate weights for the items in a weighted sum score will also be compared.