Using Concentration Analysis on Preliminary Fluids Conceptual Evaluation Data

Standardized conceptual learning assessments (SCLAs) attempt to measure individuals' conceptual understanding of a phenomenon through individual responses to several distractor driven multiple-choice items. When an individual encounters each item, researchers assume that one of several mental models are triggered. Concentration analysis is one justification of item validity that allows researchers and developers to determine the number of models each item triggers. As part of development of the Fluids Conceptual Evaluation (FCE), the authors performed concentration analysis on a set of preliminary responses from the preliminary FCE. A sample of 290 students from three universities were each randomly assigned eight two-tier items from a set of 59 such items, such that each item would have between 35 and 45 responses. Analysis of concentration factor with respect to score will show how each item will map to different prescribed models of concentration; we also examine differences in patterns that arose as an effect of sampling. Recommendations for future use of concentration analysis will be made accordingly.