This study explored the extent to which seven paper and pencil assessment items judged by experts to be aligned with forces and motion ideas actually measure those concepts for middle school students. To accomplish this, the study compared students' individual responses under paper and pencil and interview conditions while considering an elaborated model of a target science idea assessed by the items. The study sought to confirm the inferences provided by a written science assessment instrument by comparing them to alternative inferences based on student interview data.

Participants in the study were a diverse sample of 14 middle school students. Each student completed the written assessment instrument and then participated in an interview designed to confirm student idea use. All interview transcripts were coded for idea use and the results from coded data were then compared to students' performance on the written posttest. The study concludes that in most cases, the two sets of assessment evidence were in agreement and that the items provide valid inferences about students' actual understanding of the target ideas. The study also concludes that on a few occasions the written posttest overestimated students' performances when compared to data collected during the interviews. The study presents additional student ideas regarding forces and motion that have been previously undocumented in the literature. Conclusions and implications are discussed.