The purpose of this study was to compare usage levels of CompassLearning Odyssey mathematics and language arts software among fifth grade students in order to determine the relationship between usage and achievement. While educational software designed by various companies is a regular part of daily instruction in most public schools across the United States, there remains a need for research-based evidence of the efficacy of specific programs. This study used a quantitative design to compare achievement gains between 280 fifth grade students who had varying degrees of access to the software.

Being an ex post facto design, the study used data from students in five elementary schools in one district. Having partnered with state-supported independent researchers for the Enhancing Education through Technology (EETT), the district had already divided classes of teachers and students into groups that used CompassLearning software in one of three ways: mathematics only, language arts only, and both mathematics and language arts. This arrangement allowed the study to more easily examine whether or not the correlation between usage and achievement scores varied by subject area.

The study used independent t-tests to discern the relationship between usage and achievement. To nullify the effects of demographic variables of gender, SES, identification as learners with special needs, and prior achievement levels, ANCOVA analyses were conducted.

Results showed a significant relationship between CompassLearning Odyssey language arts and mathematics software and achievement as measured on the Pennsylvania System of School Assessment (PSSA). An interesting result was the finding that, on the PSSA reading posttest, users of mathematics software outperformed users of language arts software. Both groups showed greater gains than did students in the server-based version of the software.

Regarding demographic variables, gender had no effect on achievement gains. The effect of socioeconomic status was significant among users of mathematics software; conversely, the effects of prior achievement level and identification as learners with special needs were significant among users of language arts software.

This study supports the hypothesis that educational software enhances student achievement. Evidence gained in this study also identifies areas of technology-based instruction in need of further study.