Citizen Science projects offer opportunities for non-scientists to take part in scientific research. Scientific results from these projects have been well documented. However, there is limited research about how these projects affect their volunteer participants. In this study, I investigate how participation in an online, collaborative astronomical citizen science project can be associated with the scientific literacy of its participants.

Scientific literacy is measured through three elements: attitude towards science, belief in the nature of science and competencies associated with learning science. The first two elements are measured through a pre-test given to 1,385 participants when they join the project and a post-test given six months later to 125 participants. Attitude towards science was measured using nine Likert-items custom designed for this project and beliefs in the nature of science were measured using a modified version of the Nature of Science Knowledge scale. Responses were analyzed using the Rasch Rating Scale Model. Competencies are measured through analysis of discourse occurring in online asynchronous discussion forums using the Community of Inquiry framework, which describes three types of presence in the online forums: cognitive, social and teaching.

Results show that overall attitudes did not change, p = .225. However, there was significant change towards attitudes about science in the news (positive) and scientific self efficacy (negative), p < .001 and p = .035 respectively. Beliefs in the nature of science exhibited a small, but significant increase, p = .04. Relative positioning of scores on the belief items did not change much, suggesting the increase is mostly due to reinforcement of current beliefs. The cognitive and teaching presence in the online forums did not change, p = .807 and p = .505 respectively. However, the social presence did change, p = .011.

Overall, these results suggest that multi-faceted, collaborative citizen science projects can have an impact on some aspects of scientific literacy. Using the Rasch Model allowed us to uncover effects that may have otherwise been hidden. Future projects may want to include social interactivity between participants and also make participants specifically aware of how they are contributing to the entire scientific process.