Krisztina R. Tóth and Ágnes Hódi: A Comparative Study of Students’ Reading Comprehension Achievement in Computer-Based and Paper-and-pencil Tests

The integration of computers into learner assessment has opened up new opportunities in measuring learners’ reading skills. The OECD-PISA Electronic Reading Assessment and the PIRLS Web-based Reading Initiative show that large-scale learner assessment programs have also recognised the importance of the digital era and the role computers play in reading literacy. In line with international trends, the Hungarian Educational Longitudinal Programme also intends to shift from paper-and-pencil tests to a computerized environment because of the benefits of modern technology. However, the introduction of computerized tests requires a sophisticated comparative analysis of learner performance in both environments and must be carried out regarding the effect which testing format might have on learner performance. The main objectives of the present study are: (1) to provide an overview of international comparative studies and their results with regard to the test mode effect; (2) to investigate achievement differences on reading tests containing different text types administered on screen and on paper; and (3) to identify learners most affected by the testing medium. This mode effect study is based on a paper-and-pencil test and its computerized parallel test version, which is designed to measure the reading comprehension skills of Hungarian schoolchildren in Year 6. 449 learners participated in the study carried out in 2009. Our results provide empirical evidence for the test mode effect. Participants achieved better performance when reading texts on paper than on screen. Analysis that aimed at examining mode effect on different text types indicated significant differences in learners’ performance on continuous and non-continuous test types as well. Learners performed better on both texts presented in the printed medium than online. With regard to gender analysis, the delivery media had a significant impact on both boys’ and girls’ achievement as they performed significantly better on the paper test than in an electronic environment. However, learners’ computer experience did not have a notable effect on the difference in their reading performance achieved on paper or screen. Furthermore, a subgroup-level analysis, carried out by dividing the sample into subgroups on the basis of achievement, identified groups of learners that exhibited varying behaviours.

MAGYAR PEDAGÓGIA 111. Number 4. 313-332. (2011)

Levelezési cím / Address for correspondence: R. Tóth Krisztina és Hódi Ágnes, Szegedi Tudományegyetem Neveléstudományi Intézet, H–6722 Szeged, Petőfi S. sgt. 30–34.

 
PDF