Dynamic computerized-adaptive assessment of reading processes: Incremental validity on text comprehension

Abstract

This study explores the predictive and incremental validity of the Dynamic Computerized Adaptive Reading Processes Assessment Device (EDPL-BAI). This battery has 38 tasks organized in 6 modules to assess different processes related to reading comprehension (e.g. phoneme-grapheme association, executive functions, metacognitive processes).

Method: A total of 1336 Chilean students from 3rd to 6th grade (45% females) from 12 public urban and rural schools participated in a longitudinal study consisting of three assessment cycles. On the first cycle, the students were assessed with a standardized reading comprehension test (CL-PT), the Raven Progressive Matrices Test (PMR), and the complete poll of items were calibrated in a non-dynamic version of EDPL-BAI. On the second cycle, seven months after, a sub sample of 1068 students were administered the adaptive and dynamic version of the EDPL-BAI (graduated prompts were displayed after wrong responses), while the remainder 268 students were assessed with the non-dynamic version of EDPL-BAI, and for all the students a pre and post-test (using selected items from EDPL-BAI), CL-PT and PMR were administered. Finally, at the third cycle, four month after, the non-dynamic version of EDPL-BAI and CL-PT were administered to the complete sample.

Results and Conclusions: Multiple CFAs were conducted to determine the factorial structure of the EDPL-BAI, and a final model of 6 factors was retained. Predictive and incremental validity of the EDPL-BAI was explored with SEM analysis. The dynamic and non-dynamic versions of the EDPL-BAI were good predictors of reading comprehension. The dynamic version showed incremental validity over the non-dynamic version. Furthermore, graduated prompts predicted reading comprehension performance