Measuring Oral Reading Fluency (ORF) Computer-Based and Paper-Based: Examining the Mode Effect in Reading Accuracy and Reading Fluency
Internationally, teachers use oral reading fluency (ORF) measurements to monitor learning progress in reading and adapt instruction to the individual needs of students. In ORF measures, the child reads aloud single syllables, words, or short passages, and the teacher rates in parallel at which items...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Education Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7102/13/6/624 |
| _version_ | 1797595149562281984 |
|---|---|
| author | Jana Jungjohann Jeffrey M. DeVries Markus Gebhardt |
| author_facet | Jana Jungjohann Jeffrey M. DeVries Markus Gebhardt |
| author_sort | Jana Jungjohann |
| collection | DOAJ |
| description | Internationally, teachers use oral reading fluency (ORF) measurements to monitor learning progress in reading and adapt instruction to the individual needs of students. In ORF measures, the child reads aloud single syllables, words, or short passages, and the teacher rates in parallel at which items the child makes a mistake. Since administering paper-based ORF requires increased effort on the part of teachers, computer-based test administration is available. However, there are still concerns about the comparability of paper-based and computer-based test modes. In our study, we examine mode effects between paper-based and computer-based test scores for both reading speed and reading accuracy using a German-language ORF assessment for progress monitoring. 2nd- and 3rd-year-students (<i>N</i> = 359) with and without special education needs participated in the study. Results show comparable and high reliability (<i>r</i> > 0.76) and no differential item functioning for both test modes. Furthermore, students showed significantly higher reading speed on the paper-based test, while no differences were found in reading accuracy. In the absence of differential item functioning, we discuss how mean differences can be accounted for, how teachers can be trained to use the different test modes, and how computer-based tests can be safeguarded in practice. |
| first_indexed | 2024-03-11T02:32:24Z |
| format | Article |
| id | doaj.art-1c336c88cc98430b9e61b08b29451c18 |
| institution | Directory Open Access Journal |
| issn | 2227-7102 |
| language | English |
| last_indexed | 2024-03-11T02:32:24Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Education Sciences |
| spelling | doaj.art-1c336c88cc98430b9e61b08b29451c182023-11-18T10:06:53ZengMDPI AGEducation Sciences2227-71022023-06-0113662410.3390/educsci13060624Measuring Oral Reading Fluency (ORF) Computer-Based and Paper-Based: Examining the Mode Effect in Reading Accuracy and Reading FluencyJana Jungjohann0Jeffrey M. DeVries1Markus Gebhardt2Faculty of Human Science, University of Regensburg, 93053 Regensburg, GermanyCenter for Research on Education and School Development, TU Dortmund University, 44227 Dortmund, GermanyFaculty of Human Science, University of Regensburg, 93053 Regensburg, GermanyInternationally, teachers use oral reading fluency (ORF) measurements to monitor learning progress in reading and adapt instruction to the individual needs of students. In ORF measures, the child reads aloud single syllables, words, or short passages, and the teacher rates in parallel at which items the child makes a mistake. Since administering paper-based ORF requires increased effort on the part of teachers, computer-based test administration is available. However, there are still concerns about the comparability of paper-based and computer-based test modes. In our study, we examine mode effects between paper-based and computer-based test scores for both reading speed and reading accuracy using a German-language ORF assessment for progress monitoring. 2nd- and 3rd-year-students (<i>N</i> = 359) with and without special education needs participated in the study. Results show comparable and high reliability (<i>r</i> > 0.76) and no differential item functioning for both test modes. Furthermore, students showed significantly higher reading speed on the paper-based test, while no differences were found in reading accuracy. In the absence of differential item functioning, we discuss how mean differences can be accounted for, how teachers can be trained to use the different test modes, and how computer-based tests can be safeguarded in practice.https://www.mdpi.com/2227-7102/13/6/624computer-based assessmentdifferential item functioningoral reading fluencymode effectspaper-based assessmentprogress monitoring |
| spellingShingle | Jana Jungjohann Jeffrey M. DeVries Markus Gebhardt Measuring Oral Reading Fluency (ORF) Computer-Based and Paper-Based: Examining the Mode Effect in Reading Accuracy and Reading Fluency Education Sciences computer-based assessment differential item functioning oral reading fluency mode effects paper-based assessment progress monitoring |
| title | Measuring Oral Reading Fluency (ORF) Computer-Based and Paper-Based: Examining the Mode Effect in Reading Accuracy and Reading Fluency |
| title_full | Measuring Oral Reading Fluency (ORF) Computer-Based and Paper-Based: Examining the Mode Effect in Reading Accuracy and Reading Fluency |
| title_fullStr | Measuring Oral Reading Fluency (ORF) Computer-Based and Paper-Based: Examining the Mode Effect in Reading Accuracy and Reading Fluency |
| title_full_unstemmed | Measuring Oral Reading Fluency (ORF) Computer-Based and Paper-Based: Examining the Mode Effect in Reading Accuracy and Reading Fluency |
| title_short | Measuring Oral Reading Fluency (ORF) Computer-Based and Paper-Based: Examining the Mode Effect in Reading Accuracy and Reading Fluency |
| title_sort | measuring oral reading fluency orf computer based and paper based examining the mode effect in reading accuracy and reading fluency |
| topic | computer-based assessment differential item functioning oral reading fluency mode effects paper-based assessment progress monitoring |
| url | https://www.mdpi.com/2227-7102/13/6/624 |
| work_keys_str_mv | AT janajungjohann measuringoralreadingfluencyorfcomputerbasedandpaperbasedexaminingthemodeeffectinreadingaccuracyandreadingfluency AT jeffreymdevries measuringoralreadingfluencyorfcomputerbasedandpaperbasedexaminingthemodeeffectinreadingaccuracyandreadingfluency AT markusgebhardt measuringoralreadingfluencyorfcomputerbasedandpaperbasedexaminingthemodeeffectinreadingaccuracyandreadingfluency |