The value of large-scale randomised control trials in system-wide improvement: The case of the Reading Catch-Up Programme
This article illustrates the value of large-scale impact evaluations with counterfactual components. It begins by exploring the limitations of small-scale impact studies, which do not allow reliable inference to a wider population or which do not use valid comparison groups. The paper then describes...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Education Association of South Africa
2017-02-01
|
| Series: | South African Journal of Education |
| Subjects: | |
| Online Access: | http://www.sajournalofeducation.co.za/index.php/saje/article/view/1248/681 |
| _version_ | 1819295810565701632 |
|---|---|
| author | Brahm Fleisch Stephen Taylor Volker Schöer Thabo Mabogoane |
| author_facet | Brahm Fleisch Stephen Taylor Volker Schöer Thabo Mabogoane |
| author_sort | Brahm Fleisch |
| collection | DOAJ |
| description | This article illustrates the value of large-scale impact evaluations with counterfactual components. It begins by exploring the limitations of small-scale impact studies, which do not allow reliable inference to a wider population or which do not use valid comparison groups. The paper then describes the design features of a recent large-scale randomised control trial (RCT) evaluation of an intermediate phase literacy intervention that we evaluated. Using a rigorous sampling process and randomised assignment, the paper shows the value of the approach, and how the RCT method prevents researchers from reaching potentially harmful false positive findings. The paper also considers some of the limitations of the RCT method and makes recommendations to mitigate these. |
| first_indexed | 2024-12-24T04:48:08Z |
| format | Article |
| id | doaj.art-d5cf29983d5442fb9148f1ed0bc544fc |
| institution | Directory Open Access Journal |
| issn | 0256-0100 2076-3433 |
| language | English |
| last_indexed | 2024-12-24T04:48:08Z |
| publishDate | 2017-02-01 |
| publisher | Education Association of South Africa |
| record_format | Article |
| series | South African Journal of Education |
| spelling | doaj.art-d5cf29983d5442fb9148f1ed0bc544fc2022-12-21T17:14:38ZengEducation Association of South AfricaSouth African Journal of Education0256-01002076-34332017-02-0137111310.15700/saje.v37n1a1248The value of large-scale randomised control trials in system-wide improvement: The case of the Reading Catch-Up ProgrammeBrahm Fleisch0Stephen Taylor1Volker Schöer2Thabo Mabogoane3Division of Educational Leadership, Policy and Skills, Wits School of Education, University of the Witwatersrand, Johannesburg, South AfricaResearch Directorate, Department of Basic Education, Pretoria, South AfricaAfrican Micro-Economic Research Unit, School of Economic and Business Sciences, University of the Witwatersrand, Johannesburg, South AfricaDepartment of Planning, Monitoring and Evaluation, Presidency, Pretoria, South AfricaThis article illustrates the value of large-scale impact evaluations with counterfactual components. It begins by exploring the limitations of small-scale impact studies, which do not allow reliable inference to a wider population or which do not use valid comparison groups. The paper then describes the design features of a recent large-scale randomised control trial (RCT) evaluation of an intermediate phase literacy intervention that we evaluated. Using a rigorous sampling process and randomised assignment, the paper shows the value of the approach, and how the RCT method prevents researchers from reaching potentially harmful false positive findings. The paper also considers some of the limitations of the RCT method and makes recommendations to mitigate these.http://www.sajournalofeducation.co.za/index.php/saje/article/view/1248/681impact evaluationintermediate phase literacyRandomised Control Trial |
| spellingShingle | Brahm Fleisch Stephen Taylor Volker Schöer Thabo Mabogoane The value of large-scale randomised control trials in system-wide improvement: The case of the Reading Catch-Up Programme South African Journal of Education impact evaluation intermediate phase literacy Randomised Control Trial |
| title | The value of large-scale randomised control trials in system-wide improvement: The case of the Reading Catch-Up Programme |
| title_full | The value of large-scale randomised control trials in system-wide improvement: The case of the Reading Catch-Up Programme |
| title_fullStr | The value of large-scale randomised control trials in system-wide improvement: The case of the Reading Catch-Up Programme |
| title_full_unstemmed | The value of large-scale randomised control trials in system-wide improvement: The case of the Reading Catch-Up Programme |
| title_short | The value of large-scale randomised control trials in system-wide improvement: The case of the Reading Catch-Up Programme |
| title_sort | value of large scale randomised control trials in system wide improvement the case of the reading catch up programme |
| topic | impact evaluation intermediate phase literacy Randomised Control Trial |
| url | http://www.sajournalofeducation.co.za/index.php/saje/article/view/1248/681 |
| work_keys_str_mv | AT brahmfleisch thevalueoflargescalerandomisedcontroltrialsinsystemwideimprovementthecaseofthereadingcatchupprogramme AT stephentaylor thevalueoflargescalerandomisedcontroltrialsinsystemwideimprovementthecaseofthereadingcatchupprogramme AT volkerschoer thevalueoflargescalerandomisedcontroltrialsinsystemwideimprovementthecaseofthereadingcatchupprogramme AT thabomabogoane thevalueoflargescalerandomisedcontroltrialsinsystemwideimprovementthecaseofthereadingcatchupprogramme AT brahmfleisch valueoflargescalerandomisedcontroltrialsinsystemwideimprovementthecaseofthereadingcatchupprogramme AT stephentaylor valueoflargescalerandomisedcontroltrialsinsystemwideimprovementthecaseofthereadingcatchupprogramme AT volkerschoer valueoflargescalerandomisedcontroltrialsinsystemwideimprovementthecaseofthereadingcatchupprogramme AT thabomabogoane valueoflargescalerandomisedcontroltrialsinsystemwideimprovementthecaseofthereadingcatchupprogramme |