Scaling up behavioral science interventions in online education
<jats:p>Online education is rapidly expanding in response to rising demand for higher and continuing education, but many online students struggle to achieve their educational goals. Several behavioral science interventions have shown promise in raising student persistence and completion rates...
| Main Authors: | , , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Proceedings of the National Academy of Sciences
2021
|
| Online Access: | https://hdl.handle.net/1721.1/135236 |
| _version_ | 1826206690749448192 |
|---|---|
| author | Kizilcec, René F Reich, Justin Yeomans, Michael Dann, Christoph Brunskill, Emma Lopez, Glenn Turkay, Selen Williams, Joseph Jay Tingley, Dustin |
| author2 | Massachusetts Institute of Technology. Program in Comparative Media Studies/Writing |
| author_facet | Massachusetts Institute of Technology. Program in Comparative Media Studies/Writing Kizilcec, René F Reich, Justin Yeomans, Michael Dann, Christoph Brunskill, Emma Lopez, Glenn Turkay, Selen Williams, Joseph Jay Tingley, Dustin |
| author_sort | Kizilcec, René F |
| collection | MIT |
| description | <jats:p>Online education is rapidly expanding in response to rising demand for higher and continuing education, but many online students struggle to achieve their educational goals. Several behavioral science interventions have shown promise in raising student persistence and completion rates in a handful of courses, but evidence of their effectiveness across diverse educational contexts is limited. In this study, we test a set of established interventions over 2.5 y, with one-quarter million students, from nearly every country, across 247 online courses offered by Harvard, the Massachusetts Institute of Technology, and Stanford. We hypothesized that the interventions would produce medium-to-large effects as in prior studies, but this is not supported by our results. Instead, using an iterative scientific process of cyclically preregistering new hypotheses in between waves of data collection, we identified individual, contextual, and temporal conditions under which the interventions benefit students. Self-regulation interventions raised student engagement in the first few weeks but not final completion rates. Value-relevance interventions raised completion rates in developing countries to close the global achievement gap, but only in courses with a global gap. We found minimal evidence that state-of-the-art machine learning methods can forecast the occurrence of a global gap or learn effective individualized intervention policies. Scaling behavioral science interventions across various online learning contexts can reduce their average effectiveness by an order-of-magnitude. However, iterative scientific investigations can uncover what works where for whom.</jats:p> |
| first_indexed | 2024-09-23T13:37:02Z |
| format | Article |
| id | mit-1721.1/135236 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T13:37:02Z |
| publishDate | 2021 |
| publisher | Proceedings of the National Academy of Sciences |
| record_format | dspace |
| spelling | mit-1721.1/1352362022-10-01T16:05:24Z Scaling up behavioral science interventions in online education Kizilcec, René F Reich, Justin Yeomans, Michael Dann, Christoph Brunskill, Emma Lopez, Glenn Turkay, Selen Williams, Joseph Jay Tingley, Dustin Massachusetts Institute of Technology. Program in Comparative Media Studies/Writing <jats:p>Online education is rapidly expanding in response to rising demand for higher and continuing education, but many online students struggle to achieve their educational goals. Several behavioral science interventions have shown promise in raising student persistence and completion rates in a handful of courses, but evidence of their effectiveness across diverse educational contexts is limited. In this study, we test a set of established interventions over 2.5 y, with one-quarter million students, from nearly every country, across 247 online courses offered by Harvard, the Massachusetts Institute of Technology, and Stanford. We hypothesized that the interventions would produce medium-to-large effects as in prior studies, but this is not supported by our results. Instead, using an iterative scientific process of cyclically preregistering new hypotheses in between waves of data collection, we identified individual, contextual, and temporal conditions under which the interventions benefit students. Self-regulation interventions raised student engagement in the first few weeks but not final completion rates. Value-relevance interventions raised completion rates in developing countries to close the global achievement gap, but only in courses with a global gap. We found minimal evidence that state-of-the-art machine learning methods can forecast the occurrence of a global gap or learn effective individualized intervention policies. Scaling behavioral science interventions across various online learning contexts can reduce their average effectiveness by an order-of-magnitude. However, iterative scientific investigations can uncover what works where for whom.</jats:p> 2021-10-27T20:22:35Z 2021-10-27T20:22:35Z 2020 2021-03-19T13:53:39Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/135236 en 10.1073/PNAS.1921417117 Proceedings of the National Academy of Sciences of the United States of America Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf Proceedings of the National Academy of Sciences PNAS |
| spellingShingle | Kizilcec, René F Reich, Justin Yeomans, Michael Dann, Christoph Brunskill, Emma Lopez, Glenn Turkay, Selen Williams, Joseph Jay Tingley, Dustin Scaling up behavioral science interventions in online education |
| title | Scaling up behavioral science interventions in online education |
| title_full | Scaling up behavioral science interventions in online education |
| title_fullStr | Scaling up behavioral science interventions in online education |
| title_full_unstemmed | Scaling up behavioral science interventions in online education |
| title_short | Scaling up behavioral science interventions in online education |
| title_sort | scaling up behavioral science interventions in online education |
| url | https://hdl.handle.net/1721.1/135236 |
| work_keys_str_mv | AT kizilcecrenef scalingupbehavioralscienceinterventionsinonlineeducation AT reichjustin scalingupbehavioralscienceinterventionsinonlineeducation AT yeomansmichael scalingupbehavioralscienceinterventionsinonlineeducation AT dannchristoph scalingupbehavioralscienceinterventionsinonlineeducation AT brunskillemma scalingupbehavioralscienceinterventionsinonlineeducation AT lopezglenn scalingupbehavioralscienceinterventionsinonlineeducation AT turkayselen scalingupbehavioralscienceinterventionsinonlineeducation AT williamsjosephjay scalingupbehavioralscienceinterventionsinonlineeducation AT tingleydustin scalingupbehavioralscienceinterventionsinonlineeducation |