Learning Experiments Using AB Testing at Scale
We report the one of the first applications of treatment/control group learning experiments in MOOCs. We have compared the efficacy of deliberate practice-practicing a key procedure repetitively-with traditional practice on "whole problems". Evaluating the learning using traditional whole...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Computing Machinery (ACM)
2015
|
| Online Access: | http://hdl.handle.net/1721.1/99202 https://orcid.org/0000-0002-7445-9338 https://orcid.org/0000-0002-0997-2979 https://orcid.org/0000-0001-5697-1496 |
| _version_ | 1811070137684459520 |
|---|---|
| author | Chudzicki, Christopher Pritchard, David E. Chen, Zhongzhou |
| author2 | Massachusetts Institute of Technology. Department of Physics |
| author_facet | Massachusetts Institute of Technology. Department of Physics Chudzicki, Christopher Pritchard, David E. Chen, Zhongzhou |
| author_sort | Chudzicki, Christopher |
| collection | MIT |
| description | We report the one of the first applications of treatment/control group learning experiments in MOOCs. We have compared the efficacy of deliberate practice-practicing a key procedure repetitively-with traditional practice on "whole problems". Evaluating the learning using traditional whole problems we find that traditional practice outperforms drag and drop, which in turn outperforms multiple choice. In addition, we measured the amount of learning that occurs during a pretest administered in a MOOC environment that transfers to the same question if placed on the posttest. We place a limit on the amount of such transfer, which suggests that this type of learning effect is very weak compared to the learning observed throughout the entire course. |
| first_indexed | 2024-09-23T08:27:12Z |
| format | Article |
| id | mit-1721.1/99202 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T08:27:12Z |
| publishDate | 2015 |
| publisher | Association for Computing Machinery (ACM) |
| record_format | dspace |
| spelling | mit-1721.1/992022022-09-30T09:19:41Z Learning Experiments Using AB Testing at Scale Chudzicki, Christopher Pritchard, David E. Chen, Zhongzhou Massachusetts Institute of Technology. Department of Physics Pritchard, David E. Chudzicki, Christopher Pritchard, David E. Chen, Zhongzhou We report the one of the first applications of treatment/control group learning experiments in MOOCs. We have compared the efficacy of deliberate practice-practicing a key procedure repetitively-with traditional practice on "whole problems". Evaluating the learning using traditional whole problems we find that traditional practice outperforms drag and drop, which in turn outperforms multiple choice. In addition, we measured the amount of learning that occurs during a pretest administered in a MOOC environment that transfers to the same question if placed on the posttest. We place a limit on the amount of such transfer, which suggests that this type of learning effect is very weak compared to the learning observed throughout the entire course. Google (Firm) Massachusetts Institute of Technology National Science Foundation (U.S.) 2015-10-08T13:18:08Z 2015-10-08T13:18:08Z 2015-03 Article http://purl.org/eprint/type/JournalArticle 9781450334112 http://hdl.handle.net/1721.1/99202 Christopher Chudzicki, David E. Pritchard, and Zhongzhou Chen. 2015. Learning Experiments Using AB Testing at Scale. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale (L@S '15). ACM, New York, NY, USA, 405-408. https://orcid.org/0000-0002-7445-9338 https://orcid.org/0000-0002-0997-2979 https://orcid.org/0000-0001-5697-1496 en_US http://dx.doi.org/10.1145/2724660.2728703 Proceedings of the Second (2015) ACM Conference on Learning @ Scale (L@S '15) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Association for Computing Machinery (ACM) Prof. Pritchard via Barbara Williams |
| spellingShingle | Chudzicki, Christopher Pritchard, David E. Chen, Zhongzhou Learning Experiments Using AB Testing at Scale |
| title | Learning Experiments Using AB Testing at Scale |
| title_full | Learning Experiments Using AB Testing at Scale |
| title_fullStr | Learning Experiments Using AB Testing at Scale |
| title_full_unstemmed | Learning Experiments Using AB Testing at Scale |
| title_short | Learning Experiments Using AB Testing at Scale |
| title_sort | learning experiments using ab testing at scale |
| url | http://hdl.handle.net/1721.1/99202 https://orcid.org/0000-0002-7445-9338 https://orcid.org/0000-0002-0997-2979 https://orcid.org/0000-0001-5697-1496 |
| work_keys_str_mv | AT chudzickichristopher learningexperimentsusingabtestingatscale AT pritcharddavide learningexperimentsusingabtestingatscale AT chenzhongzhou learningexperimentsusingabtestingatscale |