Validating the pre/post-test in a MOOC environment
A standard method for measuring learning is to administer the same assessment before and after instruction. This pre/post-test technique is widely used in education research and has been used in our introductory physics MOOC to measure learning. One potential weakness of this paradigm...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
American Association of Physics Teachers (AAPT)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/137495 |
| _version_ | 1811076785616453632 |
|---|---|
| author | Chudzicki, Christopher Chen, Zhongzhou Zhou, Qian Alexandron, Giora Pritchard, David E. |
| author2 | Massachusetts Institute of Technology. Department of Physics |
| author_facet | Massachusetts Institute of Technology. Department of Physics Chudzicki, Christopher Chen, Zhongzhou Zhou, Qian Alexandron, Giora Pritchard, David E. |
| author_sort | Chudzicki, Christopher |
| collection | MIT |
| description | A standard method for measuring learning is to administer the same assessment before and after instruction. This pre/post-test technique is widely used in education research and has been used in our introductory physics MOOC to measure learning. One potential weakness of this paradigm is that post-test performance gains may result from exposure on the pre-test instead of instruction. This possibility is exacerbated in MOOCs where students receive multiple attempts per item, instant correct/incorrect feedback, and unlimited time (until the due date). To find the size of this problem in our recent MOOCs, we split the student population into two groups, each of which received identical post-tests but different subsets of post-test items on their group pre-test. We report a small overall advantage (2.9% ± 1.7%) on post-test items due to pre-test exposure. However, this advantage is not robust and is strongly diminished when one obviously anomalous item is removed. |
| first_indexed | 2024-09-23T10:27:33Z |
| format | Article |
| id | mit-1721.1/137495 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T10:27:33Z |
| publishDate | 2021 |
| publisher | American Association of Physics Teachers (AAPT) |
| record_format | dspace |
| spelling | mit-1721.1/1374952021-11-06T03:32:07Z Validating the pre/post-test in a MOOC environment Chudzicki, Christopher Chen, Zhongzhou Zhou, Qian Alexandron, Giora Pritchard, David E. Massachusetts Institute of Technology. Department of Physics A standard method for measuring learning is to administer the same assessment before and after instruction. This pre/post-test technique is widely used in education research and has been used in our introductory physics MOOC to measure learning. One potential weakness of this paradigm is that post-test performance gains may result from exposure on the pre-test instead of instruction. This possibility is exacerbated in MOOCs where students receive multiple attempts per item, instant correct/incorrect feedback, and unlimited time (until the due date). To find the size of this problem in our recent MOOCs, we split the student population into two groups, each of which received identical post-tests but different subsets of post-test items on their group pre-test. We report a small overall advantage (2.9% ± 1.7%) on post-test items due to pre-test exposure. However, this advantage is not robust and is strongly diminished when one obviously anomalous item is removed. 2021-11-05T14:55:14Z 2021-11-05T14:55:14Z 2015-12 2019-05-13T12:49:42Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/137495 Chudzicki, Christopher, Chen, Zhongzhou, Zhou, Qian, Alexandron, Giora and Pritchard, David E. 2015. "Validating the pre/post-test in a MOOC environment." en 10.1119/perc.2015.pr.016 Creative Commons Attribution 3.0 unported license https://creativecommons.org/licenses/by/3.0/ application/pdf American Association of Physics Teachers (AAPT) American Association of Physics Teachers |
| spellingShingle | Chudzicki, Christopher Chen, Zhongzhou Zhou, Qian Alexandron, Giora Pritchard, David E. Validating the pre/post-test in a MOOC environment |
| title | Validating the pre/post-test in a MOOC environment |
| title_full | Validating the pre/post-test in a MOOC environment |
| title_fullStr | Validating the pre/post-test in a MOOC environment |
| title_full_unstemmed | Validating the pre/post-test in a MOOC environment |
| title_short | Validating the pre/post-test in a MOOC environment |
| title_sort | validating the pre post test in a mooc environment |
| url | https://hdl.handle.net/1721.1/137495 |
| work_keys_str_mv | AT chudzickichristopher validatingthepreposttestinamoocenvironment AT chenzhongzhou validatingthepreposttestinamoocenvironment AT zhouqian validatingthepreposttestinamoocenvironment AT alexandrongiora validatingthepreposttestinamoocenvironment AT pritcharddavide validatingthepreposttestinamoocenvironment |