Automated feedback generation for introductory programming assignments
We present a new method for automatically providing feedback for introductory programming problems. In order to use this method, we need a reference implementation of the assignment, and an error model consisting of potential corrections to errors that students might make. Using this information, th...
| मुख्य लेखकों: | , , |
|---|---|
| अन्य लेखक: | |
| स्वरूप: | लेख |
| भाषा: | en_US |
| प्रकाशित: |
Association for Computing Machinery
2014
|
| ऑनलाइन पहुंच: | http://hdl.handle.net/1721.1/90851 https://orcid.org/0000-0001-7604-8252 |
| _version_ | 1826200618456317952 |
|---|---|
| author | Singh, Rishabh Gulwani, Sumit Solar-Lezama, Armando |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Singh, Rishabh Gulwani, Sumit Solar-Lezama, Armando |
| author_sort | Singh, Rishabh |
| collection | MIT |
| description | We present a new method for automatically providing feedback for introductory programming problems. In order to use this method, we need a reference implementation of the assignment, and an error model consisting of potential corrections to errors that students might make. Using this information, the system automatically derives minimal corrections to student's incorrect solutions, providing them with a measure of exactly how incorrect a given solution was, as well as feedback about what they did wrong.
We introduce a simple language for describing error models in terms of correction rules, and formally define a rule-directed translation strategy that reduces the problem of finding minimal corrections in an incorrect program to the problem of synthesizing a correct program from a sketch. We have evaluated our system on thousands of real student attempts obtained from the Introduction to Programming course at MIT (6.00) and MITx (6.00x). Our results show that relatively simple error models can correct on average 64% of all incorrect submissions in our benchmark set. |
| first_indexed | 2024-09-23T11:39:14Z |
| format | Article |
| id | mit-1721.1/90851 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T11:39:14Z |
| publishDate | 2014 |
| publisher | Association for Computing Machinery |
| record_format | dspace |
| spelling | mit-1721.1/908512022-10-01T05:01:47Z Automated feedback generation for introductory programming assignments Singh, Rishabh Gulwani, Sumit Solar-Lezama, Armando Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Singh, Rishabh Solar-Lezama, Armando We present a new method for automatically providing feedback for introductory programming problems. In order to use this method, we need a reference implementation of the assignment, and an error model consisting of potential corrections to errors that students might make. Using this information, the system automatically derives minimal corrections to student's incorrect solutions, providing them with a measure of exactly how incorrect a given solution was, as well as feedback about what they did wrong. We introduce a simple language for describing error models in terms of correction rules, and formally define a rule-directed translation strategy that reduces the problem of finding minimal corrections in an incorrect program to the problem of synthesizing a correct program from a sketch. We have evaluated our system on thousands of real student attempts obtained from the Introduction to Programming course at MIT (6.00) and MITx (6.00x). Our results show that relatively simple error models can correct on average 64% of all incorrect submissions in our benchmark set. National Science Foundation (U.S.) (grant Expeditions in Computer Augmented Program Engineering (ExCAPE NSF-1139056)) National Science Foundation (U.S.) (grant Expeditions in Computer Augmented Program Engineering (NSF-1139056)) Massachusetts Institute of Technology (MIT EECS Super UROP program) Microsoft Research (PhD Fellowship) Microsoft Research 2014-10-09T18:51:44Z 2014-10-09T18:51:44Z 2013-06 Article http://purl.org/eprint/type/ConferencePaper 03621340 http://hdl.handle.net/1721.1/90851 Singh, Rishabh, Sumit Gulwani, and Armando Solar-Lezama. “Automated Feedback Generation for Introductory Programming Assignments.” 34th ACM SIGPLAN conference on Programming language design and implementation, PLDI'13, June 16-19, 2013, Seattle, WA, USA. p.15-26. https://orcid.org/0000-0001-7604-8252 en_US http://dx.doi.org/10.1145/2499370.2462195 Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation -- PLDI '13 Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Association for Computing Machinery MIT web domain |
| spellingShingle | Singh, Rishabh Gulwani, Sumit Solar-Lezama, Armando Automated feedback generation for introductory programming assignments |
| title | Automated feedback generation for introductory programming assignments |
| title_full | Automated feedback generation for introductory programming assignments |
| title_fullStr | Automated feedback generation for introductory programming assignments |
| title_full_unstemmed | Automated feedback generation for introductory programming assignments |
| title_short | Automated feedback generation for introductory programming assignments |
| title_sort | automated feedback generation for introductory programming assignments |
| url | http://hdl.handle.net/1721.1/90851 https://orcid.org/0000-0001-7604-8252 |
| work_keys_str_mv | AT singhrishabh automatedfeedbackgenerationforintroductoryprogrammingassignments AT gulwanisumit automatedfeedbackgenerationforintroductoryprogrammingassignments AT solarlezamaarmando automatedfeedbackgenerationforintroductoryprogrammingassignments |