An Analysis of Patch Plausibility and Correctness for Generate-And-Validate Patch Generation Systems (Supplementary Material)
We analyze reported patches for three prior generate-and-validate patch generation systems (GenProg, RSRepair, and AE). Because of errors in the patch evaluation infrastructure, the majority of the reported patches violate the basic principle behind the design of these systems they do not produce...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Published: |
2015
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/97051 |
| _version_ | 1826188138966417408 |
|---|---|
| author | Qi, Zichao Long, Fan Achour, Sara Rinard, Martin |
| author2 | Martin Rinard |
| author_facet | Martin Rinard Qi, Zichao Long, Fan Achour, Sara Rinard, Martin |
| author_sort | Qi, Zichao |
| collection | MIT |
| description | We analyze reported patches for three prior generate-and-validate patch generation systems (GenProg, RSRepair, and AE). Because of errors in the patch evaluation infrastructure, the majority of the reported patches violate the basic principle behind the design of these systems they do not produce correct outputs even for the inputs in the test suite used to validate the patches. We also show that the overwhelming majority of the accepted patches are not correct and are equivalent to a single modification that simply deletes functionality. We also present Kali, a generate-and-validate patch generation system that only deletes functionality. Working with a simpler and more effectively focused search space, Kali generates at least as many correct patches as prior GenProg, RSRepair, and AE systems. Kali also generates at least as many patches that produce correct outputs for the inputs in the validation test suite as the three prior systems. We also discuss the patches produced by ClearView, a generate-and-validate binary hot patching system that leverages learned invariants to produce patches that enable systems to survive otherwise fatal defects and security attacks. Our analysis indicates that ClearView successfully patches 9 of the 10 security vulnerabilities used to evaluate the system. At least 4 of these patches are correct. |
| first_indexed | 2024-09-23T07:55:10Z |
| id | mit-1721.1/97051 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T07:55:10Z |
| publishDate | 2015 |
| record_format | dspace |
| spelling | mit-1721.1/970512019-04-08T07:08:57Z An Analysis of Patch Plausibility and Correctness for Generate-And-Validate Patch Generation Systems (Supplementary Material) Qi, Zichao Long, Fan Achour, Sara Rinard, Martin Martin Rinard Computer Architecture Automatic Repair Patch Analysis Function Elimination We analyze reported patches for three prior generate-and-validate patch generation systems (GenProg, RSRepair, and AE). Because of errors in the patch evaluation infrastructure, the majority of the reported patches violate the basic principle behind the design of these systems they do not produce correct outputs even for the inputs in the test suite used to validate the patches. We also show that the overwhelming majority of the accepted patches are not correct and are equivalent to a single modification that simply deletes functionality. We also present Kali, a generate-and-validate patch generation system that only deletes functionality. Working with a simpler and more effectively focused search space, Kali generates at least as many correct patches as prior GenProg, RSRepair, and AE systems. Kali also generates at least as many patches that produce correct outputs for the inputs in the validation test suite as the three prior systems. We also discuss the patches produced by ClearView, a generate-and-validate binary hot patching system that leverages learned invariants to produce patches that enable systems to survive otherwise fatal defects and security attacks. Our analysis indicates that ClearView successfully patches 9 of the 10 security vulnerabilities used to evaluate the system. At least 4 of these patches are correct. 2015-05-21T21:00:09Z 2015-05-21T21:00:09Z 2015-05-21 2015-05-21T21:00:09Z http://hdl.handle.net/1721.1/97051 13152246 bytes application/octet-stream |
| spellingShingle | Automatic Repair Patch Analysis Function Elimination Qi, Zichao Long, Fan Achour, Sara Rinard, Martin An Analysis of Patch Plausibility and Correctness for Generate-And-Validate Patch Generation Systems (Supplementary Material) |
| title | An Analysis of Patch Plausibility and Correctness for Generate-And-Validate Patch Generation Systems (Supplementary Material) |
| title_full | An Analysis of Patch Plausibility and Correctness for Generate-And-Validate Patch Generation Systems (Supplementary Material) |
| title_fullStr | An Analysis of Patch Plausibility and Correctness for Generate-And-Validate Patch Generation Systems (Supplementary Material) |
| title_full_unstemmed | An Analysis of Patch Plausibility and Correctness for Generate-And-Validate Patch Generation Systems (Supplementary Material) |
| title_short | An Analysis of Patch Plausibility and Correctness for Generate-And-Validate Patch Generation Systems (Supplementary Material) |
| title_sort | analysis of patch plausibility and correctness for generate and validate patch generation systems supplementary material |
| topic | Automatic Repair Patch Analysis Function Elimination |
| url | http://hdl.handle.net/1721.1/97051 |
| work_keys_str_mv | AT qizichao ananalysisofpatchplausibilityandcorrectnessforgenerateandvalidatepatchgenerationsystemssupplementarymaterial AT longfan ananalysisofpatchplausibilityandcorrectnessforgenerateandvalidatepatchgenerationsystemssupplementarymaterial AT achoursara ananalysisofpatchplausibilityandcorrectnessforgenerateandvalidatepatchgenerationsystemssupplementarymaterial AT rinardmartin ananalysisofpatchplausibilityandcorrectnessforgenerateandvalidatepatchgenerationsystemssupplementarymaterial AT qizichao analysisofpatchplausibilityandcorrectnessforgenerateandvalidatepatchgenerationsystemssupplementarymaterial AT longfan analysisofpatchplausibilityandcorrectnessforgenerateandvalidatepatchgenerationsystemssupplementarymaterial AT achoursara analysisofpatchplausibilityandcorrectnessforgenerateandvalidatepatchgenerationsystemssupplementarymaterial AT rinardmartin analysisofpatchplausibilityandcorrectnessforgenerateandvalidatepatchgenerationsystemssupplementarymaterial |