A Multimodal Deep Learning Model Using Text, Image, and Code Data for Improving Issue Classification Tasks
Issue reports are valuable resources for the continuous maintenance and improvement of software. Managing issue reports requires a significant effort from developers. To address this problem, many researchers have proposed automated techniques for classifying issue reports. However, those techniques...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/16/9456 |
| _version_ | 1797585603363078144 |
|---|---|
| author | Changwon Kwak Pilsu Jung Seonah Lee |
| author_facet | Changwon Kwak Pilsu Jung Seonah Lee |
| author_sort | Changwon Kwak |
| collection | DOAJ |
| description | Issue reports are valuable resources for the continuous maintenance and improvement of software. Managing issue reports requires a significant effort from developers. To address this problem, many researchers have proposed automated techniques for classifying issue reports. However, those techniques fall short of yielding reasonable classification accuracy. We notice that those techniques rely on text-based unimodal models. In this paper, we propose a novel multimodal model-based classification technique to use heterogeneous information in issue reports for issue classification. The proposed technique combines information from text, images, and code of issue reports. To evaluate the proposed technique, we conduct experiments with four different projects. The experiments compare the performance of the proposed technique with text-based unimodal models. Our experimental results show that the proposed technique achieves a 5.07% to 14.12% higher F1-score than the text-based unimodal models. Our findings demonstrate that utilizing heterogeneous data of issue reports helps improve the performance of issue classification. |
| first_indexed | 2024-03-11T00:08:15Z |
| format | Article |
| id | doaj.art-3ea172d030004632945b5b4c2d9551e0 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T00:08:15Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-3ea172d030004632945b5b4c2d9551e02023-11-19T00:10:03ZengMDPI AGApplied Sciences2076-34172023-08-011316945610.3390/app13169456A Multimodal Deep Learning Model Using Text, Image, and Code Data for Improving Issue Classification TasksChangwon Kwak0Pilsu Jung1Seonah Lee2Department of AI Convergence Engineering, Gyeongsang National University, 501 Jinjudaero, Jinju-si 52828, Gyeongsangnam-do, Republic of KoreaDepartment of AI Convergence Engineering, Gyeongsang National University, 501 Jinjudaero, Jinju-si 52828, Gyeongsangnam-do, Republic of KoreaDepartment of AI Convergence Engineering, Gyeongsang National University, 501 Jinjudaero, Jinju-si 52828, Gyeongsangnam-do, Republic of KoreaIssue reports are valuable resources for the continuous maintenance and improvement of software. Managing issue reports requires a significant effort from developers. To address this problem, many researchers have proposed automated techniques for classifying issue reports. However, those techniques fall short of yielding reasonable classification accuracy. We notice that those techniques rely on text-based unimodal models. In this paper, we propose a novel multimodal model-based classification technique to use heterogeneous information in issue reports for issue classification. The proposed technique combines information from text, images, and code of issue reports. To evaluate the proposed technique, we conduct experiments with four different projects. The experiments compare the performance of the proposed technique with text-based unimodal models. Our experimental results show that the proposed technique achieves a 5.07% to 14.12% higher F1-score than the text-based unimodal models. Our findings demonstrate that utilizing heterogeneous data of issue reports helps improve the performance of issue classification.https://www.mdpi.com/2076-3417/13/16/9456issue classificationissue reportsmultimodaldeep learningbugfeature |
| spellingShingle | Changwon Kwak Pilsu Jung Seonah Lee A Multimodal Deep Learning Model Using Text, Image, and Code Data for Improving Issue Classification Tasks Applied Sciences issue classification issue reports multimodal deep learning bug feature |
| title | A Multimodal Deep Learning Model Using Text, Image, and Code Data for Improving Issue Classification Tasks |
| title_full | A Multimodal Deep Learning Model Using Text, Image, and Code Data for Improving Issue Classification Tasks |
| title_fullStr | A Multimodal Deep Learning Model Using Text, Image, and Code Data for Improving Issue Classification Tasks |
| title_full_unstemmed | A Multimodal Deep Learning Model Using Text, Image, and Code Data for Improving Issue Classification Tasks |
| title_short | A Multimodal Deep Learning Model Using Text, Image, and Code Data for Improving Issue Classification Tasks |
| title_sort | multimodal deep learning model using text image and code data for improving issue classification tasks |
| topic | issue classification issue reports multimodal deep learning bug feature |
| url | https://www.mdpi.com/2076-3417/13/16/9456 |
| work_keys_str_mv | AT changwonkwak amultimodaldeeplearningmodelusingtextimageandcodedataforimprovingissueclassificationtasks AT pilsujung amultimodaldeeplearningmodelusingtextimageandcodedataforimprovingissueclassificationtasks AT seonahlee amultimodaldeeplearningmodelusingtextimageandcodedataforimprovingissueclassificationtasks AT changwonkwak multimodaldeeplearningmodelusingtextimageandcodedataforimprovingissueclassificationtasks AT pilsujung multimodaldeeplearningmodelusingtextimageandcodedataforimprovingissueclassificationtasks AT seonahlee multimodaldeeplearningmodelusingtextimageandcodedataforimprovingissueclassificationtasks |