Impact of the Volume and Distribution of Training Datasets in the Development of Deep-Learning Models for the Diagnosis of Colorectal Polyps in Endoscopy Images
Background: Establishment of an artificial intelligence model in gastrointestinal endoscopy has no standardized dataset. The optimal volume or class distribution of training datasets has not been evaluated. An artificial intelligence model was previously created by the authors to classify endoscopic...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-08-01
|
| Series: | Journal of Personalized Medicine |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-4426/12/9/1361 |
| _version_ | 1797486155350933504 |
|---|---|
| author | Eun Jeong Gong Chang Seok Bang Jae Jun Lee Young Joo Yang Gwang Ho Baik |
| author_facet | Eun Jeong Gong Chang Seok Bang Jae Jun Lee Young Joo Yang Gwang Ho Baik |
| author_sort | Eun Jeong Gong |
| collection | DOAJ |
| description | Background: Establishment of an artificial intelligence model in gastrointestinal endoscopy has no standardized dataset. The optimal volume or class distribution of training datasets has not been evaluated. An artificial intelligence model was previously created by the authors to classify endoscopic images of colorectal polyps into four categories, including advanced colorectal cancer, early cancers/high-grade dysplasia, tubular adenoma, and nonneoplasm. The aim of this study was to evaluate the impact of the volume and distribution of training dataset classes in the development of deep-learning models for colorectal polyp histopathology prediction from endoscopic images. Methods: The same 3828 endoscopic images that were used to create earlier models were used. An additional 6838 images were used to find the optimal volume and class distribution for a deep-learning model. Various amounts of data volume and class distributions were tried to establish deep-learning models. The training of deep-learning models uniformly used no-code platform Neuro-T. Accuracy was the primary outcome on four-class prediction. Results: The highest internal-test classification accuracy in the original dataset, doubled dataset, and tripled dataset was commonly shown by doubling the proportion of data for fewer categories (2:2:1:1 for advanced colorectal cancer: early cancers/high-grade dysplasia: tubular adenoma: non-neoplasm). Doubling the proportion of data for fewer categories in the original dataset showed the highest accuracy (86.4%, 95% confidence interval: 85.0–97.8%) compared to that of the doubled or tripled dataset. The total required number of images in this performance was only 2418 images. Gradient-weighted class activation mapping confirmed that the part that the deep-learning model pays attention to coincides with the part that the endoscopist pays attention to. Conclusion: As a result of a data-volume-dependent performance plateau in the classification model of colonoscopy, a dataset that has been doubled or tripled is not always beneficial to training. Deep-learning models would be more accurate if the proportion of fewer category lesions was increased. |
| first_indexed | 2024-03-09T23:29:10Z |
| format | Article |
| id | doaj.art-57067fda8e6d42ac9145e4571a1c4a92 |
| institution | Directory Open Access Journal |
| issn | 2075-4426 |
| language | English |
| last_indexed | 2024-03-09T23:29:10Z |
| publishDate | 2022-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Personalized Medicine |
| spelling | doaj.art-57067fda8e6d42ac9145e4571a1c4a922023-11-23T17:11:51ZengMDPI AGJournal of Personalized Medicine2075-44262022-08-01129136110.3390/jpm12091361Impact of the Volume and Distribution of Training Datasets in the Development of Deep-Learning Models for the Diagnosis of Colorectal Polyps in Endoscopy ImagesEun Jeong Gong0Chang Seok Bang1Jae Jun Lee2Young Joo Yang3Gwang Ho Baik4Department of Internal Medicine, Hallym University College of Medicine, Chuncheon 24253, KoreaDepartment of Internal Medicine, Hallym University College of Medicine, Chuncheon 24253, KoreaInstitute of New Frontier Research, Hallym University College of Medicine, Chuncheon 24253, KoreaDepartment of Internal Medicine, Hallym University College of Medicine, Chuncheon 24253, KoreaDepartment of Internal Medicine, Hallym University College of Medicine, Chuncheon 24253, KoreaBackground: Establishment of an artificial intelligence model in gastrointestinal endoscopy has no standardized dataset. The optimal volume or class distribution of training datasets has not been evaluated. An artificial intelligence model was previously created by the authors to classify endoscopic images of colorectal polyps into four categories, including advanced colorectal cancer, early cancers/high-grade dysplasia, tubular adenoma, and nonneoplasm. The aim of this study was to evaluate the impact of the volume and distribution of training dataset classes in the development of deep-learning models for colorectal polyp histopathology prediction from endoscopic images. Methods: The same 3828 endoscopic images that were used to create earlier models were used. An additional 6838 images were used to find the optimal volume and class distribution for a deep-learning model. Various amounts of data volume and class distributions were tried to establish deep-learning models. The training of deep-learning models uniformly used no-code platform Neuro-T. Accuracy was the primary outcome on four-class prediction. Results: The highest internal-test classification accuracy in the original dataset, doubled dataset, and tripled dataset was commonly shown by doubling the proportion of data for fewer categories (2:2:1:1 for advanced colorectal cancer: early cancers/high-grade dysplasia: tubular adenoma: non-neoplasm). Doubling the proportion of data for fewer categories in the original dataset showed the highest accuracy (86.4%, 95% confidence interval: 85.0–97.8%) compared to that of the doubled or tripled dataset. The total required number of images in this performance was only 2418 images. Gradient-weighted class activation mapping confirmed that the part that the deep-learning model pays attention to coincides with the part that the endoscopist pays attention to. Conclusion: As a result of a data-volume-dependent performance plateau in the classification model of colonoscopy, a dataset that has been doubled or tripled is not always beneficial to training. Deep-learning models would be more accurate if the proportion of fewer category lesions was increased.https://www.mdpi.com/2075-4426/12/9/1361artificial intelligenceno codeendoscopycolonoscopycolonic neoplasms |
| spellingShingle | Eun Jeong Gong Chang Seok Bang Jae Jun Lee Young Joo Yang Gwang Ho Baik Impact of the Volume and Distribution of Training Datasets in the Development of Deep-Learning Models for the Diagnosis of Colorectal Polyps in Endoscopy Images Journal of Personalized Medicine artificial intelligence no code endoscopy colonoscopy colonic neoplasms |
| title | Impact of the Volume and Distribution of Training Datasets in the Development of Deep-Learning Models for the Diagnosis of Colorectal Polyps in Endoscopy Images |
| title_full | Impact of the Volume and Distribution of Training Datasets in the Development of Deep-Learning Models for the Diagnosis of Colorectal Polyps in Endoscopy Images |
| title_fullStr | Impact of the Volume and Distribution of Training Datasets in the Development of Deep-Learning Models for the Diagnosis of Colorectal Polyps in Endoscopy Images |
| title_full_unstemmed | Impact of the Volume and Distribution of Training Datasets in the Development of Deep-Learning Models for the Diagnosis of Colorectal Polyps in Endoscopy Images |
| title_short | Impact of the Volume and Distribution of Training Datasets in the Development of Deep-Learning Models for the Diagnosis of Colorectal Polyps in Endoscopy Images |
| title_sort | impact of the volume and distribution of training datasets in the development of deep learning models for the diagnosis of colorectal polyps in endoscopy images |
| topic | artificial intelligence no code endoscopy colonoscopy colonic neoplasms |
| url | https://www.mdpi.com/2075-4426/12/9/1361 |
| work_keys_str_mv | AT eunjeonggong impactofthevolumeanddistributionoftrainingdatasetsinthedevelopmentofdeeplearningmodelsforthediagnosisofcolorectalpolypsinendoscopyimages AT changseokbang impactofthevolumeanddistributionoftrainingdatasetsinthedevelopmentofdeeplearningmodelsforthediagnosisofcolorectalpolypsinendoscopyimages AT jaejunlee impactofthevolumeanddistributionoftrainingdatasetsinthedevelopmentofdeeplearningmodelsforthediagnosisofcolorectalpolypsinendoscopyimages AT youngjooyang impactofthevolumeanddistributionoftrainingdatasetsinthedevelopmentofdeeplearningmodelsforthediagnosisofcolorectalpolypsinendoscopyimages AT gwanghobaik impactofthevolumeanddistributionoftrainingdatasetsinthedevelopmentofdeeplearningmodelsforthediagnosisofcolorectalpolypsinendoscopyimages |