Real-World Verification of Artificial Intelligence Algorithm-Assisted Auscultation of Breath Sounds in Children
Objective: Lung auscultation plays an important role in the diagnosis of pulmonary diseases in children. The objective of this study was to evaluate the use of an artificial intelligence (AI) algorithm for the detection of breath sounds in a real clinical environment among children with pulmonary di...
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-03-01
|
| Series: | Frontiers in Pediatrics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fped.2021.627337/full |
| _version_ | 1818414918214352896 |
|---|---|
| author | Jing Zhang Han-Song Wang Han-Song Wang Hong-Yuan Zhou Bin Dong Lei Zhang Fen Zhang Shi-Jian Liu Yu-Fen Wu Shu-Hua Yuan Ming-Yu Tang Wen-Fang Dong Jie Lin Ming Chen Xing Tong Lie-Bin Zhao Lie-Bin Zhao Yong Yin |
| author_facet | Jing Zhang Han-Song Wang Han-Song Wang Hong-Yuan Zhou Bin Dong Lei Zhang Fen Zhang Shi-Jian Liu Yu-Fen Wu Shu-Hua Yuan Ming-Yu Tang Wen-Fang Dong Jie Lin Ming Chen Xing Tong Lie-Bin Zhao Lie-Bin Zhao Yong Yin |
| author_sort | Jing Zhang |
| collection | DOAJ |
| description | Objective: Lung auscultation plays an important role in the diagnosis of pulmonary diseases in children. The objective of this study was to evaluate the use of an artificial intelligence (AI) algorithm for the detection of breath sounds in a real clinical environment among children with pulmonary diseases.Method: The auscultations of breath sounds were collected in the respiratory department of Shanghai Children's Medical Center (SCMC) by using an electronic stethoscope. The discrimination results for all chest locations with respect to a gold standard (GS) established by 2 experienced pediatric pulmonologists from SCMC and 6 general pediatricians were recorded. The accuracy, sensitivity, specificity, precision, and F1-score of the AI algorithm and general pediatricians with respect to the GS were evaluated. Meanwhile, the performance of the AI algorithm for different patient ages and recording locations was evaluated.Result: A total of 112 hospitalized children with pulmonary diseases were recruited for the study from May to December 2019. A total of 672 breath sounds were collected, and 627 (93.3%) breath sounds, including 159 crackles (23.1%), 264 wheeze (38.4%), and 264 normal breath sounds (38.4%), were fully analyzed by the AI algorithm. The accuracy of the detection of adventitious breath sounds by the AI algorithm and general pediatricians with respect to the GS were 77.7% and 59.9% (p < 0.001), respectively. The sensitivity, specificity, and F1-score in the detection of crackles and wheeze from the AI algorithm were higher than those from the general pediatricians (crackles 81.1 vs. 47.8%, 94.1 vs. 77.1%, and 80.9 vs. 42.74%, respectively; wheeze 86.4 vs. 82.2%, 83.0 vs. 72.1%, and 80.9 vs. 72.5%, respectively; p < 0.001). Performance varied according to the age of the patient, with patients younger than 12 months yielding the highest accuracy (81.3%, p < 0.001) among the age groups.Conclusion: In a real clinical environment, children's breath sounds were collected and transmitted remotely by an electronic stethoscope; these breath sounds could be recognized by both pediatricians and an AI algorithm. The ability of the AI algorithm to analyze adventitious breath sounds was better than that of the general pediatricians. |
| first_indexed | 2024-12-14T11:26:44Z |
| format | Article |
| id | doaj.art-6d8386029223413c925c066892cf8d7e |
| institution | Directory Open Access Journal |
| issn | 2296-2360 |
| language | English |
| last_indexed | 2024-12-14T11:26:44Z |
| publishDate | 2021-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Pediatrics |
| spelling | doaj.art-6d8386029223413c925c066892cf8d7e2022-12-21T23:03:31ZengFrontiers Media S.A.Frontiers in Pediatrics2296-23602021-03-01910.3389/fped.2021.627337627337Real-World Verification of Artificial Intelligence Algorithm-Assisted Auscultation of Breath Sounds in ChildrenJing Zhang0Han-Song Wang1Han-Song Wang2Hong-Yuan Zhou3Bin Dong4Lei Zhang5Fen Zhang6Shi-Jian Liu7Yu-Fen Wu8Shu-Hua Yuan9Ming-Yu Tang10Wen-Fang Dong11Jie Lin12Ming Chen13Xing Tong14Lie-Bin Zhao15Lie-Bin Zhao16Yong Yin17Department of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaPaediatric AI Clinical Application and Research Center, Shanghai Children's Medical Center, Shanghai, ChinaChild Health Advocacy Institute, China Hospital Development Institute of Shanghai Jiao Tong University, Shanghai, ChinaTuoxiao Intelligent Technology Company, Shanghai, ChinaPaediatric AI Clinical Application and Research Center, Shanghai Children's Medical Center, Shanghai, ChinaDepartment of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaDepartment of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaPaediatric AI Clinical Application and Research Center, Shanghai Children's Medical Center, Shanghai, ChinaDepartment of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaDepartment of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaDepartment of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaDepartment of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaDepartment of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaDepartment of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaDepartment of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaPaediatric AI Clinical Application and Research Center, Shanghai Children's Medical Center, Shanghai, ChinaChild Health Advocacy Institute, China Hospital Development Institute of Shanghai Jiao Tong University, Shanghai, ChinaDepartment of Respiratory Medicine, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, ChinaObjective: Lung auscultation plays an important role in the diagnosis of pulmonary diseases in children. The objective of this study was to evaluate the use of an artificial intelligence (AI) algorithm for the detection of breath sounds in a real clinical environment among children with pulmonary diseases.Method: The auscultations of breath sounds were collected in the respiratory department of Shanghai Children's Medical Center (SCMC) by using an electronic stethoscope. The discrimination results for all chest locations with respect to a gold standard (GS) established by 2 experienced pediatric pulmonologists from SCMC and 6 general pediatricians were recorded. The accuracy, sensitivity, specificity, precision, and F1-score of the AI algorithm and general pediatricians with respect to the GS were evaluated. Meanwhile, the performance of the AI algorithm for different patient ages and recording locations was evaluated.Result: A total of 112 hospitalized children with pulmonary diseases were recruited for the study from May to December 2019. A total of 672 breath sounds were collected, and 627 (93.3%) breath sounds, including 159 crackles (23.1%), 264 wheeze (38.4%), and 264 normal breath sounds (38.4%), were fully analyzed by the AI algorithm. The accuracy of the detection of adventitious breath sounds by the AI algorithm and general pediatricians with respect to the GS were 77.7% and 59.9% (p < 0.001), respectively. The sensitivity, specificity, and F1-score in the detection of crackles and wheeze from the AI algorithm were higher than those from the general pediatricians (crackles 81.1 vs. 47.8%, 94.1 vs. 77.1%, and 80.9 vs. 42.74%, respectively; wheeze 86.4 vs. 82.2%, 83.0 vs. 72.1%, and 80.9 vs. 72.5%, respectively; p < 0.001). Performance varied according to the age of the patient, with patients younger than 12 months yielding the highest accuracy (81.3%, p < 0.001) among the age groups.Conclusion: In a real clinical environment, children's breath sounds were collected and transmitted remotely by an electronic stethoscope; these breath sounds could be recognized by both pediatricians and an AI algorithm. The ability of the AI algorithm to analyze adventitious breath sounds was better than that of the general pediatricians.https://www.frontiersin.org/articles/10.3389/fped.2021.627337/fullauscultationbreath soundelectronic stethoscopeartificial intelligencechildren |
| spellingShingle | Jing Zhang Han-Song Wang Han-Song Wang Hong-Yuan Zhou Bin Dong Lei Zhang Fen Zhang Shi-Jian Liu Yu-Fen Wu Shu-Hua Yuan Ming-Yu Tang Wen-Fang Dong Jie Lin Ming Chen Xing Tong Lie-Bin Zhao Lie-Bin Zhao Yong Yin Real-World Verification of Artificial Intelligence Algorithm-Assisted Auscultation of Breath Sounds in Children Frontiers in Pediatrics auscultation breath sound electronic stethoscope artificial intelligence children |
| title | Real-World Verification of Artificial Intelligence Algorithm-Assisted Auscultation of Breath Sounds in Children |
| title_full | Real-World Verification of Artificial Intelligence Algorithm-Assisted Auscultation of Breath Sounds in Children |
| title_fullStr | Real-World Verification of Artificial Intelligence Algorithm-Assisted Auscultation of Breath Sounds in Children |
| title_full_unstemmed | Real-World Verification of Artificial Intelligence Algorithm-Assisted Auscultation of Breath Sounds in Children |
| title_short | Real-World Verification of Artificial Intelligence Algorithm-Assisted Auscultation of Breath Sounds in Children |
| title_sort | real world verification of artificial intelligence algorithm assisted auscultation of breath sounds in children |
| topic | auscultation breath sound electronic stethoscope artificial intelligence children |
| url | https://www.frontiersin.org/articles/10.3389/fped.2021.627337/full |
| work_keys_str_mv | AT jingzhang realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT hansongwang realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT hansongwang realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT hongyuanzhou realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT bindong realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT leizhang realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT fenzhang realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT shijianliu realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT yufenwu realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT shuhuayuan realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT mingyutang realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT wenfangdong realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT jielin realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT mingchen realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT xingtong realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT liebinzhao realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT liebinzhao realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren AT yongyin realworldverificationofartificialintelligencealgorithmassistedauscultationofbreathsoundsinchildren |