Deep learning evaluation of biomarkers from echocardiogram videos
Background: Laboratory testing is routinely used to assay blood biomarkers to provide information on physiologic state beyond what clinicians can evaluate from interpreting medical imaging. We hypothesized that deep learning interpretation of echocardiogram videos can provide additional value in und...
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2021-11-01
|
| Series: | EBioMedicine |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2352396421004060 |
| _version_ | 1818355566631714816 |
|---|---|
| author | J Weston Hughes Neal Yuan Bryan He Jiahong Ouyang Joseph Ebinger Patrick Botting Jasper Lee John Theurer James E. Tooley Koen Nieman Matthew P. Lungren David H. Liang Ingela Schnittger Jonathan H. Chen Euan A. Ashley Susan Cheng David Ouyang James Y. Zou |
| author_facet | J Weston Hughes Neal Yuan Bryan He Jiahong Ouyang Joseph Ebinger Patrick Botting Jasper Lee John Theurer James E. Tooley Koen Nieman Matthew P. Lungren David H. Liang Ingela Schnittger Jonathan H. Chen Euan A. Ashley Susan Cheng David Ouyang James Y. Zou |
| author_sort | J Weston Hughes |
| collection | DOAJ |
| description | Background: Laboratory testing is routinely used to assay blood biomarkers to provide information on physiologic state beyond what clinicians can evaluate from interpreting medical imaging. We hypothesized that deep learning interpretation of echocardiogram videos can provide additional value in understanding disease states and can evaluate common biomarkers results. Methods: We developed EchoNet-Labs, a video-based deep learning algorithm to detect evidence of anemia, elevated B-type natriuretic peptide (BNP), troponin I, and blood urea nitrogen (BUN), as well as values of ten additional lab tests directly from echocardiograms. We included patients (n = 39,460) aged 18 years or older with one or more apical-4-chamber echocardiogram videos (n = 70,066) from Stanford Healthcare for training and internal testing of EchoNet-Lab's performance in estimating the most proximal biomarker result. Without fine-tuning, the performance of EchoNet-Labs was further evaluated on an additional external test dataset (n = 1,301) from Cedars-Sinai Medical Center. We calculated the area under the curve (AUC) of the receiver operating characteristic curve for the internal and external test datasets. Findings: On the held-out test set of Stanford patients not previously seen during model training, EchoNet-Labs achieved an AUC of 0.80 (0.79-0.81) in detecting anemia (low hemoglobin), 0.86 (0.85-0.88) in detecting elevated BNP, 0.75 (0.73-0.78) in detecting elevated troponin I, and 0.74 (0.72-0.76) in detecting elevated BUN. On the external test dataset from Cedars-Sinai, EchoNet-Labs achieved an AUC of 0.80 (0.77-0.82) in detecting anemia, of 0.82 (0.79-0.84) in detecting elevated BNP, of 0.75 (0.72-0.78) in detecting elevated troponin I, and of 0.69 (0.66-0.71) in detecting elevated BUN. We further demonstrate the utility of the model in detecting abnormalities in 10 additional lab tests. We investigate the features necessary for EchoNet-Labs to make successful detection and identify potential mechanisms for each biomarker using well-known and novel explainability techniques. Interpretation: These results show that deep learning applied to diagnostic imaging can provide additional clinical value and identify phenotypic information beyond current imaging interpretation methods. Funding: J.W.H. and B.H. are supported by the NSF Graduate Research Fellowship. D.O. is supported by NIH K99 HL157421-01. J.Y.Z. is supported by NSF CAREER 1942926, NIH R21 MD012867-01, NIH P30AG059307 and by a Chan-Zuckerberg Biohub Fellowship. |
| first_indexed | 2024-12-13T19:43:21Z |
| format | Article |
| id | doaj.art-d63c1ebdec3940da9957bade5fd02847 |
| institution | Directory Open Access Journal |
| issn | 2352-3964 |
| language | English |
| last_indexed | 2024-12-13T19:43:21Z |
| publishDate | 2021-11-01 |
| publisher | Elsevier |
| record_format | Article |
| series | EBioMedicine |
| spelling | doaj.art-d63c1ebdec3940da9957bade5fd028472022-12-21T23:33:38ZengElsevierEBioMedicine2352-39642021-11-0173103613Deep learning evaluation of biomarkers from echocardiogram videosJ Weston Hughes0Neal Yuan1Bryan He2Jiahong Ouyang3Joseph Ebinger4Patrick Botting5Jasper Lee6John Theurer7James E. Tooley8Koen Nieman9Matthew P. Lungren10David H. Liang11Ingela Schnittger12Jonathan H. Chen13Euan A. Ashley14Susan Cheng15David Ouyang16James Y. Zou17Department of Computer Science, Stanford University, Palo Alto, CA 94025Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, 90048Department of Computer Science, Stanford University, Palo Alto, CA 94025Department of Electrical Engineering, Stanford University, Palo Alto, CA, 94025Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, 90048Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, 90048Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, 90048Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, 90048Department of Medicine, Stanford University, Palo Alto, CA, 94025Department of Medicine, Stanford University, Palo Alto, CA, 94025; Department of Radiology, Stanford University, Palo Alto, CA, 94025Department of Radiology, Stanford University, Palo Alto, CA, 94025Department of Medicine, Stanford University, Palo Alto, CA, 94025Department of Medicine, Stanford University, Palo Alto, CA, 94025Department of Medicine, Stanford University, Palo Alto, CA, 94025Department of Medicine, Stanford University, Palo Alto, CA, 94025Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, 90048Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, 90048; Corresponding author.Department of Computer Science, Stanford University, Palo Alto, CA 94025; Department of Electrical Engineering, Stanford University, Palo Alto, CA, 94025; Department of Biomedical Data Science, Stanford University, Palo Alto, CA, 94025; Corresponding author.Background: Laboratory testing is routinely used to assay blood biomarkers to provide information on physiologic state beyond what clinicians can evaluate from interpreting medical imaging. We hypothesized that deep learning interpretation of echocardiogram videos can provide additional value in understanding disease states and can evaluate common biomarkers results. Methods: We developed EchoNet-Labs, a video-based deep learning algorithm to detect evidence of anemia, elevated B-type natriuretic peptide (BNP), troponin I, and blood urea nitrogen (BUN), as well as values of ten additional lab tests directly from echocardiograms. We included patients (n = 39,460) aged 18 years or older with one or more apical-4-chamber echocardiogram videos (n = 70,066) from Stanford Healthcare for training and internal testing of EchoNet-Lab's performance in estimating the most proximal biomarker result. Without fine-tuning, the performance of EchoNet-Labs was further evaluated on an additional external test dataset (n = 1,301) from Cedars-Sinai Medical Center. We calculated the area under the curve (AUC) of the receiver operating characteristic curve for the internal and external test datasets. Findings: On the held-out test set of Stanford patients not previously seen during model training, EchoNet-Labs achieved an AUC of 0.80 (0.79-0.81) in detecting anemia (low hemoglobin), 0.86 (0.85-0.88) in detecting elevated BNP, 0.75 (0.73-0.78) in detecting elevated troponin I, and 0.74 (0.72-0.76) in detecting elevated BUN. On the external test dataset from Cedars-Sinai, EchoNet-Labs achieved an AUC of 0.80 (0.77-0.82) in detecting anemia, of 0.82 (0.79-0.84) in detecting elevated BNP, of 0.75 (0.72-0.78) in detecting elevated troponin I, and of 0.69 (0.66-0.71) in detecting elevated BUN. We further demonstrate the utility of the model in detecting abnormalities in 10 additional lab tests. We investigate the features necessary for EchoNet-Labs to make successful detection and identify potential mechanisms for each biomarker using well-known and novel explainability techniques. Interpretation: These results show that deep learning applied to diagnostic imaging can provide additional clinical value and identify phenotypic information beyond current imaging interpretation methods. Funding: J.W.H. and B.H. are supported by the NSF Graduate Research Fellowship. D.O. is supported by NIH K99 HL157421-01. J.Y.Z. is supported by NSF CAREER 1942926, NIH R21 MD012867-01, NIH P30AG059307 and by a Chan-Zuckerberg Biohub Fellowship.http://www.sciencedirect.com/science/article/pii/S2352396421004060Deep learningArtificial intelligenceEchocardiography |
| spellingShingle | J Weston Hughes Neal Yuan Bryan He Jiahong Ouyang Joseph Ebinger Patrick Botting Jasper Lee John Theurer James E. Tooley Koen Nieman Matthew P. Lungren David H. Liang Ingela Schnittger Jonathan H. Chen Euan A. Ashley Susan Cheng David Ouyang James Y. Zou Deep learning evaluation of biomarkers from echocardiogram videos EBioMedicine Deep learning Artificial intelligence Echocardiography |
| title | Deep learning evaluation of biomarkers from echocardiogram videos |
| title_full | Deep learning evaluation of biomarkers from echocardiogram videos |
| title_fullStr | Deep learning evaluation of biomarkers from echocardiogram videos |
| title_full_unstemmed | Deep learning evaluation of biomarkers from echocardiogram videos |
| title_short | Deep learning evaluation of biomarkers from echocardiogram videos |
| title_sort | deep learning evaluation of biomarkers from echocardiogram videos |
| topic | Deep learning Artificial intelligence Echocardiography |
| url | http://www.sciencedirect.com/science/article/pii/S2352396421004060 |
| work_keys_str_mv | AT jwestonhughes deeplearningevaluationofbiomarkersfromechocardiogramvideos AT nealyuan deeplearningevaluationofbiomarkersfromechocardiogramvideos AT bryanhe deeplearningevaluationofbiomarkersfromechocardiogramvideos AT jiahongouyang deeplearningevaluationofbiomarkersfromechocardiogramvideos AT josephebinger deeplearningevaluationofbiomarkersfromechocardiogramvideos AT patrickbotting deeplearningevaluationofbiomarkersfromechocardiogramvideos AT jasperlee deeplearningevaluationofbiomarkersfromechocardiogramvideos AT johntheurer deeplearningevaluationofbiomarkersfromechocardiogramvideos AT jamesetooley deeplearningevaluationofbiomarkersfromechocardiogramvideos AT koennieman deeplearningevaluationofbiomarkersfromechocardiogramvideos AT matthewplungren deeplearningevaluationofbiomarkersfromechocardiogramvideos AT davidhliang deeplearningevaluationofbiomarkersfromechocardiogramvideos AT ingelaschnittger deeplearningevaluationofbiomarkersfromechocardiogramvideos AT jonathanhchen deeplearningevaluationofbiomarkersfromechocardiogramvideos AT euanaashley deeplearningevaluationofbiomarkersfromechocardiogramvideos AT susancheng deeplearningevaluationofbiomarkersfromechocardiogramvideos AT davidouyang deeplearningevaluationofbiomarkersfromechocardiogramvideos AT jamesyzou deeplearningevaluationofbiomarkersfromechocardiogramvideos |