Improving Generalizability of PET DL Algorithms: List-Mode Reconstructions Improve DOTATATE PET Hepatic Lesion Detection Performance
Deep learning (DL) algorithms used for DOTATATE PET lesion detection typically require large, well-annotated training datasets. These are difficult to obtain due to low incidence of gastroenteropancreatic neuroendocrine tumors (GEP-NETs) and the high cost of manual annotation. Furthermore, networks...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-02-01
|
| Series: | Bioengineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2306-5354/11/3/226 |
| _version_ | 1827307029639725056 |
|---|---|
| author | Xinyi Yang Michael Silosky Jonathan Wehrend Daniel V. Litwiller Muthiah Nachiappan Scott D. Metzler Debashis Ghosh Fuyong Xing Bennett B. Chin |
| author_facet | Xinyi Yang Michael Silosky Jonathan Wehrend Daniel V. Litwiller Muthiah Nachiappan Scott D. Metzler Debashis Ghosh Fuyong Xing Bennett B. Chin |
| author_sort | Xinyi Yang |
| collection | DOAJ |
| description | Deep learning (DL) algorithms used for DOTATATE PET lesion detection typically require large, well-annotated training datasets. These are difficult to obtain due to low incidence of gastroenteropancreatic neuroendocrine tumors (GEP-NETs) and the high cost of manual annotation. Furthermore, networks trained and tested with data acquired from site specific PET/CT instrumentation, acquisition and processing protocols have reduced performance when tested with offsite data. This lack of generalizability requires even larger, more diverse training datasets. The objective of this study is to investigate the feasibility of improving DL algorithm performance by better matching the background noise in training datasets to higher noise, out-of-domain testing datasets. <sup>68</sup>Ga-DOTATATE PET/CT datasets were obtained from two scanners: Scanner1, a state-of-the-art digital PET/CT (GE DMI PET/CT; n = 83 subjects), and Scanner2, an older-generation analog PET/CT (GE STE; n = 123 subjects). <i>Set1</i>, the data set from Scanner1, was reconstructed with standard clinical parameters (5 min; Q.Clear) and list-mode reconstructions (VPFXS 2, 3, 4, and 5-min). <i>Set2</i>, data from Scanner2 representing out-of-domain clinical scans, used standard iterative reconstruction (5 min; OSEM). A deep neural network was trained with each dataset: Network1 for Scanner1 and Network2 for Scanner2. DL performance (Network1) was tested with out-of-domain test data (<i>Set2</i>). To evaluate the effect of training sample size, we tested DL model performance using a fraction (25%, 50% and 75%) of <i>Set1</i> for training. Scanner1, list-mode 2-min reconstructed data demonstrated the most similar noise level compared that of <i>Set2</i>, resulting in the best performance (<i>F</i><sub>1</sub> = 0.713). This was not significantly different compared to the highest performance, upper-bound limit using in-domain training for Network2 (<i>F</i><sub>1</sub> = 0.755; <i>p</i>-value = 0.103). Regarding sample size, the <i>F</i>1 score significantly increased from 25% training data (<i>F</i><sub>1</sub> = 0.478) to 100% training data (<i>F</i><sub>1</sub> = 0.713; <i>p</i> < 0.001). List-mode data from modern PET scanners can be reconstructed to better match the noise properties of older scanners. Using existing data and their associated annotations dramatically reduces the cost and effort in generating these datasets and significantly improves the performance of existing DL algorithms. List-mode reconstructions can provide an efficient, low-cost method to improve DL algorithm generalizability. |
| first_indexed | 2024-04-24T18:32:25Z |
| format | Article |
| id | doaj.art-89635c6d2eda4c16a78e37d7d4f07a8f |
| institution | Directory Open Access Journal |
| issn | 2306-5354 |
| language | English |
| last_indexed | 2024-04-24T18:32:25Z |
| publishDate | 2024-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Bioengineering |
| spelling | doaj.art-89635c6d2eda4c16a78e37d7d4f07a8f2024-03-27T13:21:49ZengMDPI AGBioengineering2306-53542024-02-0111322610.3390/bioengineering11030226Improving Generalizability of PET DL Algorithms: List-Mode Reconstructions Improve DOTATATE PET Hepatic Lesion Detection PerformanceXinyi Yang0Michael Silosky1Jonathan Wehrend2Daniel V. Litwiller3Muthiah Nachiappan4Scott D. Metzler5Debashis Ghosh6Fuyong Xing7Bennett B. Chin8Department of Biostatistics and Informatics, University of Colorado Anschutz Medical Campus, Aurora, CO 80045, USADepartment of Radiology, University of Colorado Anschutz Medical Campus, Aurora, CO 80045, USADepartment of Radiology, Santa Clara Valley Medical Center, San Jose, CA 95128, USAGE HealthCare, Denver, CO 80222, USADepartment of Radiology, University of Colorado Anschutz Medical Campus, Aurora, CO 80045, USADepartment of Radiology, University of Pennsylvania, Philadelphia, PA 19104, USADepartment of Biostatistics and Informatics, University of Colorado Anschutz Medical Campus, Aurora, CO 80045, USADepartment of Biostatistics and Informatics, University of Colorado Anschutz Medical Campus, Aurora, CO 80045, USADepartment of Radiology, University of Colorado Anschutz Medical Campus, Aurora, CO 80045, USADeep learning (DL) algorithms used for DOTATATE PET lesion detection typically require large, well-annotated training datasets. These are difficult to obtain due to low incidence of gastroenteropancreatic neuroendocrine tumors (GEP-NETs) and the high cost of manual annotation. Furthermore, networks trained and tested with data acquired from site specific PET/CT instrumentation, acquisition and processing protocols have reduced performance when tested with offsite data. This lack of generalizability requires even larger, more diverse training datasets. The objective of this study is to investigate the feasibility of improving DL algorithm performance by better matching the background noise in training datasets to higher noise, out-of-domain testing datasets. <sup>68</sup>Ga-DOTATATE PET/CT datasets were obtained from two scanners: Scanner1, a state-of-the-art digital PET/CT (GE DMI PET/CT; n = 83 subjects), and Scanner2, an older-generation analog PET/CT (GE STE; n = 123 subjects). <i>Set1</i>, the data set from Scanner1, was reconstructed with standard clinical parameters (5 min; Q.Clear) and list-mode reconstructions (VPFXS 2, 3, 4, and 5-min). <i>Set2</i>, data from Scanner2 representing out-of-domain clinical scans, used standard iterative reconstruction (5 min; OSEM). A deep neural network was trained with each dataset: Network1 for Scanner1 and Network2 for Scanner2. DL performance (Network1) was tested with out-of-domain test data (<i>Set2</i>). To evaluate the effect of training sample size, we tested DL model performance using a fraction (25%, 50% and 75%) of <i>Set1</i> for training. Scanner1, list-mode 2-min reconstructed data demonstrated the most similar noise level compared that of <i>Set2</i>, resulting in the best performance (<i>F</i><sub>1</sub> = 0.713). This was not significantly different compared to the highest performance, upper-bound limit using in-domain training for Network2 (<i>F</i><sub>1</sub> = 0.755; <i>p</i>-value = 0.103). Regarding sample size, the <i>F</i>1 score significantly increased from 25% training data (<i>F</i><sub>1</sub> = 0.478) to 100% training data (<i>F</i><sub>1</sub> = 0.713; <i>p</i> < 0.001). List-mode data from modern PET scanners can be reconstructed to better match the noise properties of older scanners. Using existing data and their associated annotations dramatically reduces the cost and effort in generating these datasets and significantly improves the performance of existing DL algorithms. List-mode reconstructions can provide an efficient, low-cost method to improve DL algorithm generalizability.https://www.mdpi.com/2306-5354/11/3/226deep learningconvolutional neural networkgastroenteropancreatic neuroendocrine tumorGEP-NETDOTATATEpositron emission tomography |
| spellingShingle | Xinyi Yang Michael Silosky Jonathan Wehrend Daniel V. Litwiller Muthiah Nachiappan Scott D. Metzler Debashis Ghosh Fuyong Xing Bennett B. Chin Improving Generalizability of PET DL Algorithms: List-Mode Reconstructions Improve DOTATATE PET Hepatic Lesion Detection Performance Bioengineering deep learning convolutional neural network gastroenteropancreatic neuroendocrine tumor GEP-NET DOTATATE positron emission tomography |
| title | Improving Generalizability of PET DL Algorithms: List-Mode Reconstructions Improve DOTATATE PET Hepatic Lesion Detection Performance |
| title_full | Improving Generalizability of PET DL Algorithms: List-Mode Reconstructions Improve DOTATATE PET Hepatic Lesion Detection Performance |
| title_fullStr | Improving Generalizability of PET DL Algorithms: List-Mode Reconstructions Improve DOTATATE PET Hepatic Lesion Detection Performance |
| title_full_unstemmed | Improving Generalizability of PET DL Algorithms: List-Mode Reconstructions Improve DOTATATE PET Hepatic Lesion Detection Performance |
| title_short | Improving Generalizability of PET DL Algorithms: List-Mode Reconstructions Improve DOTATATE PET Hepatic Lesion Detection Performance |
| title_sort | improving generalizability of pet dl algorithms list mode reconstructions improve dotatate pet hepatic lesion detection performance |
| topic | deep learning convolutional neural network gastroenteropancreatic neuroendocrine tumor GEP-NET DOTATATE positron emission tomography |
| url | https://www.mdpi.com/2306-5354/11/3/226 |
| work_keys_str_mv | AT xinyiyang improvinggeneralizabilityofpetdlalgorithmslistmodereconstructionsimprovedotatatepethepaticlesiondetectionperformance AT michaelsilosky improvinggeneralizabilityofpetdlalgorithmslistmodereconstructionsimprovedotatatepethepaticlesiondetectionperformance AT jonathanwehrend improvinggeneralizabilityofpetdlalgorithmslistmodereconstructionsimprovedotatatepethepaticlesiondetectionperformance AT danielvlitwiller improvinggeneralizabilityofpetdlalgorithmslistmodereconstructionsimprovedotatatepethepaticlesiondetectionperformance AT muthiahnachiappan improvinggeneralizabilityofpetdlalgorithmslistmodereconstructionsimprovedotatatepethepaticlesiondetectionperformance AT scottdmetzler improvinggeneralizabilityofpetdlalgorithmslistmodereconstructionsimprovedotatatepethepaticlesiondetectionperformance AT debashisghosh improvinggeneralizabilityofpetdlalgorithmslistmodereconstructionsimprovedotatatepethepaticlesiondetectionperformance AT fuyongxing improvinggeneralizabilityofpetdlalgorithmslistmodereconstructionsimprovedotatatepethepaticlesiondetectionperformance AT bennettbchin improvinggeneralizabilityofpetdlalgorithmslistmodereconstructionsimprovedotatatepethepaticlesiondetectionperformance |