Performance of AI-Based Automated Classifications of Whole-Body FDG PET in Clinical Practice: The CLARITI Project
Purpose: To assess the feasibility of a three-dimensional deep convolutional neural network (3D-CNN) for the general triage of whole-body FDG PET in daily clinical practice. Methods: An institutional clinical data warehouse working environment was devoted to this PET imaging purpose. Dedicated reque...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/9/5281 |
| _version_ | 1797603061018918912 |
|---|---|
| author | Arnaud Berenbaum Hervé Delingette Aurélien Maire Cécile Poret Claire Hassen-Khodja Stéphane Bréant Christel Daniel Patricia Martel Lamiae Grimaldi Marie Frank Emmanuel Durand Florent L. Besson |
| author_facet | Arnaud Berenbaum Hervé Delingette Aurélien Maire Cécile Poret Claire Hassen-Khodja Stéphane Bréant Christel Daniel Patricia Martel Lamiae Grimaldi Marie Frank Emmanuel Durand Florent L. Besson |
| author_sort | Arnaud Berenbaum |
| collection | DOAJ |
| description | Purpose: To assess the feasibility of a three-dimensional deep convolutional neural network (3D-CNN) for the general triage of whole-body FDG PET in daily clinical practice. Methods: An institutional clinical data warehouse working environment was devoted to this PET imaging purpose. Dedicated request procedures and data processing workflows were specifically developed within this infrastructure and applied retrospectively to a monocentric dataset as a proof of concept. A custom-made 3D-CNN was first trained and tested on an “unambiguous” well-balanced data sample, which included strictly normal and highly pathological scans. For the training phase, 90% of the data sample was used (learning set: 80%; validation set: 20%, 5-fold cross validation) and the remaining 10% constituted the test set. Finally, the model was applied to a “real-life” test set which included any scans taken. Text mining of the PET reports systematically combined with visual rechecking by an experienced reader served as the standard-of-truth for PET labeling. Results: From 8125 scans, 4963 PETs had processable cross-matched medical reports. For the “unambiguous” dataset (1084 PETs), the 3D-CNN’s overall results for sensitivity, specificity, positive and negative predictive values and likelihood ratios were 84%, 98%, 98%, 85%, 42.0 and 0.16, respectively (F1 score of 90%). When applied to the “real-life” dataset (4963 PETs), the sensitivity, NPV, LR+, LR− and F1 score substantially decreased (61%, 40%, 2.97, 0.49 and 73%, respectively), whereas the specificity and PPV remained high (79% and 90%). Conclusion: An AI-based triage of whole-body FDG PET is promising. Further studies are needed to overcome the challenges presented by the imperfection of real-life PET data. |
| first_indexed | 2024-03-11T04:25:04Z |
| format | Article |
| id | doaj.art-5a010272dfea45a5a59a9ef21b1e09f8 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T04:25:04Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-5a010272dfea45a5a59a9ef21b1e09f82023-11-17T22:31:51ZengMDPI AGApplied Sciences2076-34172023-04-01139528110.3390/app13095281Performance of AI-Based Automated Classifications of Whole-Body FDG PET in Clinical Practice: The CLARITI ProjectArnaud Berenbaum0Hervé Delingette1Aurélien Maire2Cécile Poret3Claire Hassen-Khodja4Stéphane Bréant5Christel Daniel6Patricia Martel7Lamiae Grimaldi8Marie Frank9Emmanuel Durand10Florent L. Besson11Department of Biophysics and Nuclear Medicine-Molecular Imaging, Hôpitaux Universitaires Paris-Saclay, Assistance Publique-Hôpitaux de Paris, 94270 Le Kremlin-Bicêtre, FranceINRIA EPIONE, Université Côte d’Azur, Inria Sophia Antipolis, Epione Research Project, 06902 Sophia Antipolis, FranceDepartment of Clinical Research and Innovation, Assistance Publique-Hôpitaux de Paris, 75012 Paris, FranceDepartment of Clinical Research and Innovation, Assistance Publique-Hôpitaux de Paris, 75012 Paris, FranceDepartment of Clinical Research and Innovation, Assistance Publique-Hôpitaux de Paris, 75012 Paris, FranceI&D PACTE, Assistance Publique-Hôpitaux de Paris, 75012 Paris, FranceI&D PACTE, Assistance Publique-Hôpitaux de Paris, 75012 Paris, FranceClinical Research Unit AP-HP, Paris-Saclay, Hôpital Raymond Poincare, School of Medicine Simone Veil, University Versailles Saint Quentin—University Paris Saclay, INSERM (National Institute of Health and Medical Research), CESP (Centre de Recherche en épidémiologie et Santé des Populations), Anti-Infective Evasion and Pharmacoepidemiology Team, 78180 Montigny-Le-Bretonneux, FranceClinical Research Unit AP-HP, Paris-Saclay, Hôpital Raymond Poincare, School of Medicine Simone Veil, University Versailles Saint Quentin—University Paris Saclay, INSERM (National Institute of Health and Medical Research), CESP (Centre de Recherche en épidémiologie et Santé des Populations), Anti-Infective Evasion and Pharmacoepidemiology Team, 78180 Montigny-Le-Bretonneux, FranceDepartment of Medical Information, Hôpitaux Universitaires Paris-Saclay, Assistance Publique-Hôpitaux de Paris, 94270 Le Kremlin-Bicêtre, FranceDepartment of Biophysics and Nuclear Medicine-Molecular Imaging, Hôpitaux Universitaires Paris-Saclay, Assistance Publique-Hôpitaux de Paris, 94270 Le Kremlin-Bicêtre, FranceDepartment of Biophysics and Nuclear Medicine-Molecular Imaging, Hôpitaux Universitaires Paris-Saclay, Assistance Publique-Hôpitaux de Paris, 94270 Le Kremlin-Bicêtre, FrancePurpose: To assess the feasibility of a three-dimensional deep convolutional neural network (3D-CNN) for the general triage of whole-body FDG PET in daily clinical practice. Methods: An institutional clinical data warehouse working environment was devoted to this PET imaging purpose. Dedicated request procedures and data processing workflows were specifically developed within this infrastructure and applied retrospectively to a monocentric dataset as a proof of concept. A custom-made 3D-CNN was first trained and tested on an “unambiguous” well-balanced data sample, which included strictly normal and highly pathological scans. For the training phase, 90% of the data sample was used (learning set: 80%; validation set: 20%, 5-fold cross validation) and the remaining 10% constituted the test set. Finally, the model was applied to a “real-life” test set which included any scans taken. Text mining of the PET reports systematically combined with visual rechecking by an experienced reader served as the standard-of-truth for PET labeling. Results: From 8125 scans, 4963 PETs had processable cross-matched medical reports. For the “unambiguous” dataset (1084 PETs), the 3D-CNN’s overall results for sensitivity, specificity, positive and negative predictive values and likelihood ratios were 84%, 98%, 98%, 85%, 42.0 and 0.16, respectively (F1 score of 90%). When applied to the “real-life” dataset (4963 PETs), the sensitivity, NPV, LR+, LR− and F1 score substantially decreased (61%, 40%, 2.97, 0.49 and 73%, respectively), whereas the specificity and PPV remained high (79% and 90%). Conclusion: An AI-based triage of whole-body FDG PET is promising. Further studies are needed to overcome the challenges presented by the imperfection of real-life PET data.https://www.mdpi.com/2076-3417/13/9/5281FDG PETartificial intelligencedeep learningconvolutional neural network |
| spellingShingle | Arnaud Berenbaum Hervé Delingette Aurélien Maire Cécile Poret Claire Hassen-Khodja Stéphane Bréant Christel Daniel Patricia Martel Lamiae Grimaldi Marie Frank Emmanuel Durand Florent L. Besson Performance of AI-Based Automated Classifications of Whole-Body FDG PET in Clinical Practice: The CLARITI Project Applied Sciences FDG PET artificial intelligence deep learning convolutional neural network |
| title | Performance of AI-Based Automated Classifications of Whole-Body FDG PET in Clinical Practice: The CLARITI Project |
| title_full | Performance of AI-Based Automated Classifications of Whole-Body FDG PET in Clinical Practice: The CLARITI Project |
| title_fullStr | Performance of AI-Based Automated Classifications of Whole-Body FDG PET in Clinical Practice: The CLARITI Project |
| title_full_unstemmed | Performance of AI-Based Automated Classifications of Whole-Body FDG PET in Clinical Practice: The CLARITI Project |
| title_short | Performance of AI-Based Automated Classifications of Whole-Body FDG PET in Clinical Practice: The CLARITI Project |
| title_sort | performance of ai based automated classifications of whole body fdg pet in clinical practice the clariti project |
| topic | FDG PET artificial intelligence deep learning convolutional neural network |
| url | https://www.mdpi.com/2076-3417/13/9/5281 |
| work_keys_str_mv | AT arnaudberenbaum performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT hervedelingette performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT aurelienmaire performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT cecileporet performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT clairehassenkhodja performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT stephanebreant performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT christeldaniel performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT patriciamartel performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT lamiaegrimaldi performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT mariefrank performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT emmanueldurand performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject AT florentlbesson performanceofaibasedautomatedclassificationsofwholebodyfdgpetinclinicalpracticetheclaritiproject |