Comparison and validation of seven white matter hyperintensities segmentation software in elderly patients
Background: Manual segmentation is currently the gold standard to assess white matter hyperintensities (WMH), but it is time consuming and subject to intra and inter-operator variability. Purpose: To compare automatic methods to segment white matter hyperintensities (WMH) in the elderly in order to...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2020-01-01
|
| Series: | NeuroImage: Clinical |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2213158220301947 |
| _version_ | 1818260116332347392 |
|---|---|
| author | Quentin Vanderbecq Eric Xu Sebastian Ströer Baptiste Couvy-Duchesne Mauricio Diaz Melo Didier Dormont Olivier Colliot |
| author_facet | Quentin Vanderbecq Eric Xu Sebastian Ströer Baptiste Couvy-Duchesne Mauricio Diaz Melo Didier Dormont Olivier Colliot |
| author_sort | Quentin Vanderbecq |
| collection | DOAJ |
| description | Background: Manual segmentation is currently the gold standard to assess white matter hyperintensities (WMH), but it is time consuming and subject to intra and inter-operator variability. Purpose: To compare automatic methods to segment white matter hyperintensities (WMH) in the elderly in order to assist radiologist and researchers in selecting the most relevant method for application on clinical or research data. Material and Methods: We studied a research dataset composed of 147 patients, including 97 patients from the Alzheimer's Disease Neuroimaging Initiative (ADNI) 2 database and 50 patients from ADNI 3 and a clinical routine dataset comprising 60 patients referred for cognitive impairment at the Pitié-Salpêtrière hospital (imaged using four different MRI machines). We used manual segmentation as the gold standard reference. Both manual and automatic segmentations were performed using FLAIR MRI. We compared seven freely available methods that produce segmentation mask and are usable by a radiologist without a strong knowledge of computer programming: LGA (Schmidt et al., 2012), LPA (Schmidt, 2017), BIANCA (Griffanti et al., 2016), UBO detector (Jiang et al., 2018), W2MHS (Ithapu et al., 2014), nicMSlesion (with and without retraining) (Valverde et al., 2019, 2017). The primary outcome for assessing segmentation accuracy was the Dice similarity coefficient (DSC) between the manual and the automatic segmentation software. Secondary outcomes included five other metrics. Results: A deep learning approach, NicMSlesion, retrained on data from the research dataset ADNI, performed best on this research dataset (DSC: 0.595) and its DSC was significantly higher than that of all others. However, it ranked fifth on the clinical routine dataset and its performance severely dropped on data with artifacts. On the clinical routine dataset, the three top-ranked methods were LPA, SLS and BIANCA. Their performance did not differ significantly but was significantly higher than that of other methods. Conclusion: This work provides an objective comparison of methods for WMH segmentation. Results can be used by radiologists to select a tool. |
| first_indexed | 2024-12-12T18:26:13Z |
| format | Article |
| id | doaj.art-6eb68cb781894bc783b1b99536b9114d |
| institution | Directory Open Access Journal |
| issn | 2213-1582 |
| language | English |
| last_indexed | 2024-12-12T18:26:13Z |
| publishDate | 2020-01-01 |
| publisher | Elsevier |
| record_format | Article |
| series | NeuroImage: Clinical |
| spelling | doaj.art-6eb68cb781894bc783b1b99536b9114d2022-12-22T00:16:01ZengElsevierNeuroImage: Clinical2213-15822020-01-0127102357Comparison and validation of seven white matter hyperintensities segmentation software in elderly patientsQuentin Vanderbecq0Eric Xu1Sebastian Ströer2Baptiste Couvy-Duchesne3Mauricio Diaz Melo4Didier Dormont5Olivier Colliot6Institut du Cerveau et de la Moelle épinière, ICM, F-75013 Paris, France; Inserm, U 1127, F-75013 Paris, France; CNRS, UMR 7225, F-75013 Paris, France; Sorbonne Université, F-75013 Paris, France; Inria Paris, Aramis Project-Team, F-75013 Paris, France; Corresponding author: ICM – Brain and Spinal Cord Institute ARAMIS Team, Pitié-Salpêtrière Hospital 47-83, boulevard de l’Hôpital, 75651 Paris Cedex 13, France.Department of Radiology, University Hospital La Cavale Blanche, F-29200 Brest, FranceInstitute for Molecular Bioscience, the University of Queensland, 4072 Brisbane, AustraliaInstitut du Cerveau et de la Moelle épinière, ICM, F-75013 Paris, France; Inserm, U 1127, F-75013 Paris, France; CNRS, UMR 7225, F-75013 Paris, France; Sorbonne Université, F-75013 Paris, France; Inria Paris, Aramis Project-Team, F-75013 Paris, France; Institute for Molecular Bioscience, the University of Queensland, 4072 Brisbane, AustraliaInstitut du Cerveau et de la Moelle épinière, ICM, F-75013 Paris, France; Inria Paris, Aramis Project-Team, F-75013 Paris, FranceInstitut du Cerveau et de la Moelle épinière, ICM, F-75013 Paris, France; Inserm, U 1127, F-75013 Paris, France; CNRS, UMR 7225, F-75013 Paris, France; Inria Paris, Aramis Project-Team, F-75013 Paris, France; AP-HP, Hôpital de la Pitié-Salpêtrière, Department of Neuroradiology, F-75013 Paris, FranceInstitut du Cerveau et de la Moelle épinière, ICM, F-75013 Paris, France; Inserm, U 1127, F-75013 Paris, France; CNRS, UMR 7225, F-75013 Paris, France; Sorbonne Université, F-75013 Paris, France; Inria Paris, Aramis Project-Team, F-75013 Paris, France; AP-HP, Hôpital de la Pitié-Salpêtrière, Department of Neurology, Institut de la Mémoire et de la Maladie d’Alzheimer (IM2A), F-75013 Paris, FranceBackground: Manual segmentation is currently the gold standard to assess white matter hyperintensities (WMH), but it is time consuming and subject to intra and inter-operator variability. Purpose: To compare automatic methods to segment white matter hyperintensities (WMH) in the elderly in order to assist radiologist and researchers in selecting the most relevant method for application on clinical or research data. Material and Methods: We studied a research dataset composed of 147 patients, including 97 patients from the Alzheimer's Disease Neuroimaging Initiative (ADNI) 2 database and 50 patients from ADNI 3 and a clinical routine dataset comprising 60 patients referred for cognitive impairment at the Pitié-Salpêtrière hospital (imaged using four different MRI machines). We used manual segmentation as the gold standard reference. Both manual and automatic segmentations were performed using FLAIR MRI. We compared seven freely available methods that produce segmentation mask and are usable by a radiologist without a strong knowledge of computer programming: LGA (Schmidt et al., 2012), LPA (Schmidt, 2017), BIANCA (Griffanti et al., 2016), UBO detector (Jiang et al., 2018), W2MHS (Ithapu et al., 2014), nicMSlesion (with and without retraining) (Valverde et al., 2019, 2017). The primary outcome for assessing segmentation accuracy was the Dice similarity coefficient (DSC) between the manual and the automatic segmentation software. Secondary outcomes included five other metrics. Results: A deep learning approach, NicMSlesion, retrained on data from the research dataset ADNI, performed best on this research dataset (DSC: 0.595) and its DSC was significantly higher than that of all others. However, it ranked fifth on the clinical routine dataset and its performance severely dropped on data with artifacts. On the clinical routine dataset, the three top-ranked methods were LPA, SLS and BIANCA. Their performance did not differ significantly but was significantly higher than that of other methods. Conclusion: This work provides an objective comparison of methods for WMH segmentation. Results can be used by radiologists to select a tool.http://www.sciencedirect.com/science/article/pii/S2213158220301947White matter hyperintensityDementiaArtificial intelligenceSegmentationMicrovascular |
| spellingShingle | Quentin Vanderbecq Eric Xu Sebastian Ströer Baptiste Couvy-Duchesne Mauricio Diaz Melo Didier Dormont Olivier Colliot Comparison and validation of seven white matter hyperintensities segmentation software in elderly patients NeuroImage: Clinical White matter hyperintensity Dementia Artificial intelligence Segmentation Microvascular |
| title | Comparison and validation of seven white matter hyperintensities segmentation software in elderly patients |
| title_full | Comparison and validation of seven white matter hyperintensities segmentation software in elderly patients |
| title_fullStr | Comparison and validation of seven white matter hyperintensities segmentation software in elderly patients |
| title_full_unstemmed | Comparison and validation of seven white matter hyperintensities segmentation software in elderly patients |
| title_short | Comparison and validation of seven white matter hyperintensities segmentation software in elderly patients |
| title_sort | comparison and validation of seven white matter hyperintensities segmentation software in elderly patients |
| topic | White matter hyperintensity Dementia Artificial intelligence Segmentation Microvascular |
| url | http://www.sciencedirect.com/science/article/pii/S2213158220301947 |
| work_keys_str_mv | AT quentinvanderbecq comparisonandvalidationofsevenwhitematterhyperintensitiessegmentationsoftwareinelderlypatients AT ericxu comparisonandvalidationofsevenwhitematterhyperintensitiessegmentationsoftwareinelderlypatients AT sebastianstroer comparisonandvalidationofsevenwhitematterhyperintensitiessegmentationsoftwareinelderlypatients AT baptistecouvyduchesne comparisonandvalidationofsevenwhitematterhyperintensitiessegmentationsoftwareinelderlypatients AT mauriciodiazmelo comparisonandvalidationofsevenwhitematterhyperintensitiessegmentationsoftwareinelderlypatients AT didierdormont comparisonandvalidationofsevenwhitematterhyperintensitiessegmentationsoftwareinelderlypatients AT oliviercolliot comparisonandvalidationofsevenwhitematterhyperintensitiessegmentationsoftwareinelderlypatients |