A Simulation Toolkit for Testing the Sensitivity and Accuracy of Corticometry Pipelines
In recent years, the replicability of neuroimaging findings has become an important concern to the research community. Neuroimaging pipelines consist of myriad numerical procedures, which can have a cumulative effect on the accuracy of findings. To address this problem, we propose a method for simul...
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-07-01
|
| Series: | Frontiers in Neuroinformatics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fninf.2021.665560/full |
| _version_ | 1818595002443366400 |
|---|---|
| author | Mona OmidYeganeh Najmeh Khalili-Mahani Najmeh Khalili-Mahani Patrick Bermudez Alison Ross Claude Lepage Robert D. Vincent S. Jeon Lindsay B. Lewis S. Das Alex P. Zijdenbos Pierre Rioux Reza Adalat Matthijs C. Van Eede Alan C. Evans |
| author_facet | Mona OmidYeganeh Najmeh Khalili-Mahani Najmeh Khalili-Mahani Patrick Bermudez Alison Ross Claude Lepage Robert D. Vincent S. Jeon Lindsay B. Lewis S. Das Alex P. Zijdenbos Pierre Rioux Reza Adalat Matthijs C. Van Eede Alan C. Evans |
| author_sort | Mona OmidYeganeh |
| collection | DOAJ |
| description | In recent years, the replicability of neuroimaging findings has become an important concern to the research community. Neuroimaging pipelines consist of myriad numerical procedures, which can have a cumulative effect on the accuracy of findings. To address this problem, we propose a method for simulating artificial lesions in the brain in order to estimate the sensitivity and specificity of lesion detection, using different automated corticometry pipelines. We have applied this method to different versions of two widely used neuroimaging pipelines (CIVET and FreeSurfer), in terms of coefficients of variation; sensitivity and specificity of detecting lesions in 4 different regions of interest in the cortex, while introducing variations to the lesion size, the blurring kernel used prior to statistical analyses, and different thickness metrics (in CIVET). These variations are tested in a between-subject design (in two random groups, with and without lesions, using T1-weigted MRIs of 152 individuals from the International Consortium of Brain Mapping (ICBM) dataset) and in a within-subject pre-/post-lesion design [using 21 T1-Weighted MRIs of a single adult individual, scanned in the Infant Brain Imaging Study (IBIS)]. The simulation method is sensitive to partial volume effect and lesion size. Comparisons between pipelines illustrate the ability of this method to uncover differences in sensitivity and specificity of lesion detection. We propose that this method be adopted in the workflow of software development and release. |
| first_indexed | 2024-12-16T11:09:05Z |
| format | Article |
| id | doaj.art-e4a0593f923e45669934222078da715f |
| institution | Directory Open Access Journal |
| issn | 1662-5196 |
| language | English |
| last_indexed | 2024-12-16T11:09:05Z |
| publishDate | 2021-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroinformatics |
| spelling | doaj.art-e4a0593f923e45669934222078da715f2022-12-21T22:33:47ZengFrontiers Media S.A.Frontiers in Neuroinformatics1662-51962021-07-011510.3389/fninf.2021.665560665560A Simulation Toolkit for Testing the Sensitivity and Accuracy of Corticometry PipelinesMona OmidYeganeh0Najmeh Khalili-Mahani1Najmeh Khalili-Mahani2Patrick Bermudez3Alison Ross4Claude Lepage5Robert D. Vincent6S. Jeon7Lindsay B. Lewis8S. Das9Alex P. Zijdenbos10Pierre Rioux11Reza Adalat12Matthijs C. Van Eede13Alan C. Evans14McGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaPERFORM Centre, Concordia University, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaSick Kids Research Institute, Toronto, ON, CanadaMcGill Centre for Integrative Neuroscience, Montreal Neurological Institute, Montreal, QC, CanadaIn recent years, the replicability of neuroimaging findings has become an important concern to the research community. Neuroimaging pipelines consist of myriad numerical procedures, which can have a cumulative effect on the accuracy of findings. To address this problem, we propose a method for simulating artificial lesions in the brain in order to estimate the sensitivity and specificity of lesion detection, using different automated corticometry pipelines. We have applied this method to different versions of two widely used neuroimaging pipelines (CIVET and FreeSurfer), in terms of coefficients of variation; sensitivity and specificity of detecting lesions in 4 different regions of interest in the cortex, while introducing variations to the lesion size, the blurring kernel used prior to statistical analyses, and different thickness metrics (in CIVET). These variations are tested in a between-subject design (in two random groups, with and without lesions, using T1-weigted MRIs of 152 individuals from the International Consortium of Brain Mapping (ICBM) dataset) and in a within-subject pre-/post-lesion design [using 21 T1-Weighted MRIs of a single adult individual, scanned in the Infant Brain Imaging Study (IBIS)]. The simulation method is sensitive to partial volume effect and lesion size. Comparisons between pipelines illustrate the ability of this method to uncover differences in sensitivity and specificity of lesion detection. We propose that this method be adopted in the workflow of software development and release.https://www.frontiersin.org/articles/10.3389/fninf.2021.665560/fullreproducible neuroimagingcortical thicknesslesion simulationpipeline accuracybrain morphometrystatistical parametric mapping |
| spellingShingle | Mona OmidYeganeh Najmeh Khalili-Mahani Najmeh Khalili-Mahani Patrick Bermudez Alison Ross Claude Lepage Robert D. Vincent S. Jeon Lindsay B. Lewis S. Das Alex P. Zijdenbos Pierre Rioux Reza Adalat Matthijs C. Van Eede Alan C. Evans A Simulation Toolkit for Testing the Sensitivity and Accuracy of Corticometry Pipelines Frontiers in Neuroinformatics reproducible neuroimaging cortical thickness lesion simulation pipeline accuracy brain morphometry statistical parametric mapping |
| title | A Simulation Toolkit for Testing the Sensitivity and Accuracy of Corticometry Pipelines |
| title_full | A Simulation Toolkit for Testing the Sensitivity and Accuracy of Corticometry Pipelines |
| title_fullStr | A Simulation Toolkit for Testing the Sensitivity and Accuracy of Corticometry Pipelines |
| title_full_unstemmed | A Simulation Toolkit for Testing the Sensitivity and Accuracy of Corticometry Pipelines |
| title_short | A Simulation Toolkit for Testing the Sensitivity and Accuracy of Corticometry Pipelines |
| title_sort | simulation toolkit for testing the sensitivity and accuracy of corticometry pipelines |
| topic | reproducible neuroimaging cortical thickness lesion simulation pipeline accuracy brain morphometry statistical parametric mapping |
| url | https://www.frontiersin.org/articles/10.3389/fninf.2021.665560/full |
| work_keys_str_mv | AT monaomidyeganeh asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT najmehkhalilimahani asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT najmehkhalilimahani asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT patrickbermudez asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT alisonross asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT claudelepage asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT robertdvincent asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT sjeon asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT lindsayblewis asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT sdas asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT alexpzijdenbos asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT pierrerioux asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT rezaadalat asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT matthijscvaneede asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT alancevans asimulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT monaomidyeganeh simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT najmehkhalilimahani simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT najmehkhalilimahani simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT patrickbermudez simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT alisonross simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT claudelepage simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT robertdvincent simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT sjeon simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT lindsayblewis simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT sdas simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT alexpzijdenbos simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT pierrerioux simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT rezaadalat simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT matthijscvaneede simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines AT alancevans simulationtoolkitfortestingthesensitivityandaccuracyofcorticometrypipelines |