Fairness and generalizability of OCT normative databases: a comparative analysis
Abstract Purpose In supervised Machine Learning algorithms, labels and reports are important in model development. To provide a normality assessment, the OCT has an in-built normative database that provides a color base scale from the measurement database comparison. This article aims to evaluate an...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-08-01
|
| Series: | International Journal of Retina and Vitreous |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40942-023-00459-8 |
| _version_ | 1827709210177044480 |
|---|---|
| author | Luis Filipe Nakayama Lucas Zago Ribeiro Juliana Angelica Estevão de Oliveira João Carlos Ramos Gonçalves de Matos William Greig Mitchell Fernando Korn Malerbi Leo Anthony Celi Caio Vinicius Saito Regatieri |
| author_facet | Luis Filipe Nakayama Lucas Zago Ribeiro Juliana Angelica Estevão de Oliveira João Carlos Ramos Gonçalves de Matos William Greig Mitchell Fernando Korn Malerbi Leo Anthony Celi Caio Vinicius Saito Regatieri |
| author_sort | Luis Filipe Nakayama |
| collection | DOAJ |
| description | Abstract Purpose In supervised Machine Learning algorithms, labels and reports are important in model development. To provide a normality assessment, the OCT has an in-built normative database that provides a color base scale from the measurement database comparison. This article aims to evaluate and compare normative databases of different OCT machines, analyzing patient demographic, contrast inclusion and exclusion criteria, diversity index, and statistical approach to assess their fairness and generalizability. Methods Data were retrieved from Cirrus, Avanti, Spectralis, and Triton’s FDA-approval and equipment manual. The following variables were compared: number of eyes and patients, inclusion and exclusion criteria, statistical approach, sex, race and ethnicity, age, participant country, and diversity index. Results Avanti OCT has the largest normative database (640 eyes). In every database, the inclusion and exclusion criteria were similar, including adult patients and excluding pathological eyes. Spectralis has the largest White (79.7%) proportionately representation, Cirrus has the largest Asian (24%), and Triton has the largest Black (22%) patient representation. In all databases, the statistical analysis applied was Regression models. The sex diversity index is similar in all datasets, and comparable to the ten most populous contries. Avanti dataset has the highest diversity index in terms of race, followed by Cirrus, Triton, and Spectralis. Conclusion In all analyzed databases, the data framework is static, with limited upgrade options and lacking normative databases for new modules. As a result, caution in OCT normality interpretation is warranted. To address these limitations, there is a need for more diverse, representative, and open-access datasets that take into account patient demographics, especially considering the development of supervised Machine Learning algorithms in healthcare. |
| first_indexed | 2024-03-10T17:16:44Z |
| format | Article |
| id | doaj.art-ea8702d9f4b649b285623e9d9e8f9a41 |
| institution | Directory Open Access Journal |
| issn | 2056-9920 |
| language | English |
| last_indexed | 2024-03-10T17:16:44Z |
| publishDate | 2023-08-01 |
| publisher | BMC |
| record_format | Article |
| series | International Journal of Retina and Vitreous |
| spelling | doaj.art-ea8702d9f4b649b285623e9d9e8f9a412023-11-20T10:28:43ZengBMCInternational Journal of Retina and Vitreous2056-99202023-08-01911710.1186/s40942-023-00459-8Fairness and generalizability of OCT normative databases: a comparative analysisLuis Filipe Nakayama0Lucas Zago Ribeiro1Juliana Angelica Estevão de Oliveira2João Carlos Ramos Gonçalves de Matos3William Greig Mitchell4Fernando Korn Malerbi5Leo Anthony Celi6Caio Vinicius Saito Regatieri7Laboratory of Computational Physiology, Massachusetts Institute of TechnologyDepartment of Ophthalmology, São Paulo Federal UniversityDepartment of Ophthalmology, São Paulo Federal UniversityLaboratory of Computational Physiology, Massachusetts Institute of TechnologyDepartment of Ophthalmology, Royal Victorian Eye and Ear HospitalDepartment of Ophthalmology, São Paulo Federal UniversityLaboratory of Computational Physiology, Massachusetts Institute of TechnologyDepartment of Ophthalmology, São Paulo Federal UniversityAbstract Purpose In supervised Machine Learning algorithms, labels and reports are important in model development. To provide a normality assessment, the OCT has an in-built normative database that provides a color base scale from the measurement database comparison. This article aims to evaluate and compare normative databases of different OCT machines, analyzing patient demographic, contrast inclusion and exclusion criteria, diversity index, and statistical approach to assess their fairness and generalizability. Methods Data were retrieved from Cirrus, Avanti, Spectralis, and Triton’s FDA-approval and equipment manual. The following variables were compared: number of eyes and patients, inclusion and exclusion criteria, statistical approach, sex, race and ethnicity, age, participant country, and diversity index. Results Avanti OCT has the largest normative database (640 eyes). In every database, the inclusion and exclusion criteria were similar, including adult patients and excluding pathological eyes. Spectralis has the largest White (79.7%) proportionately representation, Cirrus has the largest Asian (24%), and Triton has the largest Black (22%) patient representation. In all databases, the statistical analysis applied was Regression models. The sex diversity index is similar in all datasets, and comparable to the ten most populous contries. Avanti dataset has the highest diversity index in terms of race, followed by Cirrus, Triton, and Spectralis. Conclusion In all analyzed databases, the data framework is static, with limited upgrade options and lacking normative databases for new modules. As a result, caution in OCT normality interpretation is warranted. To address these limitations, there is a need for more diverse, representative, and open-access datasets that take into account patient demographics, especially considering the development of supervised Machine Learning algorithms in healthcare.https://doi.org/10.1186/s40942-023-00459-8Optical coherence tomographyFairnessGeneralizabilityDatabaseSupervised machine learning |
| spellingShingle | Luis Filipe Nakayama Lucas Zago Ribeiro Juliana Angelica Estevão de Oliveira João Carlos Ramos Gonçalves de Matos William Greig Mitchell Fernando Korn Malerbi Leo Anthony Celi Caio Vinicius Saito Regatieri Fairness and generalizability of OCT normative databases: a comparative analysis International Journal of Retina and Vitreous Optical coherence tomography Fairness Generalizability Database Supervised machine learning |
| title | Fairness and generalizability of OCT normative databases: a comparative analysis |
| title_full | Fairness and generalizability of OCT normative databases: a comparative analysis |
| title_fullStr | Fairness and generalizability of OCT normative databases: a comparative analysis |
| title_full_unstemmed | Fairness and generalizability of OCT normative databases: a comparative analysis |
| title_short | Fairness and generalizability of OCT normative databases: a comparative analysis |
| title_sort | fairness and generalizability of oct normative databases a comparative analysis |
| topic | Optical coherence tomography Fairness Generalizability Database Supervised machine learning |
| url | https://doi.org/10.1186/s40942-023-00459-8 |
| work_keys_str_mv | AT luisfilipenakayama fairnessandgeneralizabilityofoctnormativedatabasesacomparativeanalysis AT lucaszagoribeiro fairnessandgeneralizabilityofoctnormativedatabasesacomparativeanalysis AT julianaangelicaestevaodeoliveira fairnessandgeneralizabilityofoctnormativedatabasesacomparativeanalysis AT joaocarlosramosgoncalvesdematos fairnessandgeneralizabilityofoctnormativedatabasesacomparativeanalysis AT williamgreigmitchell fairnessandgeneralizabilityofoctnormativedatabasesacomparativeanalysis AT fernandokornmalerbi fairnessandgeneralizabilityofoctnormativedatabasesacomparativeanalysis AT leoanthonyceli fairnessandgeneralizabilityofoctnormativedatabasesacomparativeanalysis AT caioviniciussaitoregatieri fairnessandgeneralizabilityofoctnormativedatabasesacomparativeanalysis |