Adherence of randomised controlled trials using artificial intelligence in ophthalmology to CONSORT-AI guidelines: a systematic review and critical appraisal
Purpose Many efforts have been made to explore the potential of deep learning and artificial intelligence (AI) in disciplines such as medicine, including ophthalmology. This systematic review aims to evaluate the reporting quality of randomised controlled trials (RCTs) that evaluate AI technologies...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2023-12-01
|
| Series: | BMJ Health & Care Informatics |
| Online Access: | https://informatics.bmj.com/content/30/1/e100757.full |
| _version_ | 1797364558761820160 |
|---|---|
| author | Tina Felfeli Niveditha Pattathil Jonathan Z L Zhao Olapeju Sam-Oyerinde |
| author_facet | Tina Felfeli Niveditha Pattathil Jonathan Z L Zhao Olapeju Sam-Oyerinde |
| author_sort | Tina Felfeli |
| collection | DOAJ |
| description | Purpose Many efforts have been made to explore the potential of deep learning and artificial intelligence (AI) in disciplines such as medicine, including ophthalmology. This systematic review aims to evaluate the reporting quality of randomised controlled trials (RCTs) that evaluate AI technologies applied to ophthalmology.Methods A comprehensive search of three relevant databases (EMBASE, Medline, Cochrane) from 1 January 2010 to 5 February 2022 was conducted. The reporting quality of these papers was scored using the Consolidated Standards of Reporting Trials-Artificial Intelligence (CONSORT-AI) checklist and further risk of bias was assessed using the RoB-2 tool.Results The initial search yielded 2973 citations from which 5 articles satisfied the inclusion/exclusion criteria. These articles featured AI technologies applied to diabetic retinopathy screening, ophthalmologic education, fungal keratitis detection and paediatric cataract diagnosis. None of the articles reported all items in the CONSORT-AI checklist. The overall mean CONSORT-AI score of the included RCTs was 53% (range 37%–78%). The individual scores of the articles were 37% (19/51), 39% (20), 49% (25), 61% (31) and 78% (40). All articles were scored as being moderate risk, or ‘some concerns present’, regarding potential risk of bias according to the RoB-2 tool.Conclusion A small number of RCTs have been published to date on the applications of AI in ophthalmology and vision science. Adherence to the 2020 CONSORT-AI reporting guidelines is suboptimal with notable reporting items often missed. Greater adherence will help facilitate reproducibility of AI research which can be a stimulus for more AI-based RCTs and clinical applications in ophthalmology. |
| first_indexed | 2024-03-08T16:38:00Z |
| format | Article |
| id | doaj.art-2807364917c64ae9ae00ee4fe5e6ab8f |
| institution | Directory Open Access Journal |
| issn | 2632-1009 |
| language | English |
| last_indexed | 2024-03-08T16:38:00Z |
| publishDate | 2023-12-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Health & Care Informatics |
| spelling | doaj.art-2807364917c64ae9ae00ee4fe5e6ab8f2024-01-05T12:10:09ZengBMJ Publishing GroupBMJ Health & Care Informatics2632-10092023-12-0130110.1136/bmjhci-2023-100757Adherence of randomised controlled trials using artificial intelligence in ophthalmology to CONSORT-AI guidelines: a systematic review and critical appraisalTina Felfeli0Niveditha Pattathil1Jonathan Z L Zhao2Olapeju Sam-Oyerinde3Department of Ophthalmology and Vision Sciences, University of Toronto, Toronto, Ontario, CanadaQueen`s University School of Medicine, Faculty of Health Sciences, Kingston, Ontario, CanadaTemerty Faculty of Medicine, University of Toronto, Toronto, Ontario, CanadaDepartment of Ophthalmology, University College London, London, UKPurpose Many efforts have been made to explore the potential of deep learning and artificial intelligence (AI) in disciplines such as medicine, including ophthalmology. This systematic review aims to evaluate the reporting quality of randomised controlled trials (RCTs) that evaluate AI technologies applied to ophthalmology.Methods A comprehensive search of three relevant databases (EMBASE, Medline, Cochrane) from 1 January 2010 to 5 February 2022 was conducted. The reporting quality of these papers was scored using the Consolidated Standards of Reporting Trials-Artificial Intelligence (CONSORT-AI) checklist and further risk of bias was assessed using the RoB-2 tool.Results The initial search yielded 2973 citations from which 5 articles satisfied the inclusion/exclusion criteria. These articles featured AI technologies applied to diabetic retinopathy screening, ophthalmologic education, fungal keratitis detection and paediatric cataract diagnosis. None of the articles reported all items in the CONSORT-AI checklist. The overall mean CONSORT-AI score of the included RCTs was 53% (range 37%–78%). The individual scores of the articles were 37% (19/51), 39% (20), 49% (25), 61% (31) and 78% (40). All articles were scored as being moderate risk, or ‘some concerns present’, regarding potential risk of bias according to the RoB-2 tool.Conclusion A small number of RCTs have been published to date on the applications of AI in ophthalmology and vision science. Adherence to the 2020 CONSORT-AI reporting guidelines is suboptimal with notable reporting items often missed. Greater adherence will help facilitate reproducibility of AI research which can be a stimulus for more AI-based RCTs and clinical applications in ophthalmology.https://informatics.bmj.com/content/30/1/e100757.full |
| spellingShingle | Tina Felfeli Niveditha Pattathil Jonathan Z L Zhao Olapeju Sam-Oyerinde Adherence of randomised controlled trials using artificial intelligence in ophthalmology to CONSORT-AI guidelines: a systematic review and critical appraisal BMJ Health & Care Informatics |
| title | Adherence of randomised controlled trials using artificial intelligence in ophthalmology to CONSORT-AI guidelines: a systematic review and critical appraisal |
| title_full | Adherence of randomised controlled trials using artificial intelligence in ophthalmology to CONSORT-AI guidelines: a systematic review and critical appraisal |
| title_fullStr | Adherence of randomised controlled trials using artificial intelligence in ophthalmology to CONSORT-AI guidelines: a systematic review and critical appraisal |
| title_full_unstemmed | Adherence of randomised controlled trials using artificial intelligence in ophthalmology to CONSORT-AI guidelines: a systematic review and critical appraisal |
| title_short | Adherence of randomised controlled trials using artificial intelligence in ophthalmology to CONSORT-AI guidelines: a systematic review and critical appraisal |
| title_sort | adherence of randomised controlled trials using artificial intelligence in ophthalmology to consort ai guidelines a systematic review and critical appraisal |
| url | https://informatics.bmj.com/content/30/1/e100757.full |
| work_keys_str_mv | AT tinafelfeli adherenceofrandomisedcontrolledtrialsusingartificialintelligenceinophthalmologytoconsortaiguidelinesasystematicreviewandcriticalappraisal AT nivedithapattathil adherenceofrandomisedcontrolledtrialsusingartificialintelligenceinophthalmologytoconsortaiguidelinesasystematicreviewandcriticalappraisal AT jonathanzlzhao adherenceofrandomisedcontrolledtrialsusingartificialintelligenceinophthalmologytoconsortaiguidelinesasystematicreviewandcriticalappraisal AT olapejusamoyerinde adherenceofrandomisedcontrolledtrialsusingartificialintelligenceinophthalmologytoconsortaiguidelinesasystematicreviewandcriticalappraisal |