The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations
Abstract Background Clinical trial registries can be used as sources of clinical evidence for systematic review synthesis and updating. Our aim was to evaluate methods for identifying clinical trial registrations that should be screened for inclusion in updates of published systematic reviews. Metho...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2021-12-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12874-021-01485-6 |
| _version_ | 1818360977297506304 |
|---|---|
| author | Didi Surian Florence T. Bourgeois Adam G. Dunn |
| author_facet | Didi Surian Florence T. Bourgeois Adam G. Dunn |
| author_sort | Didi Surian |
| collection | DOAJ |
| description | Abstract Background Clinical trial registries can be used as sources of clinical evidence for systematic review synthesis and updating. Our aim was to evaluate methods for identifying clinical trial registrations that should be screened for inclusion in updates of published systematic reviews. Methods A set of 4644 clinical trial registrations (ClinicalTrials.gov) included in 1089 systematic reviews (PubMed) were used to evaluate two methods (document similarity and hierarchical clustering) and representations (L2-normalised TF-IDF, Latent Dirichlet Allocation, and Doc2Vec) for ranking 163,501 completed clinical trials by relevance. Clinical trial registrations were ranked for each systematic review using seeding clinical trials, simulating how new relevant clinical trials could be automatically identified for an update. Performance was measured by the number of clinical trials that need to be screened to identify all relevant clinical trials. Results Using the document similarity method with TF-IDF feature representation and Euclidean distance metric, all relevant clinical trials for half of the systematic reviews were identified after screening 99 trials (IQR 19 to 491). The best-performing hierarchical clustering was using Ward agglomerative clustering (with TF-IDF representation and Euclidean distance) and needed to screen 501 clinical trials (IQR 43 to 4363) to achieve the same result. Conclusion An evaluation using a large set of mined links between published systematic reviews and clinical trial registrations showed that document similarity outperformed hierarchical clustering for identifying relevant clinical trials to include in systematic review updates. |
| first_indexed | 2024-12-13T21:09:21Z |
| format | Article |
| id | doaj.art-e5934a60254445e2bbec7bced8062ca5 |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-12-13T21:09:21Z |
| publishDate | 2021-12-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-e5934a60254445e2bbec7bced8062ca52022-12-21T23:31:24ZengBMCBMC Medical Research Methodology1471-22882021-12-0121111010.1186/s12874-021-01485-6The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrationsDidi Surian0Florence T. Bourgeois1Adam G. Dunn2Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie UniversityComputational Health Informatics Program, Boston Children’s HospitalComputational Health Informatics Program, Boston Children’s HospitalAbstract Background Clinical trial registries can be used as sources of clinical evidence for systematic review synthesis and updating. Our aim was to evaluate methods for identifying clinical trial registrations that should be screened for inclusion in updates of published systematic reviews. Methods A set of 4644 clinical trial registrations (ClinicalTrials.gov) included in 1089 systematic reviews (PubMed) were used to evaluate two methods (document similarity and hierarchical clustering) and representations (L2-normalised TF-IDF, Latent Dirichlet Allocation, and Doc2Vec) for ranking 163,501 completed clinical trials by relevance. Clinical trial registrations were ranked for each systematic review using seeding clinical trials, simulating how new relevant clinical trials could be automatically identified for an update. Performance was measured by the number of clinical trials that need to be screened to identify all relevant clinical trials. Results Using the document similarity method with TF-IDF feature representation and Euclidean distance metric, all relevant clinical trials for half of the systematic reviews were identified after screening 99 trials (IQR 19 to 491). The best-performing hierarchical clustering was using Ward agglomerative clustering (with TF-IDF representation and Euclidean distance) and needed to screen 501 clinical trials (IQR 43 to 4363) to achieve the same result. Conclusion An evaluation using a large set of mined links between published systematic reviews and clinical trial registrations showed that document similarity outperformed hierarchical clustering for identifying relevant clinical trials to include in systematic review updates.https://doi.org/10.1186/s12874-021-01485-6Systematic reviewsTrial registrationsDocument similarityHierarchical clustering |
| spellingShingle | Didi Surian Florence T. Bourgeois Adam G. Dunn The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations BMC Medical Research Methodology Systematic reviews Trial registrations Document similarity Hierarchical clustering |
| title | The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations |
| title_full | The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations |
| title_fullStr | The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations |
| title_full_unstemmed | The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations |
| title_short | The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations |
| title_sort | automation of relevant trial registration screening for systematic review updates an evaluation study on a large dataset of clinicaltrials gov registrations |
| topic | Systematic reviews Trial registrations Document similarity Hierarchical clustering |
| url | https://doi.org/10.1186/s12874-021-01485-6 |
| work_keys_str_mv | AT didisurian theautomationofrelevanttrialregistrationscreeningforsystematicreviewupdatesanevaluationstudyonalargedatasetofclinicaltrialsgovregistrations AT florencetbourgeois theautomationofrelevanttrialregistrationscreeningforsystematicreviewupdatesanevaluationstudyonalargedatasetofclinicaltrialsgovregistrations AT adamgdunn theautomationofrelevanttrialregistrationscreeningforsystematicreviewupdatesanevaluationstudyonalargedatasetofclinicaltrialsgovregistrations AT didisurian automationofrelevanttrialregistrationscreeningforsystematicreviewupdatesanevaluationstudyonalargedatasetofclinicaltrialsgovregistrations AT florencetbourgeois automationofrelevanttrialregistrationscreeningforsystematicreviewupdatesanevaluationstudyonalargedatasetofclinicaltrialsgovregistrations AT adamgdunn automationofrelevanttrialregistrationscreeningforsystematicreviewupdatesanevaluationstudyonalargedatasetofclinicaltrialsgovregistrations |