A comment-driven evidence appraisal approach to promoting research findings into practice when only uncertain evidence is available
Abstract Background Comments in PubMed are usually short papers for supporting or refuting claims, or discussing methods and findings in original articles. This study aims to explore whether they can be used as a quick and reliable evidence appraisal instrument for promoting research findings into p...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-03-01
|
| Series: | Health Research Policy and Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12961-023-00969-9 |
| _version_ | 1797853552236822528 |
|---|---|
| author | Shuang Wang Halil Kilicoglu Jian Du |
| author_facet | Shuang Wang Halil Kilicoglu Jian Du |
| author_sort | Shuang Wang |
| collection | DOAJ |
| description | Abstract Background Comments in PubMed are usually short papers for supporting or refuting claims, or discussing methods and findings in original articles. This study aims to explore whether they can be used as a quick and reliable evidence appraisal instrument for promoting research findings into practice, especially in emergency situations such as COVID-19 in which only missing, incomplete or uncertain evidence is available. Methods Evidence–comment networks (ECNs) were constructed by linking COVID-19-related articles to the commentaries (letters, editorials or brief correspondence) they received. PubTator Central was used to extract entities with a high volume of comments from the titles and abstracts of the articles. Among them, six drugs were selected, and their evidence assertions were analysed by exploring the structural information in the ECNs as well as the sentiment of the comments (positive, negative, neutral). Recommendations in WHO guidelines were used as the gold standard control to validate the consistency, coverage and efficiency of comments in reshaping clinical knowledge claims. Results The overall positive/negative sentiments of comments were aligned with recommendations for/against the corresponding treatments in the WHO guidelines. Comment topics covered all significant points of evidence appraisal and beyond. Furthermore, comments may indicate the uncertainty regarding drug use for clinical practice. Half of the critical comments emerged 4.25 months earlier on average than the guideline release. Conclusions Comments have the potential as a support tool for rapid evidence appraisal as they have a selection effect by appraising the benefits, limitations and other clinical practice issues of concern in existing evidence. We suggest as a future direction an appraisal framework based on the comment topics and sentiment orientations to leverage the potential of scientific commentaries supporting evidence appraisal and decision-making. |
| first_indexed | 2024-04-09T19:51:22Z |
| format | Article |
| id | doaj.art-dd088c37636a45c5862d339a3e2cec59 |
| institution | Directory Open Access Journal |
| issn | 1478-4505 |
| language | English |
| last_indexed | 2024-04-09T19:51:22Z |
| publishDate | 2023-03-01 |
| publisher | BMC |
| record_format | Article |
| series | Health Research Policy and Systems |
| spelling | doaj.art-dd088c37636a45c5862d339a3e2cec592023-04-03T05:44:48ZengBMCHealth Research Policy and Systems1478-45052023-03-0121111310.1186/s12961-023-00969-9A comment-driven evidence appraisal approach to promoting research findings into practice when only uncertain evidence is availableShuang Wang0Halil Kilicoglu1Jian Du2National Institute of Health Data Science, Peking UniversitySchool of Information Sciences, University of Illinois at Urbana-ChampaignNational Institute of Health Data Science, Peking UniversityAbstract Background Comments in PubMed are usually short papers for supporting or refuting claims, or discussing methods and findings in original articles. This study aims to explore whether they can be used as a quick and reliable evidence appraisal instrument for promoting research findings into practice, especially in emergency situations such as COVID-19 in which only missing, incomplete or uncertain evidence is available. Methods Evidence–comment networks (ECNs) were constructed by linking COVID-19-related articles to the commentaries (letters, editorials or brief correspondence) they received. PubTator Central was used to extract entities with a high volume of comments from the titles and abstracts of the articles. Among them, six drugs were selected, and their evidence assertions were analysed by exploring the structural information in the ECNs as well as the sentiment of the comments (positive, negative, neutral). Recommendations in WHO guidelines were used as the gold standard control to validate the consistency, coverage and efficiency of comments in reshaping clinical knowledge claims. Results The overall positive/negative sentiments of comments were aligned with recommendations for/against the corresponding treatments in the WHO guidelines. Comment topics covered all significant points of evidence appraisal and beyond. Furthermore, comments may indicate the uncertainty regarding drug use for clinical practice. Half of the critical comments emerged 4.25 months earlier on average than the guideline release. Conclusions Comments have the potential as a support tool for rapid evidence appraisal as they have a selection effect by appraising the benefits, limitations and other clinical practice issues of concern in existing evidence. We suggest as a future direction an appraisal framework based on the comment topics and sentiment orientations to leverage the potential of scientific commentaries supporting evidence appraisal and decision-making.https://doi.org/10.1186/s12961-023-00969-9Evidence–comment networkScientific commentaryEvidence appraisalSentiment analysisEvidence-based policy-making |
| spellingShingle | Shuang Wang Halil Kilicoglu Jian Du A comment-driven evidence appraisal approach to promoting research findings into practice when only uncertain evidence is available Health Research Policy and Systems Evidence–comment network Scientific commentary Evidence appraisal Sentiment analysis Evidence-based policy-making |
| title | A comment-driven evidence appraisal approach to promoting research findings into practice when only uncertain evidence is available |
| title_full | A comment-driven evidence appraisal approach to promoting research findings into practice when only uncertain evidence is available |
| title_fullStr | A comment-driven evidence appraisal approach to promoting research findings into practice when only uncertain evidence is available |
| title_full_unstemmed | A comment-driven evidence appraisal approach to promoting research findings into practice when only uncertain evidence is available |
| title_short | A comment-driven evidence appraisal approach to promoting research findings into practice when only uncertain evidence is available |
| title_sort | comment driven evidence appraisal approach to promoting research findings into practice when only uncertain evidence is available |
| topic | Evidence–comment network Scientific commentary Evidence appraisal Sentiment analysis Evidence-based policy-making |
| url | https://doi.org/10.1186/s12961-023-00969-9 |
| work_keys_str_mv | AT shuangwang acommentdrivenevidenceappraisalapproachtopromotingresearchfindingsintopracticewhenonlyuncertainevidenceisavailable AT halilkilicoglu acommentdrivenevidenceappraisalapproachtopromotingresearchfindingsintopracticewhenonlyuncertainevidenceisavailable AT jiandu acommentdrivenevidenceappraisalapproachtopromotingresearchfindingsintopracticewhenonlyuncertainevidenceisavailable AT shuangwang commentdrivenevidenceappraisalapproachtopromotingresearchfindingsintopracticewhenonlyuncertainevidenceisavailable AT halilkilicoglu commentdrivenevidenceappraisalapproachtopromotingresearchfindingsintopracticewhenonlyuncertainevidenceisavailable AT jiandu commentdrivenevidenceappraisalapproachtopromotingresearchfindingsintopracticewhenonlyuncertainevidenceisavailable |