Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations
<p>The coronavirus disease 2019 (COVID-19) pandemic has been severely impacting global society since December 2019. The related findings such as vaccine and drug development have been reported in biomedical literature—at a rate of about 10 000 articles on COVID-19 per month. Such rapid growth...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Oxford University Press
2022
|
| _version_ | 1797108153676988416 |
|---|---|
| author | Chen, Q Allot, A Leaman, R Islamaj, R Du, J Fang, L Wang, K Xu, S Zhang, Y Bagherzadeh, P Bergler, S Bhatnagar, A Bhavsar, N Chang, Y-C Lin, S-J Tang, W Zhang, H Tavchioski, I Pollak, S Tian, S Zhang, J Otmakhova, Y Yepes, AJ Dong, H Wu, H Dufour, R Labrak, Y Chatterjee, N Tandon, K Laleye, FAA Rakotoson, L Chersoni, E Gu, J Friedrich, A Pujari, SC Chizhikova, M Sivadasan, N Vg, S Lu, Z |
| author_facet | Chen, Q Allot, A Leaman, R Islamaj, R Du, J Fang, L Wang, K Xu, S Zhang, Y Bagherzadeh, P Bergler, S Bhatnagar, A Bhavsar, N Chang, Y-C Lin, S-J Tang, W Zhang, H Tavchioski, I Pollak, S Tian, S Zhang, J Otmakhova, Y Yepes, AJ Dong, H Wu, H Dufour, R Labrak, Y Chatterjee, N Tandon, K Laleye, FAA Rakotoson, L Chersoni, E Gu, J Friedrich, A Pujari, SC Chizhikova, M Sivadasan, N Vg, S Lu, Z |
| author_sort | Chen, Q |
| collection | OXFORD |
| description | <p>The coronavirus disease 2019 (COVID-19) pandemic has been severely impacting global society since December 2019. The related findings such as vaccine and drug development have been reported in biomedical literature—at a rate of about 10 000 articles on COVID-19 per month. Such rapid growth significantly challenges manual curation and interpretation. For instance, LitCovid is a literature database of COVID-19-related articles in PubMed, which has accumulated more than 200 000 articles with millions of accesses each month by users worldwide. One primary curation task is to assign up to eight topics (e.g. Diagnosis and Treatment) to the articles in LitCovid. The annotated topics have been widely used for navigating the COVID literature, rapidly locating articles of interest and other downstream studies. However, annotating the topics has been the bottleneck of manual curation. Despite the continuing advances in biomedical text-mining methods, few have been dedicated to topic annotations in COVID-19 literature. To close the gap, we organized the BioCreative LitCovid track to call for a community effort to tackle automated topic annotation for COVID-19 literature. The BioCreative LitCovid dataset—consisting of over 30 000 articles with manually reviewed topics—was created for training and testing. It is one of the largest multi-label classification datasets in biomedical scientific literature. Nineteen teams worldwide participated and made 80 submissions in total. Most teams used hybrid systems based on transformers. The highest performing submissions achieved 0.8875, 0.9181 and 0.9394 for macro-F1-score, micro-F1-score and instance-based F1-score, respectively. Notably, these scores are substantially higher (e.g. 12%, higher for macro F1-score) than the corresponding scores of the state-of-art multi-label classification method. The level of participation and results demonstrate a successful track and help close the gap between dataset curation and method development. The dataset is publicly available via https://ftp.ncbi.nlm.nih.gov/pub/lu/LitCovid/biocreative/ for benchmarking and further development.</p>
<p>Database URLhttps://ftp.ncbi.nlm.nih.gov/pub/lu/LitCovid/biocreative/</p> |
| first_indexed | 2024-03-07T07:25:25Z |
| format | Journal article |
| id | oxford-uuid:2d7a2036-5b3b-4a94-a5e6-fcce75cfcd9b |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T07:25:25Z |
| publishDate | 2022 |
| publisher | Oxford University Press |
| record_format | dspace |
| spelling | oxford-uuid:2d7a2036-5b3b-4a94-a5e6-fcce75cfcd9b2022-11-15T16:04:05ZMulti-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations Journal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:2d7a2036-5b3b-4a94-a5e6-fcce75cfcd9bEnglishSymplectic ElementsOxford University Press2022Chen, QAllot, ALeaman, RIslamaj, RDu, JFang, LWang, KXu, SZhang, YBagherzadeh, PBergler, SBhatnagar, ABhavsar, NChang, Y-CLin, S-JTang, WZhang, HTavchioski, IPollak, STian, SZhang, JOtmakhova, YYepes, AJDong, HWu, HDufour, RLabrak, YChatterjee, NTandon, KLaleye, FAARakotoson, LChersoni, EGu, JFriedrich, APujari, SCChizhikova, MSivadasan, NVg, SLu, Z<p>The coronavirus disease 2019 (COVID-19) pandemic has been severely impacting global society since December 2019. The related findings such as vaccine and drug development have been reported in biomedical literature—at a rate of about 10 000 articles on COVID-19 per month. Such rapid growth significantly challenges manual curation and interpretation. For instance, LitCovid is a literature database of COVID-19-related articles in PubMed, which has accumulated more than 200 000 articles with millions of accesses each month by users worldwide. One primary curation task is to assign up to eight topics (e.g. Diagnosis and Treatment) to the articles in LitCovid. The annotated topics have been widely used for navigating the COVID literature, rapidly locating articles of interest and other downstream studies. However, annotating the topics has been the bottleneck of manual curation. Despite the continuing advances in biomedical text-mining methods, few have been dedicated to topic annotations in COVID-19 literature. To close the gap, we organized the BioCreative LitCovid track to call for a community effort to tackle automated topic annotation for COVID-19 literature. The BioCreative LitCovid dataset—consisting of over 30 000 articles with manually reviewed topics—was created for training and testing. It is one of the largest multi-label classification datasets in biomedical scientific literature. Nineteen teams worldwide participated and made 80 submissions in total. Most teams used hybrid systems based on transformers. The highest performing submissions achieved 0.8875, 0.9181 and 0.9394 for macro-F1-score, micro-F1-score and instance-based F1-score, respectively. Notably, these scores are substantially higher (e.g. 12%, higher for macro F1-score) than the corresponding scores of the state-of-art multi-label classification method. The level of participation and results demonstrate a successful track and help close the gap between dataset curation and method development. The dataset is publicly available via https://ftp.ncbi.nlm.nih.gov/pub/lu/LitCovid/biocreative/ for benchmarking and further development.</p> <p>Database URLhttps://ftp.ncbi.nlm.nih.gov/pub/lu/LitCovid/biocreative/</p> |
| spellingShingle | Chen, Q Allot, A Leaman, R Islamaj, R Du, J Fang, L Wang, K Xu, S Zhang, Y Bagherzadeh, P Bergler, S Bhatnagar, A Bhavsar, N Chang, Y-C Lin, S-J Tang, W Zhang, H Tavchioski, I Pollak, S Tian, S Zhang, J Otmakhova, Y Yepes, AJ Dong, H Wu, H Dufour, R Labrak, Y Chatterjee, N Tandon, K Laleye, FAA Rakotoson, L Chersoni, E Gu, J Friedrich, A Pujari, SC Chizhikova, M Sivadasan, N Vg, S Lu, Z Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations |
| title | Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations |
| title_full | Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations |
| title_fullStr | Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations |
| title_full_unstemmed | Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations |
| title_short | Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations |
| title_sort | multi label classification for biomedical literature an overview of the biocreative vii litcovid track for covid 19 literature topic annotations |
| work_keys_str_mv | AT chenq multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT allota multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT leamanr multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT islamajr multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT duj multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT fangl multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT wangk multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT xus multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT zhangy multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT bagherzadehp multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT berglers multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT bhatnagara multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT bhavsarn multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT changyc multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT linsj multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT tangw multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT zhangh multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT tavchioskii multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT pollaks multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT tians multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT zhangj multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT otmakhovay multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT yepesaj multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT dongh multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT wuh multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT dufourr multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT labraky multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT chatterjeen multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT tandonk multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT laleyefaa multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT rakotosonl multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT chersonie multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT guj multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT friedricha multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT pujarisc multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT chizhikovam multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT sivadasann multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT vgs multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations AT luz multilabelclassificationforbiomedicalliteratureanoverviewofthebiocreativeviilitcovidtrackforcovid19literaturetopicannotations |