Towards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionFDA
Abstract Accurately calling indels with next-generation sequencing (NGS) data is critical for clinical application. The precisionFDA team collaborated with the U.S. Food and Drug Administration’s (FDA’s) National Center for Toxicological Research (NCTR) and successfully completed the NCTR Indel Call...
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-04-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-024-58573-y |
| _version_ | 1797209383702102016 |
|---|---|
| author | Binsheng Gong Samir Lababidi Rebecca Kusko Khaled Bouri Sarah Prezek Vishal Thovarai Anish Prasanna Ezekiel J. Maier Mahdi Golkaram Xingqiang Sun Konstantinos Kyriakidis João Paulo Kitajima Sayed Mohammad Ebrahim Sahraeian Yunfei Guo Elaine Johanson Wendell Jones Weida Tong Joshua Xu |
| author_facet | Binsheng Gong Samir Lababidi Rebecca Kusko Khaled Bouri Sarah Prezek Vishal Thovarai Anish Prasanna Ezekiel J. Maier Mahdi Golkaram Xingqiang Sun Konstantinos Kyriakidis João Paulo Kitajima Sayed Mohammad Ebrahim Sahraeian Yunfei Guo Elaine Johanson Wendell Jones Weida Tong Joshua Xu |
| author_sort | Binsheng Gong |
| collection | DOAJ |
| description | Abstract Accurately calling indels with next-generation sequencing (NGS) data is critical for clinical application. The precisionFDA team collaborated with the U.S. Food and Drug Administration’s (FDA’s) National Center for Toxicological Research (NCTR) and successfully completed the NCTR Indel Calling from Oncopanel Sequencing Data Challenge, to evaluate the performance of indel calling pipelines. Top performers were selected based on precision, recall, and F1-score. The performance of many other pipelines was close to the top performers, which produced a top cluster of performers. The performance was significantly higher in high confidence regions and coding regions, and significantly lower in low complexity regions. Oncopanel capture and other issues may have occurred that affected the recall rate. Indels with higher variant allele frequency (VAF) may generally be called with higher confidence. Many of the indel calling pipelines had good performance. Some of them performed generally well across all three oncopanels, while others were better for a specific oncopanel. The performance of indel calling can further be improved by restricting the calls within high confidence intervals (HCIs) and coding regions, and by excluding low complexity regions (LCR) regions. Certain VAF cut-offs could be applied according to the applications. |
| first_indexed | 2024-04-24T09:53:50Z |
| format | Article |
| id | doaj.art-fa59c4fa44684290a436974c57f268f0 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-24T09:53:50Z |
| publishDate | 2024-04-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-fa59c4fa44684290a436974c57f268f02024-04-14T11:16:50ZengNature PortfolioScientific Reports2045-23222024-04-0114111210.1038/s41598-024-58573-yTowards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionFDABinsheng Gong0Samir Lababidi1Rebecca Kusko2Khaled Bouri3Sarah Prezek4Vishal Thovarai5Anish Prasanna6Ezekiel J. Maier7Mahdi Golkaram8Xingqiang Sun9Konstantinos Kyriakidis10João Paulo Kitajima11Sayed Mohammad Ebrahim Sahraeian12Yunfei Guo13Elaine Johanson14Wendell Jones15Weida Tong16Joshua Xu17Division of Bioinformatics and Biostatistics, Office of Research, National Center for Toxicological Research, Office of the Chief Scientist, Office of the Commissioner, U.S. Food and Drug AdministrationHealth Informatics Staff, Office of Data, Analytics, and Research, Office of Digital Transformation, Office of the Commissioner, U.S. Food and Drug AdministrationCellino BiotechOffice of Regulatory Science and Innovation, Office of the Chief Scientist, Office of the Commissioner, U.S. Food and Drug AdministrationBooz Allen HamiltonBooz Allen HamiltonBooz Allen HamiltonBooz Allen HamiltonIllumina, Inc.GenetalksUC Santa Cruz Genomics InstituteMendelicsRoche Sequencing SolutionsRoche Sequencing SolutionsHealth Informatics Staff, Office of Data, Analytics, and Research, Office of Digital Transformation, Office of the Commissioner, U.S. Food and Drug AdministrationQ squared Solutions GenomicsDivision of Bioinformatics and Biostatistics, Office of Research, National Center for Toxicological Research, Office of the Chief Scientist, Office of the Commissioner, U.S. Food and Drug AdministrationDivision of Bioinformatics and Biostatistics, Office of Research, National Center for Toxicological Research, Office of the Chief Scientist, Office of the Commissioner, U.S. Food and Drug AdministrationAbstract Accurately calling indels with next-generation sequencing (NGS) data is critical for clinical application. The precisionFDA team collaborated with the U.S. Food and Drug Administration’s (FDA’s) National Center for Toxicological Research (NCTR) and successfully completed the NCTR Indel Calling from Oncopanel Sequencing Data Challenge, to evaluate the performance of indel calling pipelines. Top performers were selected based on precision, recall, and F1-score. The performance of many other pipelines was close to the top performers, which produced a top cluster of performers. The performance was significantly higher in high confidence regions and coding regions, and significantly lower in low complexity regions. Oncopanel capture and other issues may have occurred that affected the recall rate. Indels with higher variant allele frequency (VAF) may generally be called with higher confidence. Many of the indel calling pipelines had good performance. Some of them performed generally well across all three oncopanels, while others were better for a specific oncopanel. The performance of indel calling can further be improved by restricting the calls within high confidence intervals (HCIs) and coding regions, and by excluding low complexity regions (LCR) regions. Certain VAF cut-offs could be applied according to the applications.https://doi.org/10.1038/s41598-024-58573-yIndel calling pipelineNext-generation sequencingPrecision medicinePersonalized medicineprecisionFDA challenge |
| spellingShingle | Binsheng Gong Samir Lababidi Rebecca Kusko Khaled Bouri Sarah Prezek Vishal Thovarai Anish Prasanna Ezekiel J. Maier Mahdi Golkaram Xingqiang Sun Konstantinos Kyriakidis João Paulo Kitajima Sayed Mohammad Ebrahim Sahraeian Yunfei Guo Elaine Johanson Wendell Jones Weida Tong Joshua Xu Towards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionFDA Scientific Reports Indel calling pipeline Next-generation sequencing Precision medicine Personalized medicine precisionFDA challenge |
| title | Towards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionFDA |
| title_full | Towards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionFDA |
| title_fullStr | Towards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionFDA |
| title_full_unstemmed | Towards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionFDA |
| title_short | Towards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionFDA |
| title_sort | towards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionfda |
| topic | Indel calling pipeline Next-generation sequencing Precision medicine Personalized medicine precisionFDA challenge |
| url | https://doi.org/10.1038/s41598-024-58573-y |
| work_keys_str_mv | AT binshenggong towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT samirlababidi towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT rebeccakusko towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT khaledbouri towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT sarahprezek towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT vishalthovarai towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT anishprasanna towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT ezekieljmaier towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT mahdigolkaram towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT xingqiangsun towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT konstantinoskyriakidis towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT joaopaulokitajima towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT sayedmohammadebrahimsahraeian towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT yunfeiguo towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT elainejohanson towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT wendelljones towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT weidatong towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda AT joshuaxu towardsaccurateindelcallingforoncopanelsequencingthroughaninternationalpipelinecompetitionatprecisionfda |