A roadmap for the generation of benchmarking resources for antimicrobial resistance detection using next generation sequencing [version 2; peer review: 1 approved, 2 approved with reservations]
Next Generation Sequencing technologies significantly impact the field of Antimicrobial Resistance (AMR) detection and monitoring, with immediate uses in diagnosis and risk assessment. For this application and in general, considerable challenges remain in demonstrating sufficient trust to act upon t...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
F1000 Research Ltd
2022-03-01
|
| Series: | F1000Research |
| Subjects: | |
| Online Access: | https://f1000research.com/articles/10-80/v2 |
| _version_ | 1798040599998234624 |
|---|---|
| author | Alison E. Mather Alice Carolyn McHardy Ivo Gut Gemma L. Kay Kevin Vanneste Arthur Pightling John Rossen Thierry Naas Silke Peter Henrik Westh Alexandre Angers-Loustau Etienne Ruppé Robert Schlaberg Marco Fabbri Alessandro Cestaro Maddalena Querci Barbara Raffael Mauro Petrillo Paul Hammer Lukas M. Weber Jean-Yves Madec Valentina Paracchini Guy Van den Eede Erik Alm Derya Aytan-Aktug Dafni Maria Kagkli Teresa Coque Kok-Gan Chan Christoph Endrullat Salvador Capella-Gutierrez Catherine Carrillo |
| author_facet | Alison E. Mather Alice Carolyn McHardy Ivo Gut Gemma L. Kay Kevin Vanneste Arthur Pightling John Rossen Thierry Naas Silke Peter Henrik Westh Alexandre Angers-Loustau Etienne Ruppé Robert Schlaberg Marco Fabbri Alessandro Cestaro Maddalena Querci Barbara Raffael Mauro Petrillo Paul Hammer Lukas M. Weber Jean-Yves Madec Valentina Paracchini Guy Van den Eede Erik Alm Derya Aytan-Aktug Dafni Maria Kagkli Teresa Coque Kok-Gan Chan Christoph Endrullat Salvador Capella-Gutierrez Catherine Carrillo |
| author_sort | Alison E. Mather |
| collection | DOAJ |
| description | Next Generation Sequencing technologies significantly impact the field of Antimicrobial Resistance (AMR) detection and monitoring, with immediate uses in diagnosis and risk assessment. For this application and in general, considerable challenges remain in demonstrating sufficient trust to act upon the meaningful information produced from raw data, partly because of the reliance on bioinformatics pipelines, which can produce different results and therefore lead to different interpretations. With the constant evolution of the field, it is difficult to identify, harmonise and recommend specific methods for large-scale implementations over time. In this article, we propose to address this challenge through establishing a transparent, performance-based, evaluation approach to provide flexibility in the bioinformatics tools of choice, while demonstrating proficiency in meeting common performance standards. The approach is two-fold: first, a community-driven effort to establish and maintain “live” (dynamic) benchmarking platforms to provide relevant performance metrics, based on different use-cases, that would evolve together with the AMR field; second, agreed and defined datasets to allow the pipelines’ implementation, validation, and quality-control over time. Following previous discussions on the main challenges linked to this approach, we provide concrete recommendations and future steps, related to different aspects of the design of benchmarks, such as the selection and the characteristics of the datasets (quality, choice of pathogens and resistances, etc.), the evaluation criteria of the pipelines, and the way these resources should be deployed in the community. |
| first_indexed | 2024-04-11T22:09:54Z |
| format | Article |
| id | doaj.art-4a5a511e12b243b38c193c6ed9946a3f |
| institution | Directory Open Access Journal |
| issn | 2046-1402 |
| language | English |
| last_indexed | 2024-04-11T22:09:54Z |
| publishDate | 2022-03-01 |
| publisher | F1000 Research Ltd |
| record_format | Article |
| series | F1000Research |
| spelling | doaj.art-4a5a511e12b243b38c193c6ed9946a3f2022-12-22T04:00:36ZengF1000 Research LtdF1000Research2046-14022022-03-0110122296A roadmap for the generation of benchmarking resources for antimicrobial resistance detection using next generation sequencing [version 2; peer review: 1 approved, 2 approved with reservations]Alison E. Mather0Alice Carolyn McHardy1Ivo Gut2Gemma L. Kay3https://orcid.org/0000-0002-1176-8459Kevin Vanneste4Arthur Pightling5John Rossen6https://orcid.org/0000-0002-7167-8623Thierry Naas7Silke Peter8Henrik Westh9Alexandre Angers-Loustau10https://orcid.org/0000-0003-1425-4143Etienne Ruppé11Robert Schlaberg12Marco Fabbri13Alessandro Cestaro14Maddalena Querci15https://orcid.org/0000-0002-0788-1627Barbara Raffael16Mauro Petrillo17https://orcid.org/0000-0002-6782-4704Paul Hammer18Lukas M. Weber19https://orcid.org/0000-0002-3282-1730Jean-Yves Madec20Valentina Paracchini21Guy Van den Eede22Erik Alm23Derya Aytan-Aktug24Dafni Maria Kagkli25Teresa Coque26Kok-Gan Chan27https://orcid.org/0000-0002-1883-1115Christoph Endrullat28https://orcid.org/0000-0001-8646-6589Salvador Capella-Gutierrez29https://orcid.org/0000-0002-0309-604XCatherine Carrillo30https://orcid.org/0000-0002-2334-8718Quadram Institute Bioscience, Norwich Research Park, Norwich, UKHelmholtz Centre for Infection Research, Braunschweig, GermanyCentro Nacional de Análisis Genómico, Centre for Genomic Regulation (CNAG-CRG), Barcelona Institute of Technology, Barcelona, SpainQuadram Institute Bioscience, Norwich Research Park, Norwich, UKTransversal activities in Applied Genomics, Sciensano, Brussels, BelgiumCenter for Food Safety and Applied Nutrition, US Food and Drug Administration, College Park, MD, USADepartment of Medical Microbiology, University Medical Center Groningen, University of Groningen, Groningen, The NetherlandsFrench-NRC for CPEs, Service de Bactériologie-Hygiène, Hôpital de Bicêtre, Le Kremlin-Bicêtre, FranceInstitute of Medical Microbiology and Hygiene, University of Tübingen, Tübingen, GermanyHvidovre University Hospital, Hvidovre, DenmarkEuropean Commission Publications Office, Luxembourg, LuxembourgIAME, Université de Paris, Paris, FranceDepartment of Pathology, University of Utah, Salt Lake City, UT, USAEuropean Commission Joint Research Centre, Ispra, ItalyFondazione Edmund Mach, San Michele all'Adige (TN), ItalyEuropean Commission Joint Research Centre, Ispra, ItalyEuropean Commission Joint Research Centre, Ispra, ItalyEuropean Commission Joint Research Centre, Ispra, ItalyBIOMES. NGS GmbH c/o Technische Hochschule Wildau, Wildau, GermanyInstitute of Molecular Life Sciences, University of Zurich, Zurich, SwitzerlandUnité Antibiorésistance et Virulence Bactériennes, ANSES Site de Lyon, Lyon, FranceEuropean Commission Joint Research Centre, Ispra, ItalyEuropean Commission Joint Research Centre, Ispra, ItalyThe European Centre for Disease Prevention and Control, Stockholm, SwedenNational Food Institute, Technical University of Denmark, Lyngby, DenmarkEuropean Commission Joint Research Centre, Ispra, ItalyServicio de Microbiología, Hospital Universitario Ramón y Cajal, Instituto Ramón y Cajal de Investigación Sanitaria (IRYCIS), Madrid, SpainInternational Genome Centre, Jiangsu University, Zhenjiang, ChinaMSD SHARP & DOHME GMBH, Haar, GermanyBarcelona Supercomputing Centre (BSC), Barcelona, SpainOttawa Laboratory – Carling, Canadian Food Inspection Agency, Ottawa, Ontario, CanadaNext Generation Sequencing technologies significantly impact the field of Antimicrobial Resistance (AMR) detection and monitoring, with immediate uses in diagnosis and risk assessment. For this application and in general, considerable challenges remain in demonstrating sufficient trust to act upon the meaningful information produced from raw data, partly because of the reliance on bioinformatics pipelines, which can produce different results and therefore lead to different interpretations. With the constant evolution of the field, it is difficult to identify, harmonise and recommend specific methods for large-scale implementations over time. In this article, we propose to address this challenge through establishing a transparent, performance-based, evaluation approach to provide flexibility in the bioinformatics tools of choice, while demonstrating proficiency in meeting common performance standards. The approach is two-fold: first, a community-driven effort to establish and maintain “live” (dynamic) benchmarking platforms to provide relevant performance metrics, based on different use-cases, that would evolve together with the AMR field; second, agreed and defined datasets to allow the pipelines’ implementation, validation, and quality-control over time. Following previous discussions on the main challenges linked to this approach, we provide concrete recommendations and future steps, related to different aspects of the design of benchmarks, such as the selection and the characteristics of the datasets (quality, choice of pathogens and resistances, etc.), the evaluation criteria of the pipelines, and the way these resources should be deployed in the community.https://f1000research.com/articles/10-80/v2Antimicrobial resistance bioinformatics next-generation sequencing benchmarkingeng |
| spellingShingle | Alison E. Mather Alice Carolyn McHardy Ivo Gut Gemma L. Kay Kevin Vanneste Arthur Pightling John Rossen Thierry Naas Silke Peter Henrik Westh Alexandre Angers-Loustau Etienne Ruppé Robert Schlaberg Marco Fabbri Alessandro Cestaro Maddalena Querci Barbara Raffael Mauro Petrillo Paul Hammer Lukas M. Weber Jean-Yves Madec Valentina Paracchini Guy Van den Eede Erik Alm Derya Aytan-Aktug Dafni Maria Kagkli Teresa Coque Kok-Gan Chan Christoph Endrullat Salvador Capella-Gutierrez Catherine Carrillo A roadmap for the generation of benchmarking resources for antimicrobial resistance detection using next generation sequencing [version 2; peer review: 1 approved, 2 approved with reservations] F1000Research Antimicrobial resistance bioinformatics next-generation sequencing benchmarking eng |
| title | A roadmap for the generation of benchmarking resources for antimicrobial resistance detection using next generation sequencing [version 2; peer review: 1 approved, 2 approved with reservations] |
| title_full | A roadmap for the generation of benchmarking resources for antimicrobial resistance detection using next generation sequencing [version 2; peer review: 1 approved, 2 approved with reservations] |
| title_fullStr | A roadmap for the generation of benchmarking resources for antimicrobial resistance detection using next generation sequencing [version 2; peer review: 1 approved, 2 approved with reservations] |
| title_full_unstemmed | A roadmap for the generation of benchmarking resources for antimicrobial resistance detection using next generation sequencing [version 2; peer review: 1 approved, 2 approved with reservations] |
| title_short | A roadmap for the generation of benchmarking resources for antimicrobial resistance detection using next generation sequencing [version 2; peer review: 1 approved, 2 approved with reservations] |
| title_sort | roadmap for the generation of benchmarking resources for antimicrobial resistance detection using next generation sequencing version 2 peer review 1 approved 2 approved with reservations |
| topic | Antimicrobial resistance bioinformatics next-generation sequencing benchmarking eng |
| url | https://f1000research.com/articles/10-80/v2 |
| work_keys_str_mv | AT alisonemather aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT alicecarolynmchardy aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT ivogut aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT gemmalkay aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT kevinvanneste aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT arthurpightling aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT johnrossen aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT thierrynaas aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT silkepeter aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT henrikwesth aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT alexandreangersloustau aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT etienneruppe aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT robertschlaberg aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT marcofabbri aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT alessandrocestaro aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT maddalenaquerci aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT barbararaffael aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT mauropetrillo aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT paulhammer aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT lukasmweber aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT jeanyvesmadec aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT valentinaparacchini aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT guyvandeneede aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT erikalm aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT deryaaytanaktug aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT dafnimariakagkli aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT teresacoque aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT kokganchan aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT christophendrullat aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT salvadorcapellagutierrez aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT catherinecarrillo aroadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT alisonemather roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT alicecarolynmchardy roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT ivogut roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT gemmalkay roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT kevinvanneste roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT arthurpightling roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT johnrossen roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT thierrynaas roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT silkepeter roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT henrikwesth roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT alexandreangersloustau roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT etienneruppe roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT robertschlaberg roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT marcofabbri roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT alessandrocestaro roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT maddalenaquerci roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT barbararaffael roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT mauropetrillo roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT paulhammer roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT lukasmweber roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT jeanyvesmadec roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT valentinaparacchini roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT guyvandeneede roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT erikalm roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT deryaaytanaktug roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT dafnimariakagkli roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT teresacoque roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT kokganchan roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT christophendrullat roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT salvadorcapellagutierrez roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations AT catherinecarrillo roadmapforthegenerationofbenchmarkingresourcesforantimicrobialresistancedetectionusingnextgenerationsequencingversion2peerreview1approved2approvedwithreservations |