The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data
Next-generation DNA sequencing (NGS) projects, such as the 1000 Genomes Project, are already revolutionizing our understanding of genetic variation among individuals. However, the massive data sets generated by NGS—the 1000 Genome pilot alone includes nearly five terabases—make writing feature-rich,...
| Main Authors: | , , , , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Cold Spring Harbor Laboratory Press
2014
|
| Online Access: | http://hdl.handle.net/1721.1/88421 https://orcid.org/0000-0002-7250-4107 |
| _version_ | 1826190170336002048 |
|---|---|
| author | McKenna, Aaron Hanna, Matthew Sivachenko, Andrey Cibulskis, Kristian Kernytsky, Andrew Garimella, Kiran Altshuler, David Gabriel, Stacey B. Daly, Mark J. DePristo, Mark A. Banks, Eric, 1976- |
| author2 | Massachusetts Institute of Technology. Department of Biology |
| author_facet | Massachusetts Institute of Technology. Department of Biology McKenna, Aaron Hanna, Matthew Sivachenko, Andrey Cibulskis, Kristian Kernytsky, Andrew Garimella, Kiran Altshuler, David Gabriel, Stacey B. Daly, Mark J. DePristo, Mark A. Banks, Eric, 1976- |
| author_sort | McKenna, Aaron |
| collection | MIT |
| description | Next-generation DNA sequencing (NGS) projects, such as the 1000 Genomes Project, are already revolutionizing our understanding of genetic variation among individuals. However, the massive data sets generated by NGS—the 1000 Genome pilot alone includes nearly five terabases—make writing feature-rich, efficient, and robust analysis tools difficult for even computationally sophisticated individuals. Indeed, many professionals are limited in the scope and the ease with which they can answer scientific questions by the complexity of accessing and manipulating the data produced by these machines. Here, we discuss our Genome Analysis Toolkit (GATK), a structured programming framework designed to ease the development of efficient and robust analysis tools for next-generation DNA sequencers using the functional programming philosophy of MapReduce. The GATK provides a small but rich set of data access patterns that encompass the majority of analysis tool needs. Separating specific analysis calculations from common data management infrastructure enables us to optimize the GATK framework for correctness, stability, and CPU and memory efficiency and to enable distributed and shared memory parallelization. We highlight the capabilities of the GATK by describing the implementation and application of robust, scale-tolerant tools like coverage calculators and single nucleotide polymorphism (SNP) calling. We conclude that the GATK programming framework enables developers and analysts to quickly and easily write efficient and robust NGS tools, many of which have already been incorporated into large-scale sequencing projects like the 1000 Genomes Project and The Cancer Genome Atlas. |
| first_indexed | 2024-09-23T08:36:10Z |
| format | Article |
| id | mit-1721.1/88421 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T08:36:10Z |
| publishDate | 2014 |
| publisher | Cold Spring Harbor Laboratory Press |
| record_format | dspace |
| spelling | mit-1721.1/884212022-09-30T09:52:47Z The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data McKenna, Aaron Hanna, Matthew Sivachenko, Andrey Cibulskis, Kristian Kernytsky, Andrew Garimella, Kiran Altshuler, David Gabriel, Stacey B. Daly, Mark J. DePristo, Mark A. Banks, Eric, 1976- Massachusetts Institute of Technology. Department of Biology Altshuler, David Next-generation DNA sequencing (NGS) projects, such as the 1000 Genomes Project, are already revolutionizing our understanding of genetic variation among individuals. However, the massive data sets generated by NGS—the 1000 Genome pilot alone includes nearly five terabases—make writing feature-rich, efficient, and robust analysis tools difficult for even computationally sophisticated individuals. Indeed, many professionals are limited in the scope and the ease with which they can answer scientific questions by the complexity of accessing and manipulating the data produced by these machines. Here, we discuss our Genome Analysis Toolkit (GATK), a structured programming framework designed to ease the development of efficient and robust analysis tools for next-generation DNA sequencers using the functional programming philosophy of MapReduce. The GATK provides a small but rich set of data access patterns that encompass the majority of analysis tool needs. Separating specific analysis calculations from common data management infrastructure enables us to optimize the GATK framework for correctness, stability, and CPU and memory efficiency and to enable distributed and shared memory parallelization. We highlight the capabilities of the GATK by describing the implementation and application of robust, scale-tolerant tools like coverage calculators and single nucleotide polymorphism (SNP) calling. We conclude that the GATK programming framework enables developers and analysts to quickly and easily write efficient and robust NGS tools, many of which have already been incorporated into large-scale sequencing projects like the 1000 Genomes Project and The Cancer Genome Atlas. National Human Genome Research Institute (U.S.) (Large Scale Sequencing and Analysis of Genomes grant (54 HG003067)) National Human Genome Research Institute (U.S.) (Joint SNP and CNV calling in 1000 Genomes sequence data grant (U01 HG005208)) 2014-07-17T14:42:01Z 2014-07-17T14:42:01Z 2010-07 Article http://purl.org/eprint/type/JournalArticle 1088-9051 http://hdl.handle.net/1721.1/88421 McKenna, A., M. Hanna, E. Banks, A. Sivachenko, K. Cibulskis, A. Kernytsky, K. Garimella, et al. “The Genome Analysis Toolkit: A MapReduce Framework for Analyzing Next-Generation DNA Sequencing Data.” Genome Research 20, no. 9 (September 1, 2010): 1297–1303. https://orcid.org/0000-0002-7250-4107 en_US http://dx.doi.org/10.1101/gr.107524.110 Genome Research Creative Commons Attribution-Noncommerical http://creativecommons.org/licenses/by-nc/3.0/ application/pdf Cold Spring Harbor Laboratory Press Cold Spring Harbor Laboratory Press |
| spellingShingle | McKenna, Aaron Hanna, Matthew Sivachenko, Andrey Cibulskis, Kristian Kernytsky, Andrew Garimella, Kiran Altshuler, David Gabriel, Stacey B. Daly, Mark J. DePristo, Mark A. Banks, Eric, 1976- The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data |
| title | The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data |
| title_full | The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data |
| title_fullStr | The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data |
| title_full_unstemmed | The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data |
| title_short | The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data |
| title_sort | genome analysis toolkit a mapreduce framework for analyzing next generation dna sequencing data |
| url | http://hdl.handle.net/1721.1/88421 https://orcid.org/0000-0002-7250-4107 |
| work_keys_str_mv | AT mckennaaaron thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT hannamatthew thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT sivachenkoandrey thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT cibulskiskristian thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT kernytskyandrew thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT garimellakiran thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT altshulerdavid thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT gabrielstaceyb thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT dalymarkj thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT depristomarka thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT bankseric1976 thegenomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT mckennaaaron genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT hannamatthew genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT sivachenkoandrey genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT cibulskiskristian genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT kernytskyandrew genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT garimellakiran genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT altshulerdavid genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT gabrielstaceyb genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT dalymarkj genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT depristomarka genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata AT bankseric1976 genomeanalysistoolkitamapreduceframeworkforanalyzingnextgenerationdnasequencingdata |