To have value, comparisons of high-throughput phenotyping methods need statistical tests of bias and variance
The gap between genomics and phenomics is narrowing. The rate at which it is narrowing, however, is being slowed by improper statistical comparison of methods. Quantification using Pearson’s correlation coefficient (r) is commonly used to assess method quality, but it is an often misleading statisti...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-01-01
|
| Series: | Frontiers in Plant Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpls.2023.1325221/full |
| _version_ | 1797351449890390016 |
|---|---|
| author | Justin M. McGrath Justin M. McGrath Matthew H. Siebers Matthew H. Siebers Peng Fu Peng Fu Stephen P. Long Stephen P. Long Stephen P. Long Carl J. Bernacchi Carl J. Bernacchi Carl J. Bernacchi |
| author_facet | Justin M. McGrath Justin M. McGrath Matthew H. Siebers Matthew H. Siebers Peng Fu Peng Fu Stephen P. Long Stephen P. Long Stephen P. Long Carl J. Bernacchi Carl J. Bernacchi Carl J. Bernacchi |
| author_sort | Justin M. McGrath |
| collection | DOAJ |
| description | The gap between genomics and phenomics is narrowing. The rate at which it is narrowing, however, is being slowed by improper statistical comparison of methods. Quantification using Pearson’s correlation coefficient (r) is commonly used to assess method quality, but it is an often misleading statistic for this purpose as it is unable to provide information about the relative quality of two methods. Using r can both erroneously discount methods that are inherently more precise and validate methods that are less accurate. These errors occur because of logical flaws inherent in the use of r when comparing methods, not as a problem of limited sample size or the unavoidable possibility of a type I error. A popular alternative to using r is to measure the limits of agreement (LOA). However both r and LOA fail to identify which instrument is more or less variable than the other and can lead to incorrect conclusions about method quality. An alternative approach, comparing variances of methods, requires repeated measurements of the same subject, but avoids incorrect conclusions. Variance comparison is arguably the most important component of method validation and, thus, when repeated measurements are possible, variance comparison provides considerable value to these studies. Statistical tests to compare variances presented here are well established, easy to interpret and ubiquitously available. The widespread use of r has potentially led to numerous incorrect conclusions about method quality, hampering development, and the approach described here would be useful to advance high throughput phenotyping methods but can also extend into any branch of science. The adoption of the statistical techniques outlined in this paper will help speed the adoption of new high throughput phenotyping techniques by indicating when one should reject a new method, outright replace an old method or conditionally use a new method. |
| first_indexed | 2024-03-08T12:59:37Z |
| format | Article |
| id | doaj.art-b299d2a174e14b9da2fb98a448f9f0c5 |
| institution | Directory Open Access Journal |
| issn | 1664-462X |
| language | English |
| last_indexed | 2024-03-08T12:59:37Z |
| publishDate | 2024-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Plant Science |
| spelling | doaj.art-b299d2a174e14b9da2fb98a448f9f0c52024-01-19T11:33:15ZengFrontiers Media S.A.Frontiers in Plant Science1664-462X2024-01-011410.3389/fpls.2023.13252211325221To have value, comparisons of high-throughput phenotyping methods need statistical tests of bias and varianceJustin M. McGrath0Justin M. McGrath1Matthew H. Siebers2Matthew H. Siebers3Peng Fu4Peng Fu5Stephen P. Long6Stephen P. Long7Stephen P. Long8Carl J. Bernacchi9Carl J. Bernacchi10Carl J. Bernacchi11Global Change and Photosynthesis Research Unit, USDA-Agricultural Research Service (ARS), Urbana, IL, United StatesDepartment of Plant Biology, University of Illinois, Urbana-Champaign, Urbana, IL, United StatesGlobal Change and Photosynthesis Research Unit, USDA-Agricultural Research Service (ARS), Urbana, IL, United StatesDepartment of Plant Biology, University of Illinois, Urbana-Champaign, Urbana, IL, United StatesCenter for Advanced Agriculture and Sustainability, Harrisburg University of Science and Technology, Harrisburg, PA, United StatesCarl R. Woese Institute for Genomic Biology, University of Illinois, Urbana-Champaign, Urbana, IL, United StatesDepartment of Plant Biology, University of Illinois, Urbana-Champaign, Urbana, IL, United StatesCarl R. Woese Institute for Genomic Biology, University of Illinois, Urbana-Champaign, Urbana, IL, United StatesDepartment of Crop Sciences, University of Illinois, Urbana-Champaign, Urbana, IL, United StatesGlobal Change and Photosynthesis Research Unit, USDA-Agricultural Research Service (ARS), Urbana, IL, United StatesDepartment of Plant Biology, University of Illinois, Urbana-Champaign, Urbana, IL, United StatesCarl R. Woese Institute for Genomic Biology, University of Illinois, Urbana-Champaign, Urbana, IL, United StatesThe gap between genomics and phenomics is narrowing. The rate at which it is narrowing, however, is being slowed by improper statistical comparison of methods. Quantification using Pearson’s correlation coefficient (r) is commonly used to assess method quality, but it is an often misleading statistic for this purpose as it is unable to provide information about the relative quality of two methods. Using r can both erroneously discount methods that are inherently more precise and validate methods that are less accurate. These errors occur because of logical flaws inherent in the use of r when comparing methods, not as a problem of limited sample size or the unavoidable possibility of a type I error. A popular alternative to using r is to measure the limits of agreement (LOA). However both r and LOA fail to identify which instrument is more or less variable than the other and can lead to incorrect conclusions about method quality. An alternative approach, comparing variances of methods, requires repeated measurements of the same subject, but avoids incorrect conclusions. Variance comparison is arguably the most important component of method validation and, thus, when repeated measurements are possible, variance comparison provides considerable value to these studies. Statistical tests to compare variances presented here are well established, easy to interpret and ubiquitously available. The widespread use of r has potentially led to numerous incorrect conclusions about method quality, hampering development, and the approach described here would be useful to advance high throughput phenotyping methods but can also extend into any branch of science. The adoption of the statistical techniques outlined in this paper will help speed the adoption of new high throughput phenotyping techniques by indicating when one should reject a new method, outright replace an old method or conditionally use a new method.https://www.frontiersin.org/articles/10.3389/fpls.2023.1325221/fullphysical sciencesstatistics method comparisonvariancebiaslimits of agreementBland and Altman |
| spellingShingle | Justin M. McGrath Justin M. McGrath Matthew H. Siebers Matthew H. Siebers Peng Fu Peng Fu Stephen P. Long Stephen P. Long Stephen P. Long Carl J. Bernacchi Carl J. Bernacchi Carl J. Bernacchi To have value, comparisons of high-throughput phenotyping methods need statistical tests of bias and variance Frontiers in Plant Science physical sciences statistics method comparison variance bias limits of agreement Bland and Altman |
| title | To have value, comparisons of high-throughput phenotyping methods need statistical tests of bias and variance |
| title_full | To have value, comparisons of high-throughput phenotyping methods need statistical tests of bias and variance |
| title_fullStr | To have value, comparisons of high-throughput phenotyping methods need statistical tests of bias and variance |
| title_full_unstemmed | To have value, comparisons of high-throughput phenotyping methods need statistical tests of bias and variance |
| title_short | To have value, comparisons of high-throughput phenotyping methods need statistical tests of bias and variance |
| title_sort | to have value comparisons of high throughput phenotyping methods need statistical tests of bias and variance |
| topic | physical sciences statistics method comparison variance bias limits of agreement Bland and Altman |
| url | https://www.frontiersin.org/articles/10.3389/fpls.2023.1325221/full |
| work_keys_str_mv | AT justinmmcgrath tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT justinmmcgrath tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT matthewhsiebers tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT matthewhsiebers tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT pengfu tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT pengfu tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT stephenplong tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT stephenplong tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT stephenplong tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT carljbernacchi tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT carljbernacchi tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance AT carljbernacchi tohavevaluecomparisonsofhighthroughputphenotypingmethodsneedstatisticaltestsofbiasandvariance |