A latent scale model to minimize subjectivity in the analysis of visual rating data for the National Turfgrass Evaluation Program
IntroductionTraditional evaluation procedure in National Turfgrass Evaluation Program (NTEP) relies on visually assessing replicated turf plots at multiple testing locations. This process yields ordinal data; however, statistical models that falsely assume these to be interval or ratio data have alm...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-07-01
|
| Series: | Frontiers in Plant Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpls.2023.1135918/full |
| _version_ | 1797778069811888128 |
|---|---|
| author | Yuanshuo Qu Len Kne Steve Graham Eric Watkins Kevin Morris |
| author_facet | Yuanshuo Qu Len Kne Steve Graham Eric Watkins Kevin Morris |
| author_sort | Yuanshuo Qu |
| collection | DOAJ |
| description | IntroductionTraditional evaluation procedure in National Turfgrass Evaluation Program (NTEP) relies on visually assessing replicated turf plots at multiple testing locations. This process yields ordinal data; however, statistical models that falsely assume these to be interval or ratio data have almost exclusively been applied in the subsequent analysis. This practice raises concerns about procedural subjectivity, preventing objective comparisons of cultivars across different test locations. It may also lead to serious errors, such as increased false alarms, failures to detect effects, and even inversions of differences among groups.MethodsWe reviewed this problem, identified sources of subjectivity, and presented a model-based approach to minimize subjectivity, allowing objective comparisons of cultivars across different locations and better monitoring of the evaluation procedure. We demonstrate how to fit the described model in a Bayesian framework with Stan, using datasets on overall turf quality ratings from the 2017 NTEP Kentucky bluegrass trials at seven testing locations.ResultsCompared with the existing method, ours allows the estimation of additional parameters, i.e., category thresholds, rating severity, and within-field spatial variations, and provides better separation of cultivar means and more realistic standard deviations.DiscussionTo implement the proposed model, additional information on rater identification, trial layout, rating date is needed. Given the model assumptions, we recommend small trials to reduce rater fatigue. For large trials, ratings can be conducted for each replication on multiple occasions instead of all at once. To minimize subjectivity, multiple raters are required. We also proposed new ideas on temporal analysis, incorporating existing knowledge of turfgrass. |
| first_indexed | 2024-03-12T23:12:02Z |
| format | Article |
| id | doaj.art-eaf60be92bdc4f17a18fc51b07bb5683 |
| institution | Directory Open Access Journal |
| issn | 1664-462X |
| language | English |
| last_indexed | 2024-03-12T23:12:02Z |
| publishDate | 2023-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Plant Science |
| spelling | doaj.art-eaf60be92bdc4f17a18fc51b07bb56832023-07-18T01:09:39ZengFrontiers Media S.A.Frontiers in Plant Science1664-462X2023-07-011410.3389/fpls.2023.11359181135918A latent scale model to minimize subjectivity in the analysis of visual rating data for the National Turfgrass Evaluation ProgramYuanshuo Qu0Len Kne1Steve Graham2Eric Watkins3Kevin Morris4National Turfgrass Evaluation Program, Beltsville, MD, United StatesU-Spatial, University of Minnesota, Minneapolis, MN, United StatesU-Spatial, University of Minnesota, Duluth, MN, United StatesDepartment of Horticultural Science, University of Minnesota, St. Paul, MN, United StatesNational Turfgrass Evaluation Program, Beltsville, MD, United StatesIntroductionTraditional evaluation procedure in National Turfgrass Evaluation Program (NTEP) relies on visually assessing replicated turf plots at multiple testing locations. This process yields ordinal data; however, statistical models that falsely assume these to be interval or ratio data have almost exclusively been applied in the subsequent analysis. This practice raises concerns about procedural subjectivity, preventing objective comparisons of cultivars across different test locations. It may also lead to serious errors, such as increased false alarms, failures to detect effects, and even inversions of differences among groups.MethodsWe reviewed this problem, identified sources of subjectivity, and presented a model-based approach to minimize subjectivity, allowing objective comparisons of cultivars across different locations and better monitoring of the evaluation procedure. We demonstrate how to fit the described model in a Bayesian framework with Stan, using datasets on overall turf quality ratings from the 2017 NTEP Kentucky bluegrass trials at seven testing locations.ResultsCompared with the existing method, ours allows the estimation of additional parameters, i.e., category thresholds, rating severity, and within-field spatial variations, and provides better separation of cultivar means and more realistic standard deviations.DiscussionTo implement the proposed model, additional information on rater identification, trial layout, rating date is needed. Given the model assumptions, we recommend small trials to reduce rater fatigue. For large trials, ratings can be conducted for each replication on multiple occasions instead of all at once. To minimize subjectivity, multiple raters are required. We also proposed new ideas on temporal analysis, incorporating existing knowledge of turfgrass.https://www.frontiersin.org/articles/10.3389/fpls.2023.1135918/fullNTEPvisual ratingscultivar evaluationsubjectivity minimizationBayesian model |
| spellingShingle | Yuanshuo Qu Len Kne Steve Graham Eric Watkins Kevin Morris A latent scale model to minimize subjectivity in the analysis of visual rating data for the National Turfgrass Evaluation Program Frontiers in Plant Science NTEP visual ratings cultivar evaluation subjectivity minimization Bayesian model |
| title | A latent scale model to minimize subjectivity in the analysis of visual rating data for the National Turfgrass Evaluation Program |
| title_full | A latent scale model to minimize subjectivity in the analysis of visual rating data for the National Turfgrass Evaluation Program |
| title_fullStr | A latent scale model to minimize subjectivity in the analysis of visual rating data for the National Turfgrass Evaluation Program |
| title_full_unstemmed | A latent scale model to minimize subjectivity in the analysis of visual rating data for the National Turfgrass Evaluation Program |
| title_short | A latent scale model to minimize subjectivity in the analysis of visual rating data for the National Turfgrass Evaluation Program |
| title_sort | latent scale model to minimize subjectivity in the analysis of visual rating data for the national turfgrass evaluation program |
| topic | NTEP visual ratings cultivar evaluation subjectivity minimization Bayesian model |
| url | https://www.frontiersin.org/articles/10.3389/fpls.2023.1135918/full |
| work_keys_str_mv | AT yuanshuoqu alatentscalemodeltominimizesubjectivityintheanalysisofvisualratingdataforthenationalturfgrassevaluationprogram AT lenkne alatentscalemodeltominimizesubjectivityintheanalysisofvisualratingdataforthenationalturfgrassevaluationprogram AT stevegraham alatentscalemodeltominimizesubjectivityintheanalysisofvisualratingdataforthenationalturfgrassevaluationprogram AT ericwatkins alatentscalemodeltominimizesubjectivityintheanalysisofvisualratingdataforthenationalturfgrassevaluationprogram AT kevinmorris alatentscalemodeltominimizesubjectivityintheanalysisofvisualratingdataforthenationalturfgrassevaluationprogram AT yuanshuoqu latentscalemodeltominimizesubjectivityintheanalysisofvisualratingdataforthenationalturfgrassevaluationprogram AT lenkne latentscalemodeltominimizesubjectivityintheanalysisofvisualratingdataforthenationalturfgrassevaluationprogram AT stevegraham latentscalemodeltominimizesubjectivityintheanalysisofvisualratingdataforthenationalturfgrassevaluationprogram AT ericwatkins latentscalemodeltominimizesubjectivityintheanalysisofvisualratingdataforthenationalturfgrassevaluationprogram AT kevinmorris latentscalemodeltominimizesubjectivityintheanalysisofvisualratingdataforthenationalturfgrassevaluationprogram |