Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness
Abstract Modern coexistence theory (MCT) offers a conceptually straightforward approach for connecting empirical observations with an elegant theoretical framework, gaining popularity rapidly over the past decade. However, beneath this surface‐level simplicity lie various assumptions and subjective...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-04-01
|
| Series: | Methods in Ecology and Evolution |
| Subjects: | |
| Online Access: | https://doi.org/10.1111/2041-210X.14227 |
| _version_ | 1797226552849596416 |
|---|---|
| author | J. Christopher D. Terry David W. Armitage |
| author_facet | J. Christopher D. Terry David W. Armitage |
| author_sort | J. Christopher D. Terry |
| collection | DOAJ |
| description | Abstract Modern coexistence theory (MCT) offers a conceptually straightforward approach for connecting empirical observations with an elegant theoretical framework, gaining popularity rapidly over the past decade. However, beneath this surface‐level simplicity lie various assumptions and subjective choices made during data analysis. These can lead researchers to draw qualitatively different conclusions from the same set of experiments. As the predictions of MCT studies are often treated as outcomes, and many readers and reviewers may not be familiar with the framework's assumptions, there is a particular risk of ‘researcher degrees of freedom’ inflating the confidence in results, thereby affecting reproducibility and predictive power. To tackle these concerns, we introduce a checklist consisting of statistical best practices to promote more robust empirical applications of MCT. Our recommendations are organised into four categories: presentation and sharing of raw data, testing model assumptions and fits, managing uncertainty associated with model coefficients and incorporating this uncertainty into coexistence predictions. We surveyed empirical MCT studies published over the past 15 years and discovered a high degree of variation in the level of statistical rigour and adherence to best practices. We present case studies to illustrate the dependence of results on seemingly innocuous choices among competition model structure and error distributions, which in some cases reversed the predicted coexistence outcomes. These results demonstrate how different analytical approaches can profoundly alter the interpretation of experimental results, underscoring the importance of carefully considering and thoroughly justifying each step taken in the analysis pathway. Our checklist serves as a resource for authors and reviewers alike, providing guidance to strengthen the empirical foundation of empirical coexistence analyses. As the field of empirical MCT shifts from a descriptive, trailblazing phase to a stage of consolidation, we emphasise the need for caution when building upon the findings of earlier studies. To ensure that progress made in the field of ecological coexistence is based on robust and reliable evidence, it is crucial to subject our predictions, conclusions and generalisability to a more rigorous assessment than is currently the trend. |
| first_indexed | 2024-04-24T14:26:44Z |
| format | Article |
| id | doaj.art-5bd6cb5ff17f49908aef9b5402b7d5c9 |
| institution | Directory Open Access Journal |
| issn | 2041-210X |
| language | English |
| last_indexed | 2024-04-24T14:26:44Z |
| publishDate | 2024-04-01 |
| publisher | Wiley |
| record_format | Article |
| series | Methods in Ecology and Evolution |
| spelling | doaj.art-5bd6cb5ff17f49908aef9b5402b7d5c92024-04-03T04:38:58ZengWileyMethods in Ecology and Evolution2041-210X2024-04-0115459461110.1111/2041-210X.14227Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustnessJ. Christopher D. Terry0David W. Armitage1Department of Biology University of Oxford Oxford UKIntegrative Community Ecology Unit Okinawa Institute of Science and Technology Graduate University Onna Okinawa JapanAbstract Modern coexistence theory (MCT) offers a conceptually straightforward approach for connecting empirical observations with an elegant theoretical framework, gaining popularity rapidly over the past decade. However, beneath this surface‐level simplicity lie various assumptions and subjective choices made during data analysis. These can lead researchers to draw qualitatively different conclusions from the same set of experiments. As the predictions of MCT studies are often treated as outcomes, and many readers and reviewers may not be familiar with the framework's assumptions, there is a particular risk of ‘researcher degrees of freedom’ inflating the confidence in results, thereby affecting reproducibility and predictive power. To tackle these concerns, we introduce a checklist consisting of statistical best practices to promote more robust empirical applications of MCT. Our recommendations are organised into four categories: presentation and sharing of raw data, testing model assumptions and fits, managing uncertainty associated with model coefficients and incorporating this uncertainty into coexistence predictions. We surveyed empirical MCT studies published over the past 15 years and discovered a high degree of variation in the level of statistical rigour and adherence to best practices. We present case studies to illustrate the dependence of results on seemingly innocuous choices among competition model structure and error distributions, which in some cases reversed the predicted coexistence outcomes. These results demonstrate how different analytical approaches can profoundly alter the interpretation of experimental results, underscoring the importance of carefully considering and thoroughly justifying each step taken in the analysis pathway. Our checklist serves as a resource for authors and reviewers alike, providing guidance to strengthen the empirical foundation of empirical coexistence analyses. As the field of empirical MCT shifts from a descriptive, trailblazing phase to a stage of consolidation, we emphasise the need for caution when building upon the findings of earlier studies. To ensure that progress made in the field of ecological coexistence is based on robust and reliable evidence, it is crucial to subject our predictions, conclusions and generalisability to a more rigorous assessment than is currently the trend.https://doi.org/10.1111/2041-210X.14227competitionexperimentsmodel selectionmodern coexistence theoryuncertainty propagation |
| spellingShingle | J. Christopher D. Terry David W. Armitage Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness Methods in Ecology and Evolution competition experiments model selection modern coexistence theory uncertainty propagation |
| title | Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title_full | Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title_fullStr | Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title_full_unstemmed | Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title_short | Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title_sort | widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| topic | competition experiments model selection modern coexistence theory uncertainty propagation |
| url | https://doi.org/10.1111/2041-210X.14227 |
| work_keys_str_mv | AT jchristopherdterry widespreadanalyticalpitfallsinempiricalcoexistencestudiesandachecklistforimprovingtheirstatisticalrobustness AT davidwarmitage widespreadanalyticalpitfallsinempiricalcoexistencestudiesandachecklistforimprovingtheirstatisticalrobustness |