Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness
1. Modern coexistence theory (MCT) offers a conceptually straightforward approach for connecting empirical observations with an elegant theoretical framework, gaining popularity rapidly over the past decade. However, beneath this surface-level simplicity lie various assumptions and subjective choice...
| Main Authors: | , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
British Ecological Society
2024
|
| _version_ | 1826314368583729152 |
|---|---|
| author | Terry, JCD Armitage, DW |
| author_facet | Terry, JCD Armitage, DW |
| author_sort | Terry, JCD |
| collection | OXFORD |
| description | 1. Modern coexistence theory (MCT) offers a conceptually straightforward approach for connecting empirical observations with an elegant theoretical framework, gaining popularity rapidly over the past decade. However, beneath this surface-level simplicity lie various assumptions and subjective choices made during data analysis. These can lead researchers to draw qualitatively different conclusions from the same set of experiments. As the predictions of MCT studies are often treated as outcomes, and many readers and reviewers may not be familiar with the framework's assumptions, there is a particular risk of ‘researcher degrees of freedom’ inflating the confidence in results, thereby affecting reproducibility and predictive power.
<br>
2. To tackle these concerns, we introduce a checklist consisting of statistical best practices to promote more robust empirical applications of MCT. Our recommendations are organised into four categories: presentation and sharing of raw data, testing model assumptions and fits, managing uncertainty associated with model coefficients and incorporating this uncertainty into coexistence predictions.
<br>
3. We surveyed empirical MCT studies published over the past 15 years and discovered a high degree of variation in the level of statistical rigour and adherence to best practices. We present case studies to illustrate the dependence of results on seemingly innocuous choices among competition model structure and error distributions, which in some cases reversed the predicted coexistence outcomes. These results demonstrate how different analytical approaches can profoundly alter the interpretation of experimental results, underscoring the importance of carefully considering and thoroughly justifying each step taken in the analysis pathway.
<br>
4.Our checklist serves as a resource for authors and reviewers alike, providing guidance to strengthen the empirical foundation of empirical coexistence analyses. As the field of empirical MCT shifts from a descriptive, trailblazing phase to a stage of consolidation, we emphasise the need for caution when building upon the findings of earlier studies. To ensure that progress made in the field of ecological coexistence is based on robust and reliable evidence, it is crucial to subject our predictions, conclusions and generalisability to a more rigorous assessment than is currently the trend.
|
| first_indexed | 2024-03-07T08:28:47Z |
| format | Journal article |
| id | oxford-uuid:2999bb42-3d9f-48dd-8ce7-57c77eec9879 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-09-25T04:31:28Z |
| publishDate | 2024 |
| publisher | British Ecological Society |
| record_format | dspace |
| spelling | oxford-uuid:2999bb42-3d9f-48dd-8ce7-57c77eec98792024-09-02T08:55:40ZWidespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustnessJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:2999bb42-3d9f-48dd-8ce7-57c77eec9879EnglishSymplectic ElementsBritish Ecological Society2024Terry, JCDArmitage, DW1. Modern coexistence theory (MCT) offers a conceptually straightforward approach for connecting empirical observations with an elegant theoretical framework, gaining popularity rapidly over the past decade. However, beneath this surface-level simplicity lie various assumptions and subjective choices made during data analysis. These can lead researchers to draw qualitatively different conclusions from the same set of experiments. As the predictions of MCT studies are often treated as outcomes, and many readers and reviewers may not be familiar with the framework's assumptions, there is a particular risk of ‘researcher degrees of freedom’ inflating the confidence in results, thereby affecting reproducibility and predictive power. <br> 2. To tackle these concerns, we introduce a checklist consisting of statistical best practices to promote more robust empirical applications of MCT. Our recommendations are organised into four categories: presentation and sharing of raw data, testing model assumptions and fits, managing uncertainty associated with model coefficients and incorporating this uncertainty into coexistence predictions. <br> 3. We surveyed empirical MCT studies published over the past 15 years and discovered a high degree of variation in the level of statistical rigour and adherence to best practices. We present case studies to illustrate the dependence of results on seemingly innocuous choices among competition model structure and error distributions, which in some cases reversed the predicted coexistence outcomes. These results demonstrate how different analytical approaches can profoundly alter the interpretation of experimental results, underscoring the importance of carefully considering and thoroughly justifying each step taken in the analysis pathway. <br> 4.Our checklist serves as a resource for authors and reviewers alike, providing guidance to strengthen the empirical foundation of empirical coexistence analyses. As the field of empirical MCT shifts from a descriptive, trailblazing phase to a stage of consolidation, we emphasise the need for caution when building upon the findings of earlier studies. To ensure that progress made in the field of ecological coexistence is based on robust and reliable evidence, it is crucial to subject our predictions, conclusions and generalisability to a more rigorous assessment than is currently the trend. |
| spellingShingle | Terry, JCD Armitage, DW Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title | Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title_full | Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title_fullStr | Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title_full_unstemmed | Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title_short | Widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| title_sort | widespread analytical pitfalls in empirical coexistence studies and a checklist for improving their statistical robustness |
| work_keys_str_mv | AT terryjcd widespreadanalyticalpitfallsinempiricalcoexistencestudiesandachecklistforimprovingtheirstatisticalrobustness AT armitagedw widespreadanalyticalpitfallsinempiricalcoexistencestudiesandachecklistforimprovingtheirstatisticalrobustness |