Fixed or random? On the reliability of mixed‐effects models for a small number of levels in grouping variables
Abstract Biological data are often intrinsically hierarchical (e.g., species from different genera, plants within different mountain regions), which made mixed‐effects models a common analysis tool in ecology and evolution because they can account for the non‐independence. Many questions around thei...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-07-01
|
| Series: | Ecology and Evolution |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/ece3.9062 |
| _version_ | 1811312444542287872 |
|---|---|
| author | Johannes Oberpriller Melina de Souza Leite Maximilian Pichler |
| author_facet | Johannes Oberpriller Melina de Souza Leite Maximilian Pichler |
| author_sort | Johannes Oberpriller |
| collection | DOAJ |
| description | Abstract Biological data are often intrinsically hierarchical (e.g., species from different genera, plants within different mountain regions), which made mixed‐effects models a common analysis tool in ecology and evolution because they can account for the non‐independence. Many questions around their practical applications are solved but one is still debated: Should we treat a grouping variable with a low number of levels as a random or fixed effect? In such situations, the variance estimate of the random effect can be imprecise, but it is unknown if this affects statistical power and type I error rates of the fixed effects of interest. Here, we analyzed the consequences of treating a grouping variable with 2–8 levels as fixed or random effect in correctly specified and alternative models (under‐ or overparametrized models). We calculated type I error rates and statistical power for all‐model specifications and quantified the influences of study design on these quantities. We found no influence of model choice on type I error rate and power on the population‐level effect (slope) for random intercept‐only models. However, with varying intercepts and slopes in the data‐generating process, using a random slope and intercept model, and switching to a fixed‐effects model, in case of a singular fit, avoids overconfidence in the results. Additionally, the number and difference between levels strongly influences power and type I error. We conclude that inferring the correct random‐effect structure is of great importance to obtain correct type I error rates. We encourage to start with a mixed‐effects model independent of the number of levels in the grouping variable and switch to a fixed‐effects model only in case of a singular fit. With these recommendations, we allow for more informative choices about study design and data analysis and make ecological inference with mixed‐effects models more robust for small number of levels. |
| first_indexed | 2024-04-13T10:37:28Z |
| format | Article |
| id | doaj.art-bf44355b10aa496a8e310fa5b2cb0c3c |
| institution | Directory Open Access Journal |
| issn | 2045-7758 |
| language | English |
| last_indexed | 2024-04-13T10:37:28Z |
| publishDate | 2022-07-01 |
| publisher | Wiley |
| record_format | Article |
| series | Ecology and Evolution |
| spelling | doaj.art-bf44355b10aa496a8e310fa5b2cb0c3c2022-12-22T02:50:01ZengWileyEcology and Evolution2045-77582022-07-01127n/an/a10.1002/ece3.9062Fixed or random? On the reliability of mixed‐effects models for a small number of levels in grouping variablesJohannes Oberpriller0Melina de Souza Leite1Maximilian Pichler2Theoretical Ecology University of Regensburg Regensburg GermanyTheoretical Ecology University of Regensburg Regensburg GermanyTheoretical Ecology University of Regensburg Regensburg GermanyAbstract Biological data are often intrinsically hierarchical (e.g., species from different genera, plants within different mountain regions), which made mixed‐effects models a common analysis tool in ecology and evolution because they can account for the non‐independence. Many questions around their practical applications are solved but one is still debated: Should we treat a grouping variable with a low number of levels as a random or fixed effect? In such situations, the variance estimate of the random effect can be imprecise, but it is unknown if this affects statistical power and type I error rates of the fixed effects of interest. Here, we analyzed the consequences of treating a grouping variable with 2–8 levels as fixed or random effect in correctly specified and alternative models (under‐ or overparametrized models). We calculated type I error rates and statistical power for all‐model specifications and quantified the influences of study design on these quantities. We found no influence of model choice on type I error rate and power on the population‐level effect (slope) for random intercept‐only models. However, with varying intercepts and slopes in the data‐generating process, using a random slope and intercept model, and switching to a fixed‐effects model, in case of a singular fit, avoids overconfidence in the results. Additionally, the number and difference between levels strongly influences power and type I error. We conclude that inferring the correct random‐effect structure is of great importance to obtain correct type I error rates. We encourage to start with a mixed‐effects model independent of the number of levels in the grouping variable and switch to a fixed‐effects model only in case of a singular fit. With these recommendations, we allow for more informative choices about study design and data analysis and make ecological inference with mixed‐effects models more robust for small number of levels.https://doi.org/10.1002/ece3.9062fixed effectsgeneralized linear modelshierarchical modelsmixed‐effects modelsmultilevel modelsrandom effects |
| spellingShingle | Johannes Oberpriller Melina de Souza Leite Maximilian Pichler Fixed or random? On the reliability of mixed‐effects models for a small number of levels in grouping variables Ecology and Evolution fixed effects generalized linear models hierarchical models mixed‐effects models multilevel models random effects |
| title | Fixed or random? On the reliability of mixed‐effects models for a small number of levels in grouping variables |
| title_full | Fixed or random? On the reliability of mixed‐effects models for a small number of levels in grouping variables |
| title_fullStr | Fixed or random? On the reliability of mixed‐effects models for a small number of levels in grouping variables |
| title_full_unstemmed | Fixed or random? On the reliability of mixed‐effects models for a small number of levels in grouping variables |
| title_short | Fixed or random? On the reliability of mixed‐effects models for a small number of levels in grouping variables |
| title_sort | fixed or random on the reliability of mixed effects models for a small number of levels in grouping variables |
| topic | fixed effects generalized linear models hierarchical models mixed‐effects models multilevel models random effects |
| url | https://doi.org/10.1002/ece3.9062 |
| work_keys_str_mv | AT johannesoberpriller fixedorrandomonthereliabilityofmixedeffectsmodelsforasmallnumberoflevelsingroupingvariables AT melinadesouzaleite fixedorrandomonthereliabilityofmixedeffectsmodelsforasmallnumberoflevelsingroupingvariables AT maximilianpichler fixedorrandomonthereliabilityofmixedeffectsmodelsforasmallnumberoflevelsingroupingvariables |