A Monte Carlo sample size study: how many countries are needed for accurate multilevel SEM?
Recently, there has been growing scientific interest for cross-national survey research. Various scholars have used multilevel techniques to link individual characteristics to aspects of the national context. At first sight, multilevel SEM seems to be a promising tool for this purpose, as it integra...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
European Survey Research Association
2009-03-01
|
| Series: | Survey Research Methods |
| Subjects: | |
| Online Access: | https://ojs.ub.uni-konstanz.de/srm/article/view/666 |
| _version_ | 1818485999918907392 |
|---|---|
| author | Bart Meuleman Jaak Billiet |
| author_facet | Bart Meuleman Jaak Billiet |
| author_sort | Bart Meuleman |
| collection | DOAJ |
| description | Recently, there has been growing scientific interest for cross-national survey research. Various scholars have used multilevel techniques to link individual characteristics to aspects of the national context. At first sight, multilevel SEM seems to be a promising tool for this purpose, as it integrates multilevel modeling within a latent variable framework. However, due to the fact that the number of countries in most international surveys does not exceed 30, the application of multilevel SEM in cross-national research is problematic.
Taking European Social Survey (ESS) data as a point of departure, this paper uses Monte Carlo studies to assess the estimation accuracy of multilevel SEM with small group sample sizes. The results indicate that a group sample size of 20 - a situation common in cross-national research - does not guarantee accurate estimation at all. Unacceptable amounts of parameter and standard error bias are present for the between-level estimates. Unless the standardized effect is very large (0.75), statistical power for detecting a significant between-level structural effect is seriously lacking. Required group sample sizes depend strongly on the specific interests of the researcher, the expected effect sizes and the complexity of the model. If the between-level model is relatively simple and one is merely interested in the between-level factor structure, a group sample size of 40 could be sufficient. To detect large (>0.50) structural effects at the between level, at least 60 groups are required. To have an acceptable probability of detecting smaller effects, more than 100 groups are needed. These guidelines are shown to be quite robust for varying cluster sizes and intra-class correlations (ICCs). |
| first_indexed | 2024-12-10T16:17:05Z |
| format | Article |
| id | doaj.art-5d70b32edc874a009dd2511d361033f3 |
| institution | Directory Open Access Journal |
| issn | 1864-3361 |
| language | English |
| last_indexed | 2024-12-10T16:17:05Z |
| publishDate | 2009-03-01 |
| publisher | European Survey Research Association |
| record_format | Article |
| series | Survey Research Methods |
| spelling | doaj.art-5d70b32edc874a009dd2511d361033f32022-12-22T01:41:56ZengEuropean Survey Research AssociationSurvey Research Methods1864-33612009-03-013110.18148/srm/2009.v3i1.6662608A Monte Carlo sample size study: how many countries are needed for accurate multilevel SEM?Bart Meuleman0Jaak Billiet1University of LeuvenUniversity of LeuvenRecently, there has been growing scientific interest for cross-national survey research. Various scholars have used multilevel techniques to link individual characteristics to aspects of the national context. At first sight, multilevel SEM seems to be a promising tool for this purpose, as it integrates multilevel modeling within a latent variable framework. However, due to the fact that the number of countries in most international surveys does not exceed 30, the application of multilevel SEM in cross-national research is problematic. Taking European Social Survey (ESS) data as a point of departure, this paper uses Monte Carlo studies to assess the estimation accuracy of multilevel SEM with small group sample sizes. The results indicate that a group sample size of 20 - a situation common in cross-national research - does not guarantee accurate estimation at all. Unacceptable amounts of parameter and standard error bias are present for the between-level estimates. Unless the standardized effect is very large (0.75), statistical power for detecting a significant between-level structural effect is seriously lacking. Required group sample sizes depend strongly on the specific interests of the researcher, the expected effect sizes and the complexity of the model. If the between-level model is relatively simple and one is merely interested in the between-level factor structure, a group sample size of 40 could be sufficient. To detect large (>0.50) structural effects at the between level, at least 60 groups are required. To have an acceptable probability of detecting smaller effects, more than 100 groups are needed. These guidelines are shown to be quite robust for varying cluster sizes and intra-class correlations (ICCs).https://ojs.ub.uni-konstanz.de/srm/article/view/666multilevel SEMsample sizeMonte Carlo studycross-national research |
| spellingShingle | Bart Meuleman Jaak Billiet A Monte Carlo sample size study: how many countries are needed for accurate multilevel SEM? Survey Research Methods multilevel SEM sample size Monte Carlo study cross-national research |
| title | A Monte Carlo sample size study: how many countries are needed for accurate multilevel SEM? |
| title_full | A Monte Carlo sample size study: how many countries are needed for accurate multilevel SEM? |
| title_fullStr | A Monte Carlo sample size study: how many countries are needed for accurate multilevel SEM? |
| title_full_unstemmed | A Monte Carlo sample size study: how many countries are needed for accurate multilevel SEM? |
| title_short | A Monte Carlo sample size study: how many countries are needed for accurate multilevel SEM? |
| title_sort | monte carlo sample size study how many countries are needed for accurate multilevel sem |
| topic | multilevel SEM sample size Monte Carlo study cross-national research |
| url | https://ojs.ub.uni-konstanz.de/srm/article/view/666 |
| work_keys_str_mv | AT bartmeuleman amontecarlosamplesizestudyhowmanycountriesareneededforaccuratemultilevelsem AT jaakbilliet amontecarlosamplesizestudyhowmanycountriesareneededforaccuratemultilevelsem AT bartmeuleman montecarlosamplesizestudyhowmanycountriesareneededforaccuratemultilevelsem AT jaakbilliet montecarlosamplesizestudyhowmanycountriesareneededforaccuratemultilevelsem |