Perceived Statistical Knowledge Level and Self-Reported Statistical Practice Among Academic Psychologists
Introduction: Publications arguing against the null hypothesis significance testing (NHST) procedure and in favor of good statistical practices have increased. The most frequently mentioned alternatives to NHST are effect size statistics (ES), confidence intervals (CIs), and meta-analyses. A recent...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2018-06-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fpsyg.2018.00996/full |
| _version_ | 1819267759730589696 |
|---|---|
| author | Laura Badenes-Ribera Dolores Frias-Navarro Nathalie O. Iotti Amparo Bonilla-Campos Claudio Longobardi |
| author_facet | Laura Badenes-Ribera Dolores Frias-Navarro Nathalie O. Iotti Amparo Bonilla-Campos Claudio Longobardi |
| author_sort | Laura Badenes-Ribera |
| collection | DOAJ |
| description | Introduction: Publications arguing against the null hypothesis significance testing (NHST) procedure and in favor of good statistical practices have increased. The most frequently mentioned alternatives to NHST are effect size statistics (ES), confidence intervals (CIs), and meta-analyses. A recent survey conducted in Spain found that academic psychologists have poor knowledge about effect size statistics, confidence intervals, and graphic displays for meta-analyses, which might lead to a misinterpretation of the results. In addition, it also found that, although the use of ES is becoming generalized, the same thing is not true for CIs. Finally, academics with greater knowledge about ES statistics presented a profile closer to good statistical practice and research design. Our main purpose was to analyze the extension of these results to a different geographical area through a replication study.Methods: For this purpose, we elaborated an on-line survey that included the same items as the original research, and we asked academic psychologists to indicate their level of knowledge about ES, their CIs, and meta-analyses, and how they use them. The sample consisted of 159 Italian academic psychologists (54.09% women, mean age of 47.65 years). The mean number of years in the position of professor was 12.90 (SD = 10.21).Results: As in the original research, the results showed that, although the use of effect size estimates is becoming generalized, an under-reporting of CIs for ES persists. The most frequent ES statistics mentioned were Cohen's d and R2/η2, which can have outliers or show non-normality or violate statistical assumptions. In addition, academics showed poor knowledge about meta-analytic displays (e.g., forest plot and funnel plot) and quality checklists for studies. Finally, academics with higher-level knowledge about ES statistics seem to have a profile closer to good statistical practices.Conclusions: Changing statistical practice is not easy.This change requires statistical training programs for academics, both graduate and undergraduate. |
| first_indexed | 2024-12-23T21:22:17Z |
| format | Article |
| id | doaj.art-01d947f898eb4236ad32fe45fd990366 |
| institution | Directory Open Access Journal |
| issn | 1664-1078 |
| language | English |
| last_indexed | 2024-12-23T21:22:17Z |
| publishDate | 2018-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj.art-01d947f898eb4236ad32fe45fd9903662022-12-21T17:30:43ZengFrontiers Media S.A.Frontiers in Psychology1664-10782018-06-01910.3389/fpsyg.2018.00996349696Perceived Statistical Knowledge Level and Self-Reported Statistical Practice Among Academic PsychologistsLaura Badenes-Ribera0Dolores Frias-Navarro1Nathalie O. Iotti2Amparo Bonilla-Campos3Claudio Longobardi4Departament de Metodologia de les Ciències del Comportament, Universitat de València, Valencia, SpainDepartament de Metodologia de les Ciències del Comportament, Universitat de València, Valencia, SpainDipartimento di Psicologia, Università degli Studi di Torino, Turin, ItalyDepartament de Metodologia de les Ciències del Comportament, Universitat de València, Valencia, SpainDipartimento di Psicologia, Università degli Studi di Torino, Turin, ItalyIntroduction: Publications arguing against the null hypothesis significance testing (NHST) procedure and in favor of good statistical practices have increased. The most frequently mentioned alternatives to NHST are effect size statistics (ES), confidence intervals (CIs), and meta-analyses. A recent survey conducted in Spain found that academic psychologists have poor knowledge about effect size statistics, confidence intervals, and graphic displays for meta-analyses, which might lead to a misinterpretation of the results. In addition, it also found that, although the use of ES is becoming generalized, the same thing is not true for CIs. Finally, academics with greater knowledge about ES statistics presented a profile closer to good statistical practice and research design. Our main purpose was to analyze the extension of these results to a different geographical area through a replication study.Methods: For this purpose, we elaborated an on-line survey that included the same items as the original research, and we asked academic psychologists to indicate their level of knowledge about ES, their CIs, and meta-analyses, and how they use them. The sample consisted of 159 Italian academic psychologists (54.09% women, mean age of 47.65 years). The mean number of years in the position of professor was 12.90 (SD = 10.21).Results: As in the original research, the results showed that, although the use of effect size estimates is becoming generalized, an under-reporting of CIs for ES persists. The most frequent ES statistics mentioned were Cohen's d and R2/η2, which can have outliers or show non-normality or violate statistical assumptions. In addition, academics showed poor knowledge about meta-analytic displays (e.g., forest plot and funnel plot) and quality checklists for studies. Finally, academics with higher-level knowledge about ES statistics seem to have a profile closer to good statistical practices.Conclusions: Changing statistical practice is not easy.This change requires statistical training programs for academics, both graduate and undergraduate.https://www.frontiersin.org/article/10.3389/fpsyg.2018.00996/fulleffect sizeconfidence intervalmeta-analysissurvey studyeducation |
| spellingShingle | Laura Badenes-Ribera Dolores Frias-Navarro Nathalie O. Iotti Amparo Bonilla-Campos Claudio Longobardi Perceived Statistical Knowledge Level and Self-Reported Statistical Practice Among Academic Psychologists Frontiers in Psychology effect size confidence interval meta-analysis survey study education |
| title | Perceived Statistical Knowledge Level and Self-Reported Statistical Practice Among Academic Psychologists |
| title_full | Perceived Statistical Knowledge Level and Self-Reported Statistical Practice Among Academic Psychologists |
| title_fullStr | Perceived Statistical Knowledge Level and Self-Reported Statistical Practice Among Academic Psychologists |
| title_full_unstemmed | Perceived Statistical Knowledge Level and Self-Reported Statistical Practice Among Academic Psychologists |
| title_short | Perceived Statistical Knowledge Level and Self-Reported Statistical Practice Among Academic Psychologists |
| title_sort | perceived statistical knowledge level and self reported statistical practice among academic psychologists |
| topic | effect size confidence interval meta-analysis survey study education |
| url | https://www.frontiersin.org/article/10.3389/fpsyg.2018.00996/full |
| work_keys_str_mv | AT laurabadenesribera perceivedstatisticalknowledgelevelandselfreportedstatisticalpracticeamongacademicpsychologists AT doloresfriasnavarro perceivedstatisticalknowledgelevelandselfreportedstatisticalpracticeamongacademicpsychologists AT nathalieoiotti perceivedstatisticalknowledgelevelandselfreportedstatisticalpracticeamongacademicpsychologists AT amparobonillacampos perceivedstatisticalknowledgelevelandselfreportedstatisticalpracticeamongacademicpsychologists AT claudiolongobardi perceivedstatisticalknowledgelevelandselfreportedstatisticalpracticeamongacademicpsychologists |