Significant Differences and Experimental Designs Do Not Necessarily Imply Clinical Relevance: Effect Sizes and Causality Claims in Antidepressant Treatments
Clinical trials are the backbone of medical scientific research. However, this experimental strategy has some drawbacks. We focused on two issues: (a) The internal validity ensured by clinical trial procedures does not necessarily allow for generalization of efficacy results to causal claims about e...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Journal of Clinical Medicine |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-0383/12/9/3181 |
| _version_ | 1797602306623012864 |
|---|---|
| author | Iván Sánchez-Iglesias Celia Martín-Aguilar |
| author_facet | Iván Sánchez-Iglesias Celia Martín-Aguilar |
| author_sort | Iván Sánchez-Iglesias |
| collection | DOAJ |
| description | Clinical trials are the backbone of medical scientific research. However, this experimental strategy has some drawbacks. We focused on two issues: (a) The internal validity ensured by clinical trial procedures does not necessarily allow for generalization of efficacy results to causal claims about effectiveness in the population. (b) Statistical significance does not imply clinical or practical significance; <i>p</i>-values should be supplemented with effect size (ES) estimators and an interpretation of the magnitude of the effects found. We conducted a systematic review (from 2000 to 2020) on Scopus, PubMed, and four ProQuest databases, including PsycINFO. We searched for experimental studies with significant effects of pharmacological treatments on depressive symptoms, measured with a specific scale for depression. We assessed the claims of effectiveness, and reporting and interpreting of effect sizes in a small, unbiased sample of clinical trials (<i>n</i> = 10). Only 30% of the studies acknowledged that efficacy does not necessarily translate to effectiveness. Only 20% reported ES indices, and only 40% interpreted the magnitude of their findings. We encourage reflection on the applicability of results derived from clinical trials about the efficacy of antidepressant treatments, which often influence daily clinical decision-making. Comparing experimental results of antidepressants with supplementary observational studies can provide clinicians with greater flexibility in prescribing medication based on patient characteristics. Furthermore, the ES of a treatment should be considered, as treatments with a small effect may be worthwhile in certain circumstances, while treatments with a large effect may be justified despite additional costs or complications. Therefore, researchers are encouraged to report and interpret ES and explicitly discuss the suitability of their sample for the clinical population to which the antidepressant treatment will be applied. |
| first_indexed | 2024-03-11T04:15:14Z |
| format | Article |
| id | doaj.art-6e26d5ffd911487d9a85a04771df0bdc |
| institution | Directory Open Access Journal |
| issn | 2077-0383 |
| language | English |
| last_indexed | 2024-03-11T04:15:14Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Clinical Medicine |
| spelling | doaj.art-6e26d5ffd911487d9a85a04771df0bdc2023-11-17T23:11:47ZengMDPI AGJournal of Clinical Medicine2077-03832023-04-01129318110.3390/jcm12093181Significant Differences and Experimental Designs Do Not Necessarily Imply Clinical Relevance: Effect Sizes and Causality Claims in Antidepressant TreatmentsIván Sánchez-Iglesias0Celia Martín-Aguilar1Department of Psychobiology & Behavioral Sciences Methods, Complutense University of Madrid, 28223 Madrid, SpainCentro Universitario San Rafael-Nebrija, Universidad Nebrija, 28036 Madrid, SpainClinical trials are the backbone of medical scientific research. However, this experimental strategy has some drawbacks. We focused on two issues: (a) The internal validity ensured by clinical trial procedures does not necessarily allow for generalization of efficacy results to causal claims about effectiveness in the population. (b) Statistical significance does not imply clinical or practical significance; <i>p</i>-values should be supplemented with effect size (ES) estimators and an interpretation of the magnitude of the effects found. We conducted a systematic review (from 2000 to 2020) on Scopus, PubMed, and four ProQuest databases, including PsycINFO. We searched for experimental studies with significant effects of pharmacological treatments on depressive symptoms, measured with a specific scale for depression. We assessed the claims of effectiveness, and reporting and interpreting of effect sizes in a small, unbiased sample of clinical trials (<i>n</i> = 10). Only 30% of the studies acknowledged that efficacy does not necessarily translate to effectiveness. Only 20% reported ES indices, and only 40% interpreted the magnitude of their findings. We encourage reflection on the applicability of results derived from clinical trials about the efficacy of antidepressant treatments, which often influence daily clinical decision-making. Comparing experimental results of antidepressants with supplementary observational studies can provide clinicians with greater flexibility in prescribing medication based on patient characteristics. Furthermore, the ES of a treatment should be considered, as treatments with a small effect may be worthwhile in certain circumstances, while treatments with a large effect may be justified despite additional costs or complications. Therefore, researchers are encouraged to report and interpret ES and explicitly discuss the suitability of their sample for the clinical population to which the antidepressant treatment will be applied.https://www.mdpi.com/2077-0383/12/9/3181antidepressantsexperimental designrandomized clinical trialRCTsignificance testingclinical relevance |
| spellingShingle | Iván Sánchez-Iglesias Celia Martín-Aguilar Significant Differences and Experimental Designs Do Not Necessarily Imply Clinical Relevance: Effect Sizes and Causality Claims in Antidepressant Treatments Journal of Clinical Medicine antidepressants experimental design randomized clinical trial RCT significance testing clinical relevance |
| title | Significant Differences and Experimental Designs Do Not Necessarily Imply Clinical Relevance: Effect Sizes and Causality Claims in Antidepressant Treatments |
| title_full | Significant Differences and Experimental Designs Do Not Necessarily Imply Clinical Relevance: Effect Sizes and Causality Claims in Antidepressant Treatments |
| title_fullStr | Significant Differences and Experimental Designs Do Not Necessarily Imply Clinical Relevance: Effect Sizes and Causality Claims in Antidepressant Treatments |
| title_full_unstemmed | Significant Differences and Experimental Designs Do Not Necessarily Imply Clinical Relevance: Effect Sizes and Causality Claims in Antidepressant Treatments |
| title_short | Significant Differences and Experimental Designs Do Not Necessarily Imply Clinical Relevance: Effect Sizes and Causality Claims in Antidepressant Treatments |
| title_sort | significant differences and experimental designs do not necessarily imply clinical relevance effect sizes and causality claims in antidepressant treatments |
| topic | antidepressants experimental design randomized clinical trial RCT significance testing clinical relevance |
| url | https://www.mdpi.com/2077-0383/12/9/3181 |
| work_keys_str_mv | AT ivansancheziglesias significantdifferencesandexperimentaldesignsdonotnecessarilyimplyclinicalrelevanceeffectsizesandcausalityclaimsinantidepressanttreatments AT celiamartinaguilar significantdifferencesandexperimentaldesignsdonotnecessarilyimplyclinicalrelevanceeffectsizesandcausalityclaimsinantidepressanttreatments |