Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors [version 1; peer review: 1 approved, 2 approved with reservations]
Sound reporting of research results is fundamental to good science. Unfortunately, poor reporting is common and does not improve with editorial educational strategies. We investigated whether publicly highlighting poor reporting at a journal can lead to improved reporting practices. We also investig...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
F1000 Research Ltd
2023-11-01
|
| Series: | F1000Research |
| Subjects: | |
| Online Access: | https://f1000research.com/articles/12-1483/v1 |
| _version_ | 1797317686499213312 |
|---|---|
| author | Simon Gandevia Lucy Robertson Annie Butler Elizabeth Bye Georgia Fisher Martin Héroux Joanna Diong |
| author_facet | Simon Gandevia Lucy Robertson Annie Butler Elizabeth Bye Georgia Fisher Martin Héroux Joanna Diong |
| author_sort | Simon Gandevia |
| collection | DOAJ |
| description | Sound reporting of research results is fundamental to good science. Unfortunately, poor reporting is common and does not improve with editorial educational strategies. We investigated whether publicly highlighting poor reporting at a journal can lead to improved reporting practices. We also investigated whether reporting practices that are required or strongly encouraged in journal Information for Authors are enforced by journal editors and staff. A 2016 audit highlighted poor reporting practices in the Journal of Neurophysiology. In August 2016 and 2018, the American Physiological Society updated the Information for Authors, which included the introduction of several required or strongly encouraged reporting practices. We audited Journal of Neurophysiology papers published in 2019 and 2020 (downloaded through the library of the University of New South Wales) on reporting items selected from the 2016 audit, the newly introduced reporting practices, and items from previous audits. Summary statistics (means, counts) were used to summarize audit results. In total, 580 papers were audited. Compared to results from the 2016 audit, several reporting practices remained unchanged or worsened. For example, 60% of papers erroneously reported standard errors of the mean, 23% of papers included undefined measures of variability, 40% of papers failed to define a statistical threshold for their tests, and when present, 64% of papers with p-values between 0.05 and 0.1 misinterpreted them as statistical trends. As for the newly introduced reporting practices, required practices were consistently adhered to by 34 to 37% of papers, while strongly encouraged practices were consistently adhered to by 9 to 26% of papers. Adherence to the other audited reporting practices was comparable to our previous audits. Publicly highlighting poor reporting practices did little to improve research reporting. Similarly, requiring or strongly encouraging reporting practices was only partly effective. Although the present audit focused on a single journal, this is likely not an isolated case. Stronger, more strategic measures are required to improve poor research reporting. |
| first_indexed | 2024-03-08T03:39:33Z |
| format | Article |
| id | doaj.art-e36f9bb7bb8141c8a827cccf2f088fc1 |
| institution | Directory Open Access Journal |
| issn | 2046-1402 |
| language | English |
| last_indexed | 2024-03-08T03:39:33Z |
| publishDate | 2023-11-01 |
| publisher | F1000 Research Ltd |
| record_format | Article |
| series | F1000Research |
| spelling | doaj.art-e36f9bb7bb8141c8a827cccf2f088fc12024-02-10T01:00:30ZengF1000 Research LtdF1000Research2046-14022023-11-0112156438Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors [version 1; peer review: 1 approved, 2 approved with reservations]Simon Gandevia0Lucy Robertson1Annie Butler2Elizabeth Bye3Georgia Fisher4Martin Héroux5https://orcid.org/0000-0002-3354-7104Joanna Diong6Neuroscience Research Australia, Sydney, NSW, 2031, AustraliaNeuroscience Research Australia, Sydney, NSW, 2031, AustraliaNeuroscience Research Australia, Sydney, NSW, 2031, AustraliaNeuroscience Research Australia, Sydney, NSW, 2031, AustraliaNeuroscience Research Australia, Sydney, NSW, 2031, AustraliaNeuroscience Research Australia, Sydney, NSW, 2031, AustraliaNeuroscience Research Australia, Sydney, NSW, 2031, AustraliaSound reporting of research results is fundamental to good science. Unfortunately, poor reporting is common and does not improve with editorial educational strategies. We investigated whether publicly highlighting poor reporting at a journal can lead to improved reporting practices. We also investigated whether reporting practices that are required or strongly encouraged in journal Information for Authors are enforced by journal editors and staff. A 2016 audit highlighted poor reporting practices in the Journal of Neurophysiology. In August 2016 and 2018, the American Physiological Society updated the Information for Authors, which included the introduction of several required or strongly encouraged reporting practices. We audited Journal of Neurophysiology papers published in 2019 and 2020 (downloaded through the library of the University of New South Wales) on reporting items selected from the 2016 audit, the newly introduced reporting practices, and items from previous audits. Summary statistics (means, counts) were used to summarize audit results. In total, 580 papers were audited. Compared to results from the 2016 audit, several reporting practices remained unchanged or worsened. For example, 60% of papers erroneously reported standard errors of the mean, 23% of papers included undefined measures of variability, 40% of papers failed to define a statistical threshold for their tests, and when present, 64% of papers with p-values between 0.05 and 0.1 misinterpreted them as statistical trends. As for the newly introduced reporting practices, required practices were consistently adhered to by 34 to 37% of papers, while strongly encouraged practices were consistently adhered to by 9 to 26% of papers. Adherence to the other audited reporting practices was comparable to our previous audits. Publicly highlighting poor reporting practices did little to improve research reporting. Similarly, requiring or strongly encouraging reporting practices was only partly effective. Although the present audit focused on a single journal, this is likely not an isolated case. Stronger, more strategic measures are required to improve poor research reporting.https://f1000research.com/articles/12-1483/v1Meta-research research quality scientific reporting reproducibilityeng |
| spellingShingle | Simon Gandevia Lucy Robertson Annie Butler Elizabeth Bye Georgia Fisher Martin Héroux Joanna Diong Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors [version 1; peer review: 1 approved, 2 approved with reservations] F1000Research Meta-research research quality scientific reporting reproducibility eng |
| title | Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors [version 1; peer review: 1 approved, 2 approved with reservations] |
| title_full | Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors [version 1; peer review: 1 approved, 2 approved with reservations] |
| title_fullStr | Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors [version 1; peer review: 1 approved, 2 approved with reservations] |
| title_full_unstemmed | Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors [version 1; peer review: 1 approved, 2 approved with reservations] |
| title_short | Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors [version 1; peer review: 1 approved, 2 approved with reservations] |
| title_sort | poor statistical reporting inadequate data presentation and spin persist despite journal awareness and updated information for authors version 1 peer review 1 approved 2 approved with reservations |
| topic | Meta-research research quality scientific reporting reproducibility eng |
| url | https://f1000research.com/articles/12-1483/v1 |
| work_keys_str_mv | AT simongandevia poorstatisticalreportinginadequatedatapresentationandspinpersistdespitejournalawarenessandupdatedinformationforauthorsversion1peerreview1approved2approvedwithreservations AT lucyrobertson poorstatisticalreportinginadequatedatapresentationandspinpersistdespitejournalawarenessandupdatedinformationforauthorsversion1peerreview1approved2approvedwithreservations AT anniebutler poorstatisticalreportinginadequatedatapresentationandspinpersistdespitejournalawarenessandupdatedinformationforauthorsversion1peerreview1approved2approvedwithreservations AT elizabethbye poorstatisticalreportinginadequatedatapresentationandspinpersistdespitejournalawarenessandupdatedinformationforauthorsversion1peerreview1approved2approvedwithreservations AT georgiafisher poorstatisticalreportinginadequatedatapresentationandspinpersistdespitejournalawarenessandupdatedinformationforauthorsversion1peerreview1approved2approvedwithreservations AT martinheroux poorstatisticalreportinginadequatedatapresentationandspinpersistdespitejournalawarenessandupdatedinformationforauthorsversion1peerreview1approved2approvedwithreservations AT joannadiong poorstatisticalreportinginadequatedatapresentationandspinpersistdespitejournalawarenessandupdatedinformationforauthorsversion1peerreview1approved2approvedwithreservations |