Replication, Communication, and the Population Dynamics of Scientific Discovery.
Many published research results are false (Ioannidis, 2005), and controversy continues over the roles of replication and publication policy in improving the reliability of research. Addressing these problems is frustrated by the lack of a formal framework that jointly represents hypothesis formation...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2015-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0136088 |
| _version_ | 1819108500207304704 |
|---|---|
| author | Richard McElreath Paul E Smaldino |
| author_facet | Richard McElreath Paul E Smaldino |
| author_sort | Richard McElreath |
| collection | DOAJ |
| description | Many published research results are false (Ioannidis, 2005), and controversy continues over the roles of replication and publication policy in improving the reliability of research. Addressing these problems is frustrated by the lack of a formal framework that jointly represents hypothesis formation, replication, publication bias, and variation in research quality. We develop a mathematical model of scientific discovery that combines all of these elements. This model provides both a dynamic model of research as well as a formal framework for reasoning about the normative structure of science. We show that replication may serve as a ratchet that gradually separates true hypotheses from false, but the same factors that make initial findings unreliable also make replications unreliable. The most important factors in improving the reliability of research are the rate of false positives and the base rate of true hypotheses, and we offer suggestions for addressing each. Our results also bring clarity to verbal debates about the communication of research. Surprisingly, publication bias is not always an obstacle, but instead may have positive impacts-suppression of negative novel findings is often beneficial. We also find that communication of negative replications may aid true discovery even when attempts to replicate have diminished power. The model speaks constructively to ongoing debates about the design and conduct of science, focusing analysis and discussion on precise, internally consistent models, as well as highlighting the importance of population dynamics. |
| first_indexed | 2024-12-22T03:10:55Z |
| format | Article |
| id | doaj.art-3e11eb1083b64cd8942dd2ea7a56a9d5 |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-22T03:10:55Z |
| publishDate | 2015-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-3e11eb1083b64cd8942dd2ea7a56a9d52022-12-21T18:40:55ZengPublic Library of Science (PLoS)PLoS ONE1932-62032015-01-01108e013608810.1371/journal.pone.0136088Replication, Communication, and the Population Dynamics of Scientific Discovery.Richard McElreathPaul E SmaldinoMany published research results are false (Ioannidis, 2005), and controversy continues over the roles of replication and publication policy in improving the reliability of research. Addressing these problems is frustrated by the lack of a formal framework that jointly represents hypothesis formation, replication, publication bias, and variation in research quality. We develop a mathematical model of scientific discovery that combines all of these elements. This model provides both a dynamic model of research as well as a formal framework for reasoning about the normative structure of science. We show that replication may serve as a ratchet that gradually separates true hypotheses from false, but the same factors that make initial findings unreliable also make replications unreliable. The most important factors in improving the reliability of research are the rate of false positives and the base rate of true hypotheses, and we offer suggestions for addressing each. Our results also bring clarity to verbal debates about the communication of research. Surprisingly, publication bias is not always an obstacle, but instead may have positive impacts-suppression of negative novel findings is often beneficial. We also find that communication of negative replications may aid true discovery even when attempts to replicate have diminished power. The model speaks constructively to ongoing debates about the design and conduct of science, focusing analysis and discussion on precise, internally consistent models, as well as highlighting the importance of population dynamics.https://doi.org/10.1371/journal.pone.0136088 |
| spellingShingle | Richard McElreath Paul E Smaldino Replication, Communication, and the Population Dynamics of Scientific Discovery. PLoS ONE |
| title | Replication, Communication, and the Population Dynamics of Scientific Discovery. |
| title_full | Replication, Communication, and the Population Dynamics of Scientific Discovery. |
| title_fullStr | Replication, Communication, and the Population Dynamics of Scientific Discovery. |
| title_full_unstemmed | Replication, Communication, and the Population Dynamics of Scientific Discovery. |
| title_short | Replication, Communication, and the Population Dynamics of Scientific Discovery. |
| title_sort | replication communication and the population dynamics of scientific discovery |
| url | https://doi.org/10.1371/journal.pone.0136088 |
| work_keys_str_mv | AT richardmcelreath replicationcommunicationandthepopulationdynamicsofscientificdiscovery AT paulesmaldino replicationcommunicationandthepopulationdynamicsofscientificdiscovery |