How Subtle Protocol Choices Can Affect Biological Conclusions: Great Tits' Response to Allopatric Mobbing Calls
In the last ten years, numerous replicated studies showed divergent results from the original papers, leading to the recognition that science may be facing a replication crisis. Apart from fraud or natural population variability, different results may emerge from flexibility in the protocol and/or r...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Animal Behavior and Cognition
2021-05-01
|
| Series: | Animal Behavior and Cognition |
| Subjects: | |
| Online Access: | http://animalbehaviorandcognition.org/article.php?id=1260 |
| _version_ | 1818642098305368064 |
|---|---|
| author | Ambre Salis Jean-Paul Lena Thierry Lengagne |
| author_facet | Ambre Salis Jean-Paul Lena Thierry Lengagne |
| author_sort | Ambre Salis |
| collection | DOAJ |
| description | In the last ten years, numerous replicated studies showed divergent results from the original papers, leading to the recognition that science may be facing a replication crisis. Apart from fraud or natural population variability, different results may emerge from flexibility in the protocol and/or restricted sample sizes. Replication studies are therefore fundamental to assess how robust a finding can be. However, while the original authors may be prone to p-hacking (to collect data, select data or use statistical analyses until nonsignificant results become significant), the replication-authors are, on the contrary, probably unwittingly prone to show opposite results (i.e., null-hacking). In this study, we face the unique opportunity to compare replicated studies with no null-hacking bias. Two teams independently investigated the response of great tits (Parus major) to mobbing calls of an allopatric species, in their natural and reversed order, on the same population. We first discuss how subtle protocol choices, especially regarding the soundtrack preparation and playback methodology, can explain variation in the results. In addition, we show that, although the effect sizes of the differences of interest are similar, biological conclusions solely based on the p-value would be different. Thus, we note the pitfall of reliance on p values, especially with small samples. |
| first_indexed | 2024-12-16T23:37:39Z |
| format | Article |
| id | doaj.art-2c6b26c8f59e4ed2bfe26826633b1ea8 |
| institution | Directory Open Access Journal |
| issn | 2372-5052 2372-4323 |
| language | English |
| last_indexed | 2024-12-16T23:37:39Z |
| publishDate | 2021-05-01 |
| publisher | Animal Behavior and Cognition |
| record_format | Article |
| series | Animal Behavior and Cognition |
| spelling | doaj.art-2c6b26c8f59e4ed2bfe26826633b1ea82022-12-21T22:11:43ZengAnimal Behavior and CognitionAnimal Behavior and Cognition2372-50522372-43232021-05-018215216510.26451/abc.08.02.05.2021How Subtle Protocol Choices Can Affect Biological Conclusions: Great Tits' Response to Allopatric Mobbing CallsAmbre SalisJean-Paul LenaThierry LengagneIn the last ten years, numerous replicated studies showed divergent results from the original papers, leading to the recognition that science may be facing a replication crisis. Apart from fraud or natural population variability, different results may emerge from flexibility in the protocol and/or restricted sample sizes. Replication studies are therefore fundamental to assess how robust a finding can be. However, while the original authors may be prone to p-hacking (to collect data, select data or use statistical analyses until nonsignificant results become significant), the replication-authors are, on the contrary, probably unwittingly prone to show opposite results (i.e., null-hacking). In this study, we face the unique opportunity to compare replicated studies with no null-hacking bias. Two teams independently investigated the response of great tits (Parus major) to mobbing calls of an allopatric species, in their natural and reversed order, on the same population. We first discuss how subtle protocol choices, especially regarding the soundtrack preparation and playback methodology, can explain variation in the results. In addition, we show that, although the effect sizes of the differences of interest are similar, biological conclusions solely based on the p-value would be different. Thus, we note the pitfall of reliance on p values, especially with small samples.http://animalbehaviorandcognition.org/article.php?id=1260animal communicationheterospecific communicationmobbingreplication crisissyntax |
| spellingShingle | Ambre Salis Jean-Paul Lena Thierry Lengagne How Subtle Protocol Choices Can Affect Biological Conclusions: Great Tits' Response to Allopatric Mobbing Calls Animal Behavior and Cognition animal communication heterospecific communication mobbing replication crisis syntax |
| title | How Subtle Protocol Choices Can Affect Biological Conclusions: Great Tits' Response to Allopatric Mobbing Calls |
| title_full | How Subtle Protocol Choices Can Affect Biological Conclusions: Great Tits' Response to Allopatric Mobbing Calls |
| title_fullStr | How Subtle Protocol Choices Can Affect Biological Conclusions: Great Tits' Response to Allopatric Mobbing Calls |
| title_full_unstemmed | How Subtle Protocol Choices Can Affect Biological Conclusions: Great Tits' Response to Allopatric Mobbing Calls |
| title_short | How Subtle Protocol Choices Can Affect Biological Conclusions: Great Tits' Response to Allopatric Mobbing Calls |
| title_sort | how subtle protocol choices can affect biological conclusions great tits response to allopatric mobbing calls |
| topic | animal communication heterospecific communication mobbing replication crisis syntax |
| url | http://animalbehaviorandcognition.org/article.php?id=1260 |
| work_keys_str_mv | AT ambresalis howsubtleprotocolchoicescanaffectbiologicalconclusionsgreattitsresponsetoallopatricmobbingcalls AT jeanpaullena howsubtleprotocolchoicescanaffectbiologicalconclusionsgreattitsresponsetoallopatricmobbingcalls AT thierrylengagne howsubtleprotocolchoicescanaffectbiologicalconclusionsgreattitsresponsetoallopatricmobbingcalls |