How the public, and scientists, perceive advancement of knowledge from conflicting study results
Science often advances through disagreement among scientists and the studies they produce. For members of the public, however, conflicting results from scientific studies may trigger a sense of uncertainty that in turn leads to a feeling that nothing new has been learned from those studies. In sever...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Cambridge University Press
2019-11-01
|
| Series: | Judgment and Decision Making |
| Subjects: | |
| Online Access: | http://journal.sjdm.org/19/191002b/jdm191002b.pdf |
| _version_ | 1827886955315068928 |
|---|---|
| author | Derek J. Koehler Gordon Pennycook |
| author_facet | Derek J. Koehler Gordon Pennycook |
| author_sort | Derek J. Koehler |
| collection | DOAJ |
| description | Science often
advances through disagreement among scientists and the studies they produce.
For members of the public, however, conflicting results from scientific studies
may trigger a sense of uncertainty that in turn leads to a feeling that nothing
new has been learned from those studies. In several scenario studies,
participants read about pairs of highly similar scientific studies with results
that either agreed or disagreed, and were asked, “When we take the results of
these two studies together, do we now know more, less, or the same as we did
before about (the study topic)?” We find that over half of participants do not
feel that “we know more” as the result of the two new studies when the second
study fails to replicate the first. When the two study results strongly
conflict (e.g., one finds a positive and the other a negative association
between two variables), a non-trivial proportion of participants actually say
that “we know less” than we did before. Such a sentiment arguably violates
normative principles of statistical and scientific inference positing that new
study findings can never reduce our level of knowledge (and that only
completely uninformative studies can leave our level of knowledge unchanged).
Drawing attention to possible moderating variables, or to sample size
considerations, did not influence people's perceptions of knowledge
advancement. Scientist members of the American Academy of Arts and Sciences,
when presented with the same scenarios, were less inclined to say that nothing
new is learned from conflicting study results. |
| first_indexed | 2024-03-12T20:10:22Z |
| format | Article |
| id | doaj.art-172e3b528da34e86a659537af854bda1 |
| institution | Directory Open Access Journal |
| issn | 1930-2975 |
| language | English |
| last_indexed | 2024-03-12T20:10:22Z |
| publishDate | 2019-11-01 |
| publisher | Cambridge University Press |
| record_format | Article |
| series | Judgment and Decision Making |
| spelling | doaj.art-172e3b528da34e86a659537af854bda12023-08-02T01:44:32ZengCambridge University PressJudgment and Decision Making1930-29752019-11-01146671682How the public, and scientists, perceive advancement of knowledge from conflicting study resultsDerek J. KoehlerGordon PennycookScience often advances through disagreement among scientists and the studies they produce. For members of the public, however, conflicting results from scientific studies may trigger a sense of uncertainty that in turn leads to a feeling that nothing new has been learned from those studies. In several scenario studies, participants read about pairs of highly similar scientific studies with results that either agreed or disagreed, and were asked, “When we take the results of these two studies together, do we now know more, less, or the same as we did before about (the study topic)?” We find that over half of participants do not feel that “we know more” as the result of the two new studies when the second study fails to replicate the first. When the two study results strongly conflict (e.g., one finds a positive and the other a negative association between two variables), a non-trivial proportion of participants actually say that “we know less” than we did before. Such a sentiment arguably violates normative principles of statistical and scientific inference positing that new study findings can never reduce our level of knowledge (and that only completely uninformative studies can leave our level of knowledge unchanged). Drawing attention to possible moderating variables, or to sample size considerations, did not influence people's perceptions of knowledge advancement. Scientist members of the American Academy of Arts and Sciences, when presented with the same scenarios, were less inclined to say that nothing new is learned from conflicting study results.http://journal.sjdm.org/19/191002b/jdm191002b.pdfknowledge science inference conflicting resultsnakeywords |
| spellingShingle | Derek J. Koehler Gordon Pennycook How the public, and scientists, perceive advancement of knowledge from conflicting study results Judgment and Decision Making knowledge science inference conflicting resultsnakeywords |
| title | How the public, and
scientists, perceive advancement of knowledge from conflicting study
results |
| title_full | How the public, and
scientists, perceive advancement of knowledge from conflicting study
results |
| title_fullStr | How the public, and
scientists, perceive advancement of knowledge from conflicting study
results |
| title_full_unstemmed | How the public, and
scientists, perceive advancement of knowledge from conflicting study
results |
| title_short | How the public, and
scientists, perceive advancement of knowledge from conflicting study
results |
| title_sort | how the public and scientists perceive advancement of knowledge from conflicting study results |
| topic | knowledge science inference conflicting resultsnakeywords |
| url | http://journal.sjdm.org/19/191002b/jdm191002b.pdf |
| work_keys_str_mv | AT derekjkoehler howthepublicandscientistsperceiveadvancementofknowledgefromconflictingstudyresults AT gordonpennycook howthepublicandscientistsperceiveadvancementofknowledgefromconflictingstudyresults |