Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements.
Professional fact-checkers and fact-checking organizations provide a critical public service. Skeptics of modern media, however, often question the accuracy and objectivity of fact-checkers. The current study assessed agreement among two independent fact-checkers, The Washington Post and PolitiFact,...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2023-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0289004 |
| _version_ | 1797650682148290560 |
|---|---|
| author | David M Markowitz Timothy R Levine Kim B Serota Alivia D Moore |
| author_facet | David M Markowitz Timothy R Levine Kim B Serota Alivia D Moore |
| author_sort | David M Markowitz |
| collection | DOAJ |
| description | Professional fact-checkers and fact-checking organizations provide a critical public service. Skeptics of modern media, however, often question the accuracy and objectivity of fact-checkers. The current study assessed agreement among two independent fact-checkers, The Washington Post and PolitiFact, regarding the false and misleading statements of then President Donald J. Trump. Differences in statement selection and deceptiveness scaling were investigated. The Washington Post checked PolitiFact fact-checks 77.4% of the time (22.6% selection disagreement). Moderate agreement was observed for deceptiveness scaling. Nearly complete agreement was observed for bottom-line attributed veracity. Additional cross-checking with other sources (Snopes, FactCheck.org), original sources, and with fact-checking for the first 100 days of President Joe Biden's administration were inconsistent with potential ideology effects. Our evidence suggests fact-checking is a difficult enterprise, there is considerable variability between fact-checkers in the raw number of statements that are checked, and finally, selection and scaling account for apparent discrepancies among fact-checkers. |
| first_indexed | 2024-03-11T16:03:53Z |
| format | Article |
| id | doaj.art-3e79d8d9749d4d27ab10c68da2a86483 |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-03-11T16:03:53Z |
| publishDate | 2023-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-3e79d8d9749d4d27ab10c68da2a864832023-10-25T05:31:16ZengPublic Library of Science (PLoS)PLoS ONE1932-62032023-01-01187e028900410.1371/journal.pone.0289004Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements.David M MarkowitzTimothy R LevineKim B SerotaAlivia D MooreProfessional fact-checkers and fact-checking organizations provide a critical public service. Skeptics of modern media, however, often question the accuracy and objectivity of fact-checkers. The current study assessed agreement among two independent fact-checkers, The Washington Post and PolitiFact, regarding the false and misleading statements of then President Donald J. Trump. Differences in statement selection and deceptiveness scaling were investigated. The Washington Post checked PolitiFact fact-checks 77.4% of the time (22.6% selection disagreement). Moderate agreement was observed for deceptiveness scaling. Nearly complete agreement was observed for bottom-line attributed veracity. Additional cross-checking with other sources (Snopes, FactCheck.org), original sources, and with fact-checking for the first 100 days of President Joe Biden's administration were inconsistent with potential ideology effects. Our evidence suggests fact-checking is a difficult enterprise, there is considerable variability between fact-checkers in the raw number of statements that are checked, and finally, selection and scaling account for apparent discrepancies among fact-checkers.https://doi.org/10.1371/journal.pone.0289004 |
| spellingShingle | David M Markowitz Timothy R Levine Kim B Serota Alivia D Moore Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements. PLoS ONE |
| title | Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements. |
| title_full | Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements. |
| title_fullStr | Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements. |
| title_full_unstemmed | Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements. |
| title_short | Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements. |
| title_sort | cross checking journalistic fact checkers the role of sampling and scaling in interpreting false and misleading statements |
| url | https://doi.org/10.1371/journal.pone.0289004 |
| work_keys_str_mv | AT davidmmarkowitz crosscheckingjournalisticfactcheckerstheroleofsamplingandscalingininterpretingfalseandmisleadingstatements AT timothyrlevine crosscheckingjournalisticfactcheckerstheroleofsamplingandscalingininterpretingfalseandmisleadingstatements AT kimbserota crosscheckingjournalisticfactcheckerstheroleofsamplingandscalingininterpretingfalseandmisleadingstatements AT aliviadmoore crosscheckingjournalisticfactcheckerstheroleofsamplingandscalingininterpretingfalseandmisleadingstatements |