Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments
The heuristics-and-biases research program highlights reasons for expecting people to be poor intuitive forecasters. This article tests the power of a cognitive-debiasing training module (“CHAMPS KNOW”) to improve probability judgments in a four-year series of geopolitical forecasting tournaments sp...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Cambridge University Press
2016-09-01
|
| Series: | Judgment and Decision Making |
| Subjects: | |
| Online Access: | https://www.cambridge.org/core/product/identifier/S1930297500004599/type/journal_article |
| _version_ | 1797702555530166272 |
|---|---|
| author | Welton Chang Eva Chen Barbara Mellers Philip Tetlock |
| author_facet | Welton Chang Eva Chen Barbara Mellers Philip Tetlock |
| author_sort | Welton Chang |
| collection | DOAJ |
| description | The heuristics-and-biases research program highlights reasons for expecting people to be poor intuitive forecasters. This article tests the power of a cognitive-debiasing training module (“CHAMPS KNOW”) to improve probability judgments in a four-year series of geopolitical forecasting tournaments sponsored by the U.S. intelligence community. Although the training lasted less than one hour, it consistently improved accuracy (Brier scores) by 6 to 11% over the control condition. Cognitive ability and practice also made largely independent contributions to predictive accuracy. Given the brevity of the training tutorials and the heterogeneity of the problems posed, the observed effects are likely to be lower-bound estimates of what could be achieved by more intensive interventions. Future work should isolate which prongs of the multipronged CHAMPS KNOW training were most effective in improving judgment on which categories of problems. |
| first_indexed | 2024-03-12T04:52:47Z |
| format | Article |
| id | doaj.art-4a1c7d6e3d5d4ebbae1cdaad7c6dc1e2 |
| institution | Directory Open Access Journal |
| issn | 1930-2975 |
| language | English |
| last_indexed | 2024-03-12T04:52:47Z |
| publishDate | 2016-09-01 |
| publisher | Cambridge University Press |
| record_format | Article |
| series | Judgment and Decision Making |
| spelling | doaj.art-4a1c7d6e3d5d4ebbae1cdaad7c6dc1e22023-09-03T09:20:21ZengCambridge University PressJudgment and Decision Making1930-29752016-09-011150952610.1017/S1930297500004599Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournamentsWelton Chang0Eva Chen1Barbara Mellers2Philip Tetlock3Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104University of PennsylvaniaUniversity of PennsylvaniaUniversity of PennsylvaniaThe heuristics-and-biases research program highlights reasons for expecting people to be poor intuitive forecasters. This article tests the power of a cognitive-debiasing training module (“CHAMPS KNOW”) to improve probability judgments in a four-year series of geopolitical forecasting tournaments sponsored by the U.S. intelligence community. Although the training lasted less than one hour, it consistently improved accuracy (Brier scores) by 6 to 11% over the control condition. Cognitive ability and practice also made largely independent contributions to predictive accuracy. Given the brevity of the training tutorials and the heterogeneity of the problems posed, the observed effects are likely to be lower-bound estimates of what could be achieved by more intensive interventions. Future work should isolate which prongs of the multipronged CHAMPS KNOW training were most effective in improving judgment on which categories of problems.https://www.cambridge.org/core/product/identifier/S1930297500004599/type/journal_articleforecastingprobability judgmenttrainingpracticecognitive debiasing |
| spellingShingle | Welton Chang Eva Chen Barbara Mellers Philip Tetlock Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments Judgment and Decision Making forecasting probability judgment training practice cognitive debiasing |
| title | Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments |
| title_full | Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments |
| title_fullStr | Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments |
| title_full_unstemmed | Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments |
| title_short | Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments |
| title_sort | developing expert political judgment the impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments |
| topic | forecasting probability judgment training practice cognitive debiasing |
| url | https://www.cambridge.org/core/product/identifier/S1930297500004599/type/journal_article |
| work_keys_str_mv | AT weltonchang developingexpertpoliticaljudgmenttheimpactoftrainingandpracticeonjudgmentalaccuracyingeopoliticalforecastingtournaments AT evachen developingexpertpoliticaljudgmenttheimpactoftrainingandpracticeonjudgmentalaccuracyingeopoliticalforecastingtournaments AT barbaramellers developingexpertpoliticaljudgmenttheimpactoftrainingandpracticeonjudgmentalaccuracyingeopoliticalforecastingtournaments AT philiptetlock developingexpertpoliticaljudgmenttheimpactoftrainingandpracticeonjudgmentalaccuracyingeopoliticalforecastingtournaments |