Effect of confidence interval construction on judgment accuracy
Three experiments (N = 550) examined the effect of an interval construction elicitation method used in several expert elicitation studies on judgment accuracy. Participants made judgments about topics that were either searchable or unsearchable online using one of two order variations of the interva...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Cambridge University Press
2020-09-01
|
| Series: | Judgment and Decision Making |
| Subjects: | |
| Online Access: | https://www.cambridge.org/core/product/identifier/S1930297500007920/type/journal_article |
| _version_ | 1797702541355515904 |
|---|---|
| author | David R. Mandel Robert N. Collins Evan F. Risko Jonathan A. Fugelsang |
| author_facet | David R. Mandel Robert N. Collins Evan F. Risko Jonathan A. Fugelsang |
| author_sort | David R. Mandel |
| collection | DOAJ |
| description | Three experiments (N = 550) examined the effect of an interval construction elicitation method used in several expert elicitation studies on judgment accuracy. Participants made judgments about topics that were either searchable or unsearchable online using one of two order variations of the interval construction procedure. One group of participants provided their best judgment (one step) prior to constructing an interval (i.e., lower bound, upper bound, and a confidence rating that the correct value fell in the range provided), whereas another group of participants provided their best judgment last, after the three-step confidence interval was constructed. The overall effect of this elicitation method was not significant in 8 out of 9 univariate tests. Moreover, the calibration of confidence intervals was not affected by elicitation order. The findings warrant skepticism regarding the benefit of prior confidence interval construction for improving judgment accuracy. |
| first_indexed | 2024-03-12T04:51:32Z |
| format | Article |
| id | doaj.art-13cb2bd02e734a8490a4d2d9badd0926 |
| institution | Directory Open Access Journal |
| issn | 1930-2975 |
| language | English |
| last_indexed | 2024-03-12T04:51:32Z |
| publishDate | 2020-09-01 |
| publisher | Cambridge University Press |
| record_format | Article |
| series | Judgment and Decision Making |
| spelling | doaj.art-13cb2bd02e734a8490a4d2d9badd09262023-09-03T09:20:26ZengCambridge University PressJudgment and Decision Making1930-29752020-09-011578379710.1017/S1930297500007920Effect of confidence interval construction on judgment accuracyDavid R. Mandel0https://orcid.org/0000-0003-1036-2286Robert N. Collins1https://orcid.org/0000-0002-1714-7215Evan F. Risko2https://orcid.org/0000-0001-5702-0350Jonathan A. Fugelsang3https://orcid.org/0000-0002-6342-7023Intelligence, Influence and Collaboration Section, Defence Research and Development CanadaIntelligence, Influence and Collaboration Section, Defence Research and Development CanadaHumanSystems® IncorporatedHumanSystems® IncorporatedThree experiments (N = 550) examined the effect of an interval construction elicitation method used in several expert elicitation studies on judgment accuracy. Participants made judgments about topics that were either searchable or unsearchable online using one of two order variations of the interval construction procedure. One group of participants provided their best judgment (one step) prior to constructing an interval (i.e., lower bound, upper bound, and a confidence rating that the correct value fell in the range provided), whereas another group of participants provided their best judgment last, after the three-step confidence interval was constructed. The overall effect of this elicitation method was not significant in 8 out of 9 univariate tests. Moreover, the calibration of confidence intervals was not affected by elicitation order. The findings warrant skepticism regarding the benefit of prior confidence interval construction for improving judgment accuracy.https://www.cambridge.org/core/product/identifier/S1930297500007920/type/journal_articlejudgmentaccuracyconfidence intervalsorder effect |
| spellingShingle | David R. Mandel Robert N. Collins Evan F. Risko Jonathan A. Fugelsang Effect of confidence interval construction on judgment accuracy Judgment and Decision Making judgment accuracy confidence intervals order effect |
| title | Effect of confidence interval construction on judgment accuracy |
| title_full | Effect of confidence interval construction on judgment accuracy |
| title_fullStr | Effect of confidence interval construction on judgment accuracy |
| title_full_unstemmed | Effect of confidence interval construction on judgment accuracy |
| title_short | Effect of confidence interval construction on judgment accuracy |
| title_sort | effect of confidence interval construction on judgment accuracy |
| topic | judgment accuracy confidence intervals order effect |
| url | https://www.cambridge.org/core/product/identifier/S1930297500007920/type/journal_article |
| work_keys_str_mv | AT davidrmandel effectofconfidenceintervalconstructiononjudgmentaccuracy AT robertncollins effectofconfidenceintervalconstructiononjudgmentaccuracy AT evanfrisko effectofconfidenceintervalconstructiononjudgmentaccuracy AT jonathanafugelsang effectofconfidenceintervalconstructiononjudgmentaccuracy |