An Extension of the Bland–Altman Plot for Analyzing the Agreement of More than Two Raters
The Bland–Altman plot is the most common method to analyze and visualize agreement between raters or methods of quantitative outcomes in health research. While very useful for studies with two raters, a limitation of the classical Bland–Altman plot is that it is specifically used for studies with tw...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-01-01
|
| Series: | Diagnostics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-4418/11/1/54 |
| _version_ | 1797542659252813824 |
|---|---|
| author | Sören Möller Birgit Debrabant Ulrich Halekoh Andreas Kristian Petersen Oke Gerke |
| author_facet | Sören Möller Birgit Debrabant Ulrich Halekoh Andreas Kristian Petersen Oke Gerke |
| author_sort | Sören Möller |
| collection | DOAJ |
| description | The Bland–Altman plot is the most common method to analyze and visualize agreement between raters or methods of quantitative outcomes in health research. While very useful for studies with two raters, a limitation of the classical Bland–Altman plot is that it is specifically used for studies with two raters. We propose an extension of the Bland–Altman plot suitable for more than two raters and derive the approximate limits of agreement with <inline-formula><math display="inline"><semantics><mrow><mn>95</mn><mo>%</mo></mrow></semantics></math></inline-formula> confidence intervals. We validated the suggested limit of agreement by a simulation study. Moreover, we offer suggestions on how to present bias, heterogeneity among raters, as well as the uncertainty of the limits of agreement. The resulting plot could be utilized to investigate and present agreement in studies with more than two raters. |
| first_indexed | 2024-03-10T13:33:41Z |
| format | Article |
| id | doaj.art-3a70dd86496d47a981002c9f3d3a493d |
| institution | Directory Open Access Journal |
| issn | 2075-4418 |
| language | English |
| last_indexed | 2024-03-10T13:33:41Z |
| publishDate | 2021-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Diagnostics |
| spelling | doaj.art-3a70dd86496d47a981002c9f3d3a493d2023-11-21T07:42:12ZengMDPI AGDiagnostics2075-44182021-01-011115410.3390/diagnostics11010054An Extension of the Bland–Altman Plot for Analyzing the Agreement of More than Two RatersSören Möller0Birgit Debrabant1Ulrich Halekoh2Andreas Kristian Petersen3Oke Gerke4Department of Clinical Research, University of Southern Denmark, 5000 Odense C, DenmarkDepartment of Public Health, Epidemiology, Biostatistics and Biodemography, University of Southern Denmark, 5000 Odense C, DenmarkDepartment of Public Health, Epidemiology, Biostatistics and Biodemography, University of Southern Denmark, 5000 Odense C, DenmarkDepartment of Research and Learning, Hospital of Southern Jutland, 6200 Aabenraa, DenmarkDepartment of Clinical Research, University of Southern Denmark, 5000 Odense C, DenmarkThe Bland–Altman plot is the most common method to analyze and visualize agreement between raters or methods of quantitative outcomes in health research. While very useful for studies with two raters, a limitation of the classical Bland–Altman plot is that it is specifically used for studies with two raters. We propose an extension of the Bland–Altman plot suitable for more than two raters and derive the approximate limits of agreement with <inline-formula><math display="inline"><semantics><mrow><mn>95</mn><mo>%</mo></mrow></semantics></math></inline-formula> confidence intervals. We validated the suggested limit of agreement by a simulation study. Moreover, we offer suggestions on how to present bias, heterogeneity among raters, as well as the uncertainty of the limits of agreement. The resulting plot could be utilized to investigate and present agreement in studies with more than two raters.https://www.mdpi.com/2075-4418/11/1/54Bland–Altman plotagreementvisualizationsimulation studymethod comparisoninter-rater |
| spellingShingle | Sören Möller Birgit Debrabant Ulrich Halekoh Andreas Kristian Petersen Oke Gerke An Extension of the Bland–Altman Plot for Analyzing the Agreement of More than Two Raters Diagnostics Bland–Altman plot agreement visualization simulation study method comparison inter-rater |
| title | An Extension of the Bland–Altman Plot for Analyzing the Agreement of More than Two Raters |
| title_full | An Extension of the Bland–Altman Plot for Analyzing the Agreement of More than Two Raters |
| title_fullStr | An Extension of the Bland–Altman Plot for Analyzing the Agreement of More than Two Raters |
| title_full_unstemmed | An Extension of the Bland–Altman Plot for Analyzing the Agreement of More than Two Raters |
| title_short | An Extension of the Bland–Altman Plot for Analyzing the Agreement of More than Two Raters |
| title_sort | extension of the bland altman plot for analyzing the agreement of more than two raters |
| topic | Bland–Altman plot agreement visualization simulation study method comparison inter-rater |
| url | https://www.mdpi.com/2075-4418/11/1/54 |
| work_keys_str_mv | AT sorenmoller anextensionoftheblandaltmanplotforanalyzingtheagreementofmorethantworaters AT birgitdebrabant anextensionoftheblandaltmanplotforanalyzingtheagreementofmorethantworaters AT ulrichhalekoh anextensionoftheblandaltmanplotforanalyzingtheagreementofmorethantworaters AT andreaskristianpetersen anextensionoftheblandaltmanplotforanalyzingtheagreementofmorethantworaters AT okegerke anextensionoftheblandaltmanplotforanalyzingtheagreementofmorethantworaters AT sorenmoller extensionoftheblandaltmanplotforanalyzingtheagreementofmorethantworaters AT birgitdebrabant extensionoftheblandaltmanplotforanalyzingtheagreementofmorethantworaters AT ulrichhalekoh extensionoftheblandaltmanplotforanalyzingtheagreementofmorethantworaters AT andreaskristianpetersen extensionoftheblandaltmanplotforanalyzingtheagreementofmorethantworaters AT okegerke extensionoftheblandaltmanplotforanalyzingtheagreementofmorethantworaters |