An Extension of the Bland–Altman Plot for Analyzing the Agreement of More than Two Raters

The Bland–Altman plot is the most common method to analyze and visualize agreement between raters or methods of quantitative outcomes in health research. While very useful for studies with two raters, a limitation of the classical Bland–Altman plot is that it is specifically used for studies with tw...

Full description

Bibliographic Details
Main Authors: Sören Möller, Birgit Debrabant, Ulrich Halekoh, Andreas Kristian Petersen, Oke Gerke
Format: Article
Language:English
Published: MDPI AG 2021-01-01
Series:Diagnostics
Subjects:
Online Access:https://www.mdpi.com/2075-4418/11/1/54
Description
Summary:The Bland–Altman plot is the most common method to analyze and visualize agreement between raters or methods of quantitative outcomes in health research. While very useful for studies with two raters, a limitation of the classical Bland–Altman plot is that it is specifically used for studies with two raters. We propose an extension of the Bland–Altman plot suitable for more than two raters and derive the approximate limits of agreement with <inline-formula><math display="inline"><semantics><mrow><mn>95</mn><mo>%</mo></mrow></semantics></math></inline-formula> confidence intervals. We validated the suggested limit of agreement by a simulation study. Moreover, we offer suggestions on how to present bias, heterogeneity among raters, as well as the uncertainty of the limits of agreement. The resulting plot could be utilized to investigate and present agreement in studies with more than two raters.
ISSN:2075-4418