The clinical meaning of the area under a receiver operating characteristic curve for the evaluation of the performance of disease markers
OBJECTIVES The area under a receiver operating characteristic (ROC) curve (AUC) is a popular measure of pure diagnostic accuracy that is independent from the proportion of diseased subjects in the analysed sample. However, its actual usefulness in the clinical context has been questioned, because it...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Korean Society of Epidemiology
2022-10-01
|
| Series: | Epidemiology and Health |
| Subjects: | |
| Online Access: | http://www.e-epih.org/upload/epih-44-e2022088.pdf |
| _version_ | 1797768377569116160 |
|---|---|
| author | Stefano Parodi Damiano Verda Francesca Bagnasco Marco Muselli |
| author_facet | Stefano Parodi Damiano Verda Francesca Bagnasco Marco Muselli |
| author_sort | Stefano Parodi |
| collection | DOAJ |
| description | OBJECTIVES The area under a receiver operating characteristic (ROC) curve (AUC) is a popular measure of pure diagnostic accuracy that is independent from the proportion of diseased subjects in the analysed sample. However, its actual usefulness in the clinical context has been questioned, because it does not seem to be directly related to the actual performance of a diagnostic marker in identifying diseased and non-diseased subjects in real clinical settings. This study evaluates the relationship between the AUC and the proportion of correct classifications (global diagnostic accuracy, GDA) in relation to the shape of the corresponding ROC curves. METHODS We demonstrate that AUC represents an upward-biased measure of GDA at an optimal accuracy cut-off for balanced groups. The magnitude of bias depends on the shape of the ROC plot and on the proportion of diseased and non-diseased subjects. In proper curves, the bias is independent from the diseased/non-diseased ratio and can be easily estimated and removed. Moreover, a comparison between 2 partial AUCs can be replaced by a more powerful test for the corresponding whole AUCs. RESULTS Applications to 3 real datasets are provided: a marker for a hormone deficit in children, 2 tumour markers for malignant mesothelioma, and 2 gene expression profiles in ovarian cancer patients. CONCLUSIONS The AUC is a measure of accuracy with potential clinical relevance for the evaluation of disease markers. The clinical meaning of ROC parameters should always be evaluated with an analysis of the shape of the corresponding ROC curve. |
| first_indexed | 2024-03-12T20:53:37Z |
| format | Article |
| id | doaj.art-e82f12f8150049dca2ce179b4ed19bb5 |
| institution | Directory Open Access Journal |
| issn | 2092-7193 |
| language | English |
| last_indexed | 2024-03-12T20:53:37Z |
| publishDate | 2022-10-01 |
| publisher | Korean Society of Epidemiology |
| record_format | Article |
| series | Epidemiology and Health |
| spelling | doaj.art-e82f12f8150049dca2ce179b4ed19bb52023-08-01T00:41:57ZengKorean Society of EpidemiologyEpidemiology and Health2092-71932022-10-014410.4178/epih.e20220881336The clinical meaning of the area under a receiver operating characteristic curve for the evaluation of the performance of disease markersStefano ParodiDamiano VerdaFrancesca BagnascoMarco MuselliOBJECTIVES The area under a receiver operating characteristic (ROC) curve (AUC) is a popular measure of pure diagnostic accuracy that is independent from the proportion of diseased subjects in the analysed sample. However, its actual usefulness in the clinical context has been questioned, because it does not seem to be directly related to the actual performance of a diagnostic marker in identifying diseased and non-diseased subjects in real clinical settings. This study evaluates the relationship between the AUC and the proportion of correct classifications (global diagnostic accuracy, GDA) in relation to the shape of the corresponding ROC curves. METHODS We demonstrate that AUC represents an upward-biased measure of GDA at an optimal accuracy cut-off for balanced groups. The magnitude of bias depends on the shape of the ROC plot and on the proportion of diseased and non-diseased subjects. In proper curves, the bias is independent from the diseased/non-diseased ratio and can be easily estimated and removed. Moreover, a comparison between 2 partial AUCs can be replaced by a more powerful test for the corresponding whole AUCs. RESULTS Applications to 3 real datasets are provided: a marker for a hormone deficit in children, 2 tumour markers for malignant mesothelioma, and 2 gene expression profiles in ovarian cancer patients. CONCLUSIONS The AUC is a measure of accuracy with potential clinical relevance for the evaluation of disease markers. The clinical meaning of ROC parameters should always be evaluated with an analysis of the shape of the corresponding ROC curve.http://www.e-epih.org/upload/epih-44-e2022088.pdfreceiver operating characteristic curvebiomarkersdiagnostic accuracybinormal modelmetz and kronman test |
| spellingShingle | Stefano Parodi Damiano Verda Francesca Bagnasco Marco Muselli The clinical meaning of the area under a receiver operating characteristic curve for the evaluation of the performance of disease markers Epidemiology and Health receiver operating characteristic curve biomarkers diagnostic accuracy binormal model metz and kronman test |
| title | The clinical meaning of the area under a receiver operating characteristic curve for the evaluation of the performance of disease markers |
| title_full | The clinical meaning of the area under a receiver operating characteristic curve for the evaluation of the performance of disease markers |
| title_fullStr | The clinical meaning of the area under a receiver operating characteristic curve for the evaluation of the performance of disease markers |
| title_full_unstemmed | The clinical meaning of the area under a receiver operating characteristic curve for the evaluation of the performance of disease markers |
| title_short | The clinical meaning of the area under a receiver operating characteristic curve for the evaluation of the performance of disease markers |
| title_sort | clinical meaning of the area under a receiver operating characteristic curve for the evaluation of the performance of disease markers |
| topic | receiver operating characteristic curve biomarkers diagnostic accuracy binormal model metz and kronman test |
| url | http://www.e-epih.org/upload/epih-44-e2022088.pdf |
| work_keys_str_mv | AT stefanoparodi theclinicalmeaningoftheareaunderareceiveroperatingcharacteristiccurvefortheevaluationoftheperformanceofdiseasemarkers AT damianoverda theclinicalmeaningoftheareaunderareceiveroperatingcharacteristiccurvefortheevaluationoftheperformanceofdiseasemarkers AT francescabagnasco theclinicalmeaningoftheareaunderareceiveroperatingcharacteristiccurvefortheevaluationoftheperformanceofdiseasemarkers AT marcomuselli theclinicalmeaningoftheareaunderareceiveroperatingcharacteristiccurvefortheevaluationoftheperformanceofdiseasemarkers AT stefanoparodi clinicalmeaningoftheareaunderareceiveroperatingcharacteristiccurvefortheevaluationoftheperformanceofdiseasemarkers AT damianoverda clinicalmeaningoftheareaunderareceiveroperatingcharacteristiccurvefortheevaluationoftheperformanceofdiseasemarkers AT francescabagnasco clinicalmeaningoftheareaunderareceiveroperatingcharacteristiccurvefortheevaluationoftheperformanceofdiseasemarkers AT marcomuselli clinicalmeaningoftheareaunderareceiveroperatingcharacteristiccurvefortheevaluationoftheperformanceofdiseasemarkers |