Fool's gold: Why imperfect reference tests are undermining the evaluation of novel diagnostics: a reevaluation of 5 diagnostic tests for leptospirosis.
BACKGROUND: We observed that some patients with clinical leptospirosis supported by positive results of rapid tests were negative for leptospirosis on the basis of our diagnostic gold standard, which involves isolation of Leptospira species from blood culture and/or a positive result of a microscopi...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
2012
|
| _version_ | 1797099810287779840 |
|---|---|
| author | Limmathurotsakul, D Turner, E Wuthiekanun, V Thaipadungpanit, J Suputtamongkol, Y Chierakul, W Smythe, L Day, N Cooper, B Peacock, S |
| author_facet | Limmathurotsakul, D Turner, E Wuthiekanun, V Thaipadungpanit, J Suputtamongkol, Y Chierakul, W Smythe, L Day, N Cooper, B Peacock, S |
| author_sort | Limmathurotsakul, D |
| collection | OXFORD |
| description | BACKGROUND: We observed that some patients with clinical leptospirosis supported by positive results of rapid tests were negative for leptospirosis on the basis of our diagnostic gold standard, which involves isolation of Leptospira species from blood culture and/or a positive result of a microscopic agglutination test (MAT). We hypothesized that our reference standard was imperfect and used statistical modeling to investigate this hypothesis. METHODS: Data for 1652 patients with suspected leptospirosis recruited during three observational studies and one randomized control trial that described the application of culture, MAT, immunofluorescence assay (IFA), lateral flow (LF) and/or PCR targeting the 16S rRNA gene were reevaluated using Bayesian latent class models and random-effects meta-analysis. RESULTS: The estimated sensitivities of culture alone, MAT alone, and culture plus MAT (for which the result was considered positive if one or both tests had a positive result) were 10.5% (95% credible interval [CrI], 2.7%-27.5%), 49.8% (95% CrI, 37.6%-60.8%), and 55.5% (95% CrI, 42.9%-67.7%), respectively. These low sensitivities were present across all 4 studies. The estimated specificity of MAT alone (and of culture plus MAT) was 98.8% (95% CrI, 92.8%-100.0%). The estimated sensitivities and specificities of PCR (52.7% [95% CrI, 45.2%-60.6%] and 97.2% [95% CrI, 92.0%-99.8%], respectively), lateral flow test (85.6% [95% CrI, 77.5%-93.2%] and 96.2% [95% CrI, 87.7%-99.8%], respectively), and immunofluorescence assay (45.5% [95% CrI, 33.3%-60.9%] and 96.8% [95% CrI, 92.8%-99.8%], respectively) were considerably different from estimates in which culture plus MAT was considered a perfect gold standard test. CONCLUSIONS: Our findings show that culture plus MAT is an imperfect gold standard against which to compare alterative tests for the diagnosis of leptospirosis. Rapid point-of-care tests for this infection would bring an important improvement in patient care, but their future evaluation will require careful consideration of the reference test(s) used and the inclusion of appropriate statistical models. |
| first_indexed | 2024-03-07T05:28:53Z |
| format | Journal article |
| id | oxford-uuid:e184fbb4-0e76-47f9-b145-f4c769ef4854 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T05:28:53Z |
| publishDate | 2012 |
| record_format | dspace |
| spelling | oxford-uuid:e184fbb4-0e76-47f9-b145-f4c769ef48542022-03-27T09:55:04ZFool's gold: Why imperfect reference tests are undermining the evaluation of novel diagnostics: a reevaluation of 5 diagnostic tests for leptospirosis.Journal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:e184fbb4-0e76-47f9-b145-f4c769ef4854EnglishSymplectic Elements at Oxford2012Limmathurotsakul, DTurner, EWuthiekanun, VThaipadungpanit, JSuputtamongkol, YChierakul, WSmythe, LDay, NCooper, BPeacock, SBACKGROUND: We observed that some patients with clinical leptospirosis supported by positive results of rapid tests were negative for leptospirosis on the basis of our diagnostic gold standard, which involves isolation of Leptospira species from blood culture and/or a positive result of a microscopic agglutination test (MAT). We hypothesized that our reference standard was imperfect and used statistical modeling to investigate this hypothesis. METHODS: Data for 1652 patients with suspected leptospirosis recruited during three observational studies and one randomized control trial that described the application of culture, MAT, immunofluorescence assay (IFA), lateral flow (LF) and/or PCR targeting the 16S rRNA gene were reevaluated using Bayesian latent class models and random-effects meta-analysis. RESULTS: The estimated sensitivities of culture alone, MAT alone, and culture plus MAT (for which the result was considered positive if one or both tests had a positive result) were 10.5% (95% credible interval [CrI], 2.7%-27.5%), 49.8% (95% CrI, 37.6%-60.8%), and 55.5% (95% CrI, 42.9%-67.7%), respectively. These low sensitivities were present across all 4 studies. The estimated specificity of MAT alone (and of culture plus MAT) was 98.8% (95% CrI, 92.8%-100.0%). The estimated sensitivities and specificities of PCR (52.7% [95% CrI, 45.2%-60.6%] and 97.2% [95% CrI, 92.0%-99.8%], respectively), lateral flow test (85.6% [95% CrI, 77.5%-93.2%] and 96.2% [95% CrI, 87.7%-99.8%], respectively), and immunofluorescence assay (45.5% [95% CrI, 33.3%-60.9%] and 96.8% [95% CrI, 92.8%-99.8%], respectively) were considerably different from estimates in which culture plus MAT was considered a perfect gold standard test. CONCLUSIONS: Our findings show that culture plus MAT is an imperfect gold standard against which to compare alterative tests for the diagnosis of leptospirosis. Rapid point-of-care tests for this infection would bring an important improvement in patient care, but their future evaluation will require careful consideration of the reference test(s) used and the inclusion of appropriate statistical models. |
| spellingShingle | Limmathurotsakul, D Turner, E Wuthiekanun, V Thaipadungpanit, J Suputtamongkol, Y Chierakul, W Smythe, L Day, N Cooper, B Peacock, S Fool's gold: Why imperfect reference tests are undermining the evaluation of novel diagnostics: a reevaluation of 5 diagnostic tests for leptospirosis. |
| title | Fool's gold: Why imperfect reference tests are undermining the evaluation of novel diagnostics: a reevaluation of 5 diagnostic tests for leptospirosis. |
| title_full | Fool's gold: Why imperfect reference tests are undermining the evaluation of novel diagnostics: a reevaluation of 5 diagnostic tests for leptospirosis. |
| title_fullStr | Fool's gold: Why imperfect reference tests are undermining the evaluation of novel diagnostics: a reevaluation of 5 diagnostic tests for leptospirosis. |
| title_full_unstemmed | Fool's gold: Why imperfect reference tests are undermining the evaluation of novel diagnostics: a reevaluation of 5 diagnostic tests for leptospirosis. |
| title_short | Fool's gold: Why imperfect reference tests are undermining the evaluation of novel diagnostics: a reevaluation of 5 diagnostic tests for leptospirosis. |
| title_sort | fool s gold why imperfect reference tests are undermining the evaluation of novel diagnostics a reevaluation of 5 diagnostic tests for leptospirosis |
| work_keys_str_mv | AT limmathurotsakuld foolsgoldwhyimperfectreferencetestsareunderminingtheevaluationofnoveldiagnosticsareevaluationof5diagnostictestsforleptospirosis AT turnere foolsgoldwhyimperfectreferencetestsareunderminingtheevaluationofnoveldiagnosticsareevaluationof5diagnostictestsforleptospirosis AT wuthiekanunv foolsgoldwhyimperfectreferencetestsareunderminingtheevaluationofnoveldiagnosticsareevaluationof5diagnostictestsforleptospirosis AT thaipadungpanitj foolsgoldwhyimperfectreferencetestsareunderminingtheevaluationofnoveldiagnosticsareevaluationof5diagnostictestsforleptospirosis AT suputtamongkoly foolsgoldwhyimperfectreferencetestsareunderminingtheevaluationofnoveldiagnosticsareevaluationof5diagnostictestsforleptospirosis AT chierakulw foolsgoldwhyimperfectreferencetestsareunderminingtheevaluationofnoveldiagnosticsareevaluationof5diagnostictestsforleptospirosis AT smythel foolsgoldwhyimperfectreferencetestsareunderminingtheevaluationofnoveldiagnosticsareevaluationof5diagnostictestsforleptospirosis AT dayn foolsgoldwhyimperfectreferencetestsareunderminingtheevaluationofnoveldiagnosticsareevaluationof5diagnostictestsforleptospirosis AT cooperb foolsgoldwhyimperfectreferencetestsareunderminingtheevaluationofnoveldiagnosticsareevaluationof5diagnostictestsforleptospirosis AT peacocks foolsgoldwhyimperfectreferencetestsareunderminingtheevaluationofnoveldiagnosticsareevaluationof5diagnostictestsforleptospirosis |