How to Determine the Accuracy of an Alternative Diagnostic Test when It Is Actually Better than the Reference Tests: A Re-Evaluation of Diagnostic Tests for Scrub Typhus Using Bayesian LCMs.
The indirect immunofluorescence assay (IFA) is considered a reference test for scrub typhus. Recently, the Scrub Typhus Infection Criteria (STIC; a combination of culture, PCR assays and IFA IgM) were proposed as a reference standard for evaluating alternative diagnostic tests. Here, we use Bayesian...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2015-01-01
|
| Series: | PLoS ONE |
| Online Access: | http://europepmc.org/articles/PMC4449177?pdf=render |
| _version_ | 1818319836020736000 |
|---|---|
| author | Cherry Lim Daniel H Paris Stuart D Blacksell Achara Laongnualpanich Pacharee Kantipong Wirongrong Chierakul Vanaporn Wuthiekanun Nicholas P J Day Ben S Cooper Direk Limmathurotsakul |
| author_facet | Cherry Lim Daniel H Paris Stuart D Blacksell Achara Laongnualpanich Pacharee Kantipong Wirongrong Chierakul Vanaporn Wuthiekanun Nicholas P J Day Ben S Cooper Direk Limmathurotsakul |
| author_sort | Cherry Lim |
| collection | DOAJ |
| description | The indirect immunofluorescence assay (IFA) is considered a reference test for scrub typhus. Recently, the Scrub Typhus Infection Criteria (STIC; a combination of culture, PCR assays and IFA IgM) were proposed as a reference standard for evaluating alternative diagnostic tests. Here, we use Bayesian latent class models (LCMs) to estimate the true accuracy of each diagnostic test, and of STIC, for diagnosing scrub typhus.Data from 161 patients with undifferentiated fever were re-evaluated using Bayesian LCMs. Every patient was evaluated for the presence of an eschar, and tested with blood culture for Orientia tsutsugamushi, three different PCR assays, IFA IgM, and the Panbio IgM immunochromatographic test (ICT). True sensitivity and specificity of culture (24.4% and 100%), 56kDa PCR assay (56.8% and 98.4%), 47kDa PCR assay (63.2% and 96.1%), groEL PCR assay (71.4% and 93.0%), IFA IgM (70.0% and 83.8%), PanBio IgM ICT (72.8% and 96.8%), presence of eschar (42.7% and 98.9%) and STIC (90.5% and 82.5%) estimated by Bayesian LCM were considerably different from those obtained when using STIC as a reference standard. The IgM ICT had comparable sensitivity and significantly higher specificity compared to IFA (p=0.34 and p<0.001, respectively).The low specificity of STIC was caused by the low specificity of IFA IgM. Neither STIC nor IFA IgM can be used as reference standards against which to evaluate alternative diagnostic tests. Further evaluation of new diagnostic tests should be done with a carefully selected set of diagnostic tests and appropriate statistical models. |
| first_indexed | 2024-12-13T10:15:26Z |
| format | Article |
| id | doaj.art-90f820dc4b1f4587a3eb23a9386a59aa |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-13T10:15:26Z |
| publishDate | 2015-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-90f820dc4b1f4587a3eb23a9386a59aa2022-12-21T23:51:19ZengPublic Library of Science (PLoS)PLoS ONE1932-62032015-01-01105e011493010.1371/journal.pone.0114930How to Determine the Accuracy of an Alternative Diagnostic Test when It Is Actually Better than the Reference Tests: A Re-Evaluation of Diagnostic Tests for Scrub Typhus Using Bayesian LCMs.Cherry LimDaniel H ParisStuart D BlacksellAchara LaongnualpanichPacharee KantipongWirongrong ChierakulVanaporn WuthiekanunNicholas P J DayBen S CooperDirek LimmathurotsakulThe indirect immunofluorescence assay (IFA) is considered a reference test for scrub typhus. Recently, the Scrub Typhus Infection Criteria (STIC; a combination of culture, PCR assays and IFA IgM) were proposed as a reference standard for evaluating alternative diagnostic tests. Here, we use Bayesian latent class models (LCMs) to estimate the true accuracy of each diagnostic test, and of STIC, for diagnosing scrub typhus.Data from 161 patients with undifferentiated fever were re-evaluated using Bayesian LCMs. Every patient was evaluated for the presence of an eschar, and tested with blood culture for Orientia tsutsugamushi, three different PCR assays, IFA IgM, and the Panbio IgM immunochromatographic test (ICT). True sensitivity and specificity of culture (24.4% and 100%), 56kDa PCR assay (56.8% and 98.4%), 47kDa PCR assay (63.2% and 96.1%), groEL PCR assay (71.4% and 93.0%), IFA IgM (70.0% and 83.8%), PanBio IgM ICT (72.8% and 96.8%), presence of eschar (42.7% and 98.9%) and STIC (90.5% and 82.5%) estimated by Bayesian LCM were considerably different from those obtained when using STIC as a reference standard. The IgM ICT had comparable sensitivity and significantly higher specificity compared to IFA (p=0.34 and p<0.001, respectively).The low specificity of STIC was caused by the low specificity of IFA IgM. Neither STIC nor IFA IgM can be used as reference standards against which to evaluate alternative diagnostic tests. Further evaluation of new diagnostic tests should be done with a carefully selected set of diagnostic tests and appropriate statistical models.http://europepmc.org/articles/PMC4449177?pdf=render |
| spellingShingle | Cherry Lim Daniel H Paris Stuart D Blacksell Achara Laongnualpanich Pacharee Kantipong Wirongrong Chierakul Vanaporn Wuthiekanun Nicholas P J Day Ben S Cooper Direk Limmathurotsakul How to Determine the Accuracy of an Alternative Diagnostic Test when It Is Actually Better than the Reference Tests: A Re-Evaluation of Diagnostic Tests for Scrub Typhus Using Bayesian LCMs. PLoS ONE |
| title | How to Determine the Accuracy of an Alternative Diagnostic Test when It Is Actually Better than the Reference Tests: A Re-Evaluation of Diagnostic Tests for Scrub Typhus Using Bayesian LCMs. |
| title_full | How to Determine the Accuracy of an Alternative Diagnostic Test when It Is Actually Better than the Reference Tests: A Re-Evaluation of Diagnostic Tests for Scrub Typhus Using Bayesian LCMs. |
| title_fullStr | How to Determine the Accuracy of an Alternative Diagnostic Test when It Is Actually Better than the Reference Tests: A Re-Evaluation of Diagnostic Tests for Scrub Typhus Using Bayesian LCMs. |
| title_full_unstemmed | How to Determine the Accuracy of an Alternative Diagnostic Test when It Is Actually Better than the Reference Tests: A Re-Evaluation of Diagnostic Tests for Scrub Typhus Using Bayesian LCMs. |
| title_short | How to Determine the Accuracy of an Alternative Diagnostic Test when It Is Actually Better than the Reference Tests: A Re-Evaluation of Diagnostic Tests for Scrub Typhus Using Bayesian LCMs. |
| title_sort | how to determine the accuracy of an alternative diagnostic test when it is actually better than the reference tests a re evaluation of diagnostic tests for scrub typhus using bayesian lcms |
| url | http://europepmc.org/articles/PMC4449177?pdf=render |
| work_keys_str_mv | AT cherrylim howtodeterminetheaccuracyofanalternativediagnostictestwhenitisactuallybetterthanthereferencetestsareevaluationofdiagnostictestsforscrubtyphususingbayesianlcms AT danielhparis howtodeterminetheaccuracyofanalternativediagnostictestwhenitisactuallybetterthanthereferencetestsareevaluationofdiagnostictestsforscrubtyphususingbayesianlcms AT stuartdblacksell howtodeterminetheaccuracyofanalternativediagnostictestwhenitisactuallybetterthanthereferencetestsareevaluationofdiagnostictestsforscrubtyphususingbayesianlcms AT acharalaongnualpanich howtodeterminetheaccuracyofanalternativediagnostictestwhenitisactuallybetterthanthereferencetestsareevaluationofdiagnostictestsforscrubtyphususingbayesianlcms AT pachareekantipong howtodeterminetheaccuracyofanalternativediagnostictestwhenitisactuallybetterthanthereferencetestsareevaluationofdiagnostictestsforscrubtyphususingbayesianlcms AT wirongrongchierakul howtodeterminetheaccuracyofanalternativediagnostictestwhenitisactuallybetterthanthereferencetestsareevaluationofdiagnostictestsforscrubtyphususingbayesianlcms AT vanapornwuthiekanun howtodeterminetheaccuracyofanalternativediagnostictestwhenitisactuallybetterthanthereferencetestsareevaluationofdiagnostictestsforscrubtyphususingbayesianlcms AT nicholaspjday howtodeterminetheaccuracyofanalternativediagnostictestwhenitisactuallybetterthanthereferencetestsareevaluationofdiagnostictestsforscrubtyphususingbayesianlcms AT benscooper howtodeterminetheaccuracyofanalternativediagnostictestwhenitisactuallybetterthanthereferencetestsareevaluationofdiagnostictestsforscrubtyphususingbayesianlcms AT direklimmathurotsakul howtodeterminetheaccuracyofanalternativediagnostictestwhenitisactuallybetterthanthereferencetestsareevaluationofdiagnostictestsforscrubtyphususingbayesianlcms |