An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study
BackgroundEarly detection of mild cognitive impairment (MCI) symptoms is an important step to its diagnosis and intervention. We developed a new screening test called “Efficient Online MCI Screening System” (EOmciSS) for use in community-dwelling older adults. It is a self-pa...
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2023-01-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2023/1/e40858 |
| _version_ | 1797734467282927616 |
|---|---|
| author | Jingsong Wu Jingnan Tu Zhizhen Liu Lei Cao Youze He Jia Huang Jing Tao Mabel N K Wong Lidian Chen Tatia M C Lee Chetwyn C H Chan |
| author_facet | Jingsong Wu Jingnan Tu Zhizhen Liu Lei Cao Youze He Jia Huang Jing Tao Mabel N K Wong Lidian Chen Tatia M C Lee Chetwyn C H Chan |
| author_sort | Jingsong Wu |
| collection | DOAJ |
| description |
BackgroundEarly detection of mild cognitive impairment (MCI) symptoms is an important step to its diagnosis and intervention. We developed a new screening test called “Efficient Online MCI Screening System” (EOmciSS) for use in community-dwelling older adults. It is a self-paced cognitive test to be completed within 10 minutes on tablets or smartphones in homes or care centers for older adults.
ObjectiveThis study aims to test the validity of EOmciSS for identifying community-dwelling older adults with MCI risks.
MethodsParticipants (N=827) completed EOmciSS and other screening tests for MCI. The psychometric properties tested were “subscale item difficulty,” “discriminative index,” “internal consistency,” and “construct validity.” We also tested between-group discrimination using the cross-validation method in an MCI group and a normal cognitive function (NCF) group.
ResultsA total of 3 accuracy factors and 1 reaction time factor explained the structure of the 20 item factors. The difficulty level of accuracy factors (ie, “trail making,” “clock drawing,” “cube copying,” “delayed recall”) was 0.63-0.99, whereas that of the reaction time factor was 0.77-0.95. The discriminative index of the medium-to-high-difficulty item factors was 0.39-0.97. The internal consistency (Cronbach α) ranged from .41 (for few item factors) to .96. The training data set contained 9 item factors (CC-Acc1, P<.001; CD-Acc1, P=.07; CD-Acc2, P=.06; CD-Acc3, P<.001; TM-Acc4, P=.07; DR-Acc1, P=.03; RS, P=.06; DR-RT1, P=.02; and DR-RT2, P=.05) that were significant predictors for an MCI classification versus NCF classification. Depressive symptoms were identified as significant factors (P<.001) influencing the performance of participants, and were an integral part of our test system. Age (P=.15), number of years of education (P=.18), and proficiency in using an electronic device (P=.39) did not significantly influence the scores nor classification of participants. Application of the MCI/NCF cutoff score (7.90 out of 9.67) to the validation data set yielded an area under the curve of 0.912 (P<.001; 95% CI 0.868-0.955). The sensitivity was 84.9%, specificity was 85.1%, and the Youden index was 0.70.
ConclusionsEOmciSS was valid and reliable for identifying older adults with significant risks of MCI. Our results indicate that EOmciSS has higher sensitivity and specificity than those of the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment and the Computerized Cognitive Screen. The user interface, online operation, and self-paced format allowed the test system to be operated by older adults or their caregivers in different settings (eg, home or care centers for older adults). Depressive symptoms should be an integral part in future MCI screening systems because they influence the test performance and, hence, MCI risk.
Trial RegistrationChinese Clinical Trial Registry ChiCTR2000039411; http://www.chictr.org.cn/showprojen.aspx?proj=62903 |
| first_indexed | 2024-03-12T12:44:38Z |
| format | Article |
| id | doaj.art-939a8a852acd4ec2bd7db19ec10fadf5 |
| institution | Directory Open Access Journal |
| issn | 1438-8871 |
| language | English |
| last_indexed | 2024-03-12T12:44:38Z |
| publishDate | 2023-01-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj.art-939a8a852acd4ec2bd7db19ec10fadf52023-08-28T23:31:50ZengJMIR PublicationsJournal of Medical Internet Research1438-88712023-01-0125e4085810.2196/40858An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation StudyJingsong Wuhttps://orcid.org/0000-0001-6544-2553Jingnan Tuhttps://orcid.org/0000-0002-3668-7073Zhizhen Liuhttps://orcid.org/0000-0002-4594-0325Lei Caohttps://orcid.org/0000-0002-2285-1629Youze Hehttps://orcid.org/0000-0003-3917-983XJia Huanghttps://orcid.org/0000-0002-0365-9164Jing Taohttps://orcid.org/0000-0001-9751-5294Mabel N K Wonghttps://orcid.org/0000-0002-8160-2921Lidian Chenhttps://orcid.org/0000-0002-6454-7932Tatia M C Leehttps://orcid.org/0000-0002-9992-6609Chetwyn C H Chanhttps://orcid.org/0000-0003-0307-6337 BackgroundEarly detection of mild cognitive impairment (MCI) symptoms is an important step to its diagnosis and intervention. We developed a new screening test called “Efficient Online MCI Screening System” (EOmciSS) for use in community-dwelling older adults. It is a self-paced cognitive test to be completed within 10 minutes on tablets or smartphones in homes or care centers for older adults. ObjectiveThis study aims to test the validity of EOmciSS for identifying community-dwelling older adults with MCI risks. MethodsParticipants (N=827) completed EOmciSS and other screening tests for MCI. The psychometric properties tested were “subscale item difficulty,” “discriminative index,” “internal consistency,” and “construct validity.” We also tested between-group discrimination using the cross-validation method in an MCI group and a normal cognitive function (NCF) group. ResultsA total of 3 accuracy factors and 1 reaction time factor explained the structure of the 20 item factors. The difficulty level of accuracy factors (ie, “trail making,” “clock drawing,” “cube copying,” “delayed recall”) was 0.63-0.99, whereas that of the reaction time factor was 0.77-0.95. The discriminative index of the medium-to-high-difficulty item factors was 0.39-0.97. The internal consistency (Cronbach α) ranged from .41 (for few item factors) to .96. The training data set contained 9 item factors (CC-Acc1, P<.001; CD-Acc1, P=.07; CD-Acc2, P=.06; CD-Acc3, P<.001; TM-Acc4, P=.07; DR-Acc1, P=.03; RS, P=.06; DR-RT1, P=.02; and DR-RT2, P=.05) that were significant predictors for an MCI classification versus NCF classification. Depressive symptoms were identified as significant factors (P<.001) influencing the performance of participants, and were an integral part of our test system. Age (P=.15), number of years of education (P=.18), and proficiency in using an electronic device (P=.39) did not significantly influence the scores nor classification of participants. Application of the MCI/NCF cutoff score (7.90 out of 9.67) to the validation data set yielded an area under the curve of 0.912 (P<.001; 95% CI 0.868-0.955). The sensitivity was 84.9%, specificity was 85.1%, and the Youden index was 0.70. ConclusionsEOmciSS was valid and reliable for identifying older adults with significant risks of MCI. Our results indicate that EOmciSS has higher sensitivity and specificity than those of the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment and the Computerized Cognitive Screen. The user interface, online operation, and self-paced format allowed the test system to be operated by older adults or their caregivers in different settings (eg, home or care centers for older adults). Depressive symptoms should be an integral part in future MCI screening systems because they influence the test performance and, hence, MCI risk. Trial RegistrationChinese Clinical Trial Registry ChiCTR2000039411; http://www.chictr.org.cn/showprojen.aspx?proj=62903https://www.jmir.org/2023/1/e40858 |
| spellingShingle | Jingsong Wu Jingnan Tu Zhizhen Liu Lei Cao Youze He Jia Huang Jing Tao Mabel N K Wong Lidian Chen Tatia M C Lee Chetwyn C H Chan An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study Journal of Medical Internet Research |
| title | An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study |
| title_full | An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study |
| title_fullStr | An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study |
| title_full_unstemmed | An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study |
| title_short | An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study |
| title_sort | effective test eomciss for screening older adults with mild cognitive impairment in a community setting development and validation study |
| url | https://www.jmir.org/2023/1/e40858 |
| work_keys_str_mv | AT jingsongwu aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT jingnantu aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT zhizhenliu aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT leicao aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT youzehe aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT jiahuang aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT jingtao aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT mabelnkwong aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT lidianchen aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT tatiamclee aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT chetwynchchan aneffectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT jingsongwu effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT jingnantu effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT zhizhenliu effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT leicao effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT youzehe effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT jiahuang effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT jingtao effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT mabelnkwong effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT lidianchen effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT tatiamclee effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy AT chetwynchchan effectivetesteomcissforscreeningolderadultswithmildcognitiveimpairmentinacommunitysettingdevelopmentandvalidationstudy |