Episodix: a serious game to detect cognitive impairment in senior adults. A psychometric study
Introduction Assessment of episodic memory is traditionally used to evaluate potential cognitive impairments in senior adults. The present article discusses the capabilities of Episodix, a game to assess the aforementioned cognitive area, as a valid tool to discriminate among mild cognitive impairme...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2018-09-01
|
| Series: | PeerJ |
| Subjects: | |
| Online Access: | https://peerj.com/articles/5478.pdf |
| _version_ | 1797418921023766528 |
|---|---|
| author | Sonia Valladares-Rodriguez Manuel J. Fernández-Iglesias Luis Anido-Rifón David Facal Roberto Pérez-Rodríguez |
| author_facet | Sonia Valladares-Rodriguez Manuel J. Fernández-Iglesias Luis Anido-Rifón David Facal Roberto Pérez-Rodríguez |
| author_sort | Sonia Valladares-Rodriguez |
| collection | DOAJ |
| description | Introduction Assessment of episodic memory is traditionally used to evaluate potential cognitive impairments in senior adults. The present article discusses the capabilities of Episodix, a game to assess the aforementioned cognitive area, as a valid tool to discriminate among mild cognitive impairment (MCI), Alzheimer’s disease (AD) and healthy individuals (HC); that is, it studies the game’s psychometric validity study to assess cognitive impairment. Materials and Methods After a preliminary study, a new pilot study, statistically significant for the Galician population, was carried out from a cross-sectional sample of senior adults as target users. A total of 64 individuals (28 HC, 16 MCI, 20 AD) completed the experiment from an initial sample of 74. Participants were administered a collection of classical pen-and-paper tests and interacted with the games developed. A total of six machine learning classification techniques were applied and four relevant performance metrics were computed to assess the classification power of the tool according to participants’ cognitive status. Results According to the classification performance metrics computed, the best classification result is obtained using the Extra Trees Classifier (F1 = 0.97 and Cohen’s kappa coefficient = 0.97). Precision and recall values are also high, above 0.9 for all cognitive groups. Moreover, according to the standard interpretation of Cohen’s kappa index, classification is almost perfect (i.e., 0.81–1.00) for the complete dataset for all algorithms. Limitations Weaknesses (e.g., accessibility, sample size or speed of stimuli) detected during the preliminary study were addressed and solved. Nevertheless, additional research is needed to improve the resolution of the game for the identification of specific cognitive impairments, as well as to achieve a complete validation of the psychometric properties of the digital game. Conclusion Promising results obtained about psychometric validity of Episodix, represent a relevant step ahead towards the introduction of serious games and machine learning in regular clinical practice for detecting MCI or AD. However, more research is needed to explore the introduction of item response theory in this game and to obtain the required normative data for clinical validity. |
| first_indexed | 2024-03-09T06:41:00Z |
| format | Article |
| id | doaj.art-261b13ea900943669fc968c7ea14ffed |
| institution | Directory Open Access Journal |
| issn | 2167-8359 |
| language | English |
| last_indexed | 2024-03-09T06:41:00Z |
| publishDate | 2018-09-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ |
| spelling | doaj.art-261b13ea900943669fc968c7ea14ffed2023-12-03T10:50:52ZengPeerJ Inc.PeerJ2167-83592018-09-016e547810.7717/peerj.5478Episodix: a serious game to detect cognitive impairment in senior adults. A psychometric studySonia Valladares-Rodriguez0Manuel J. Fernández-Iglesias1Luis Anido-Rifón2David Facal3Roberto Pérez-Rodríguez4Department of Telematics Engineering, Universidad de Vigo, Vigo, SpainDepartment of Telematics Engineering, Universidad de Vigo, Vigo, SpainDepartment of Telematics Engineering, Universidad de Vigo, Vigo, SpainDepartment of Developmental Psychology, Universidad de Santiago de Compostela, Santiago de Compostela, SpainDepartment of Telematics Engineering, Universidad de Vigo, Vigo, SpainIntroduction Assessment of episodic memory is traditionally used to evaluate potential cognitive impairments in senior adults. The present article discusses the capabilities of Episodix, a game to assess the aforementioned cognitive area, as a valid tool to discriminate among mild cognitive impairment (MCI), Alzheimer’s disease (AD) and healthy individuals (HC); that is, it studies the game’s psychometric validity study to assess cognitive impairment. Materials and Methods After a preliminary study, a new pilot study, statistically significant for the Galician population, was carried out from a cross-sectional sample of senior adults as target users. A total of 64 individuals (28 HC, 16 MCI, 20 AD) completed the experiment from an initial sample of 74. Participants were administered a collection of classical pen-and-paper tests and interacted with the games developed. A total of six machine learning classification techniques were applied and four relevant performance metrics were computed to assess the classification power of the tool according to participants’ cognitive status. Results According to the classification performance metrics computed, the best classification result is obtained using the Extra Trees Classifier (F1 = 0.97 and Cohen’s kappa coefficient = 0.97). Precision and recall values are also high, above 0.9 for all cognitive groups. Moreover, according to the standard interpretation of Cohen’s kappa index, classification is almost perfect (i.e., 0.81–1.00) for the complete dataset for all algorithms. Limitations Weaknesses (e.g., accessibility, sample size or speed of stimuli) detected during the preliminary study were addressed and solved. Nevertheless, additional research is needed to improve the resolution of the game for the identification of specific cognitive impairments, as well as to achieve a complete validation of the psychometric properties of the digital game. Conclusion Promising results obtained about psychometric validity of Episodix, represent a relevant step ahead towards the introduction of serious games and machine learning in regular clinical practice for detecting MCI or AD. However, more research is needed to explore the introduction of item response theory in this game and to obtain the required normative data for clinical validity.https://peerj.com/articles/5478.pdfHealth gamesComputational neurosciencesDementiaMachine learningPsychometric criterion validityUsability study |
| spellingShingle | Sonia Valladares-Rodriguez Manuel J. Fernández-Iglesias Luis Anido-Rifón David Facal Roberto Pérez-Rodríguez Episodix: a serious game to detect cognitive impairment in senior adults. A psychometric study PeerJ Health games Computational neurosciences Dementia Machine learning Psychometric criterion validity Usability study |
| title | Episodix: a serious game to detect cognitive impairment in senior adults. A psychometric study |
| title_full | Episodix: a serious game to detect cognitive impairment in senior adults. A psychometric study |
| title_fullStr | Episodix: a serious game to detect cognitive impairment in senior adults. A psychometric study |
| title_full_unstemmed | Episodix: a serious game to detect cognitive impairment in senior adults. A psychometric study |
| title_short | Episodix: a serious game to detect cognitive impairment in senior adults. A psychometric study |
| title_sort | episodix a serious game to detect cognitive impairment in senior adults a psychometric study |
| topic | Health games Computational neurosciences Dementia Machine learning Psychometric criterion validity Usability study |
| url | https://peerj.com/articles/5478.pdf |
| work_keys_str_mv | AT soniavalladaresrodriguez episodixaseriousgametodetectcognitiveimpairmentinsenioradultsapsychometricstudy AT manueljfernandeziglesias episodixaseriousgametodetectcognitiveimpairmentinsenioradultsapsychometricstudy AT luisanidorifon episodixaseriousgametodetectcognitiveimpairmentinsenioradultsapsychometricstudy AT davidfacal episodixaseriousgametodetectcognitiveimpairmentinsenioradultsapsychometricstudy AT robertoperezrodriguez episodixaseriousgametodetectcognitiveimpairmentinsenioradultsapsychometricstudy |