Deep Learning for Identification of Acute Illness and Facial Cues of Illness
Background: The inclusion of facial and bodily cues (clinical gestalt) in machine learning (ML) models improves the assessment of patients' health status, as shown in genetic syndromes and acute coronary syndrome. It is unknown if the inclusion of clinical gestalt improves ML-based classificati...
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-07-01
|
| Series: | Frontiers in Medicine |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fmed.2021.661309/full |
| _version_ | 1819088162978267136 |
|---|---|
| author | Castela Forte Castela Forte Castela Forte Andrei Voinea Malina Chichirau Galiya Yeshmagambetova Lea M. Albrecht Chiara Erfurt Liliane A. Freundt Luisa Oliveira e Carmo Robert H. Henning Iwan C. C. van der Horst Tina Sundelin Tina Sundelin Marco A. Wiering John Axelsson John Axelsson Anne H. Epema |
| author_facet | Castela Forte Castela Forte Castela Forte Andrei Voinea Malina Chichirau Galiya Yeshmagambetova Lea M. Albrecht Chiara Erfurt Liliane A. Freundt Luisa Oliveira e Carmo Robert H. Henning Iwan C. C. van der Horst Tina Sundelin Tina Sundelin Marco A. Wiering John Axelsson John Axelsson Anne H. Epema |
| author_sort | Castela Forte |
| collection | DOAJ |
| description | Background: The inclusion of facial and bodily cues (clinical gestalt) in machine learning (ML) models improves the assessment of patients' health status, as shown in genetic syndromes and acute coronary syndrome. It is unknown if the inclusion of clinical gestalt improves ML-based classification of acutely ill patients. As in previous research in ML analysis of medical images, simulated or augmented data may be used to assess the usability of clinical gestalt.Objective: To assess whether a deep learning algorithm trained on a dataset of simulated and augmented facial photographs reflecting acutely ill patients can distinguish between healthy and LPS-infused, acutely ill individuals.Methods: Photographs from twenty-six volunteers whose facial features were manipulated to resemble a state of acute illness were used to extract features of illness and generate a synthetic dataset of acutely ill photographs, using a neural transfer convolutional neural network (NT-CNN) for data augmentation. Then, four distinct CNNs were trained on different parts of the facial photographs and concatenated into one final, stacked CNN which classified individuals as healthy or acutely ill. Finally, the stacked CNN was validated in an external dataset of volunteers injected with lipopolysaccharide (LPS).Results: In the external validation set, the four individual feature models distinguished acutely ill patients with sensitivities ranging from 10.5% (95% CI, 1.3–33.1% for the skin model) to 89.4% (66.9–98.7%, for the nose model). Specificity ranged from 42.1% (20.3–66.5%) for the nose model and 94.7% (73.9–99.9%) for skin. The stacked model combining all four facial features achieved an area under the receiver characteristic operating curve (AUROC) of 0.67 (0.62–0.71) and distinguished acutely ill patients with a sensitivity of 100% (82.35–100.00%) and specificity of 42.11% (20.25–66.50%).Conclusion: A deep learning algorithm trained on a synthetic, augmented dataset of facial photographs distinguished between healthy and simulated acutely ill individuals, demonstrating that synthetically generated data can be used to develop algorithms for health conditions in which large datasets are difficult to obtain. These results support the potential of facial feature analysis algorithms to support the diagnosis of acute illness. |
| first_indexed | 2024-12-21T21:47:40Z |
| format | Article |
| id | doaj.art-01a011865ac34edfbeb5bdb1abd8695e |
| institution | Directory Open Access Journal |
| issn | 2296-858X |
| language | English |
| last_indexed | 2024-12-21T21:47:40Z |
| publishDate | 2021-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Medicine |
| spelling | doaj.art-01a011865ac34edfbeb5bdb1abd8695e2022-12-21T18:49:11ZengFrontiers Media S.A.Frontiers in Medicine2296-858X2021-07-01810.3389/fmed.2021.661309661309Deep Learning for Identification of Acute Illness and Facial Cues of IllnessCastela Forte0Castela Forte1Castela Forte2Andrei Voinea3Malina Chichirau4Galiya Yeshmagambetova5Lea M. Albrecht6Chiara Erfurt7Liliane A. Freundt8Luisa Oliveira e Carmo9Robert H. Henning10Iwan C. C. van der Horst11Tina Sundelin12Tina Sundelin13Marco A. Wiering14John Axelsson15John Axelsson16Anne H. Epema17Department of Clinical Pharmacy and Pharmacology, University Medical Center Groningen, University of Groningen, Groningen, NetherlandsDepartment of Anesthesiology, University Medical Center Groningen, University of Groningen, Groningen, NetherlandsBernoulli Institute for Mathematics, Computer Science and Artificial Intelligence, University of Groningen, Groningen, NetherlandsBernoulli Institute for Mathematics, Computer Science and Artificial Intelligence, University of Groningen, Groningen, NetherlandsBernoulli Institute for Mathematics, Computer Science and Artificial Intelligence, University of Groningen, Groningen, NetherlandsBernoulli Institute for Mathematics, Computer Science and Artificial Intelligence, University of Groningen, Groningen, NetherlandsDepartment of Anesthesiology, University Medical Center Groningen, University of Groningen, Groningen, NetherlandsDepartment of Anesthesiology, University Medical Center Groningen, University of Groningen, Groningen, NetherlandsDepartment of Anesthesiology, University Medical Center Groningen, University of Groningen, Groningen, NetherlandsDepartment of Anesthesiology, University Medical Center Groningen, University of Groningen, Groningen, NetherlandsDepartment of Clinical Pharmacy and Pharmacology, University Medical Center Groningen, University of Groningen, Groningen, NetherlandsDepartment of Intensive Care Medicine, Maastricht University Medical Centre+, University Maastricht, Maastricht, NetherlandsDepartment of Psychology, Stress Research Institute, Stockholm University, Stockholm, SwedenDepartment of Clinical Neuroscience, Karolinska Institutet, Stockholm, SwedenBernoulli Institute for Mathematics, Computer Science and Artificial Intelligence, University of Groningen, Groningen, NetherlandsDepartment of Psychology, Stress Research Institute, Stockholm University, Stockholm, SwedenDepartment of Clinical Neuroscience, Karolinska Institutet, Stockholm, SwedenDepartment of Anesthesiology, University Medical Center Groningen, University of Groningen, Groningen, NetherlandsBackground: The inclusion of facial and bodily cues (clinical gestalt) in machine learning (ML) models improves the assessment of patients' health status, as shown in genetic syndromes and acute coronary syndrome. It is unknown if the inclusion of clinical gestalt improves ML-based classification of acutely ill patients. As in previous research in ML analysis of medical images, simulated or augmented data may be used to assess the usability of clinical gestalt.Objective: To assess whether a deep learning algorithm trained on a dataset of simulated and augmented facial photographs reflecting acutely ill patients can distinguish between healthy and LPS-infused, acutely ill individuals.Methods: Photographs from twenty-six volunteers whose facial features were manipulated to resemble a state of acute illness were used to extract features of illness and generate a synthetic dataset of acutely ill photographs, using a neural transfer convolutional neural network (NT-CNN) for data augmentation. Then, four distinct CNNs were trained on different parts of the facial photographs and concatenated into one final, stacked CNN which classified individuals as healthy or acutely ill. Finally, the stacked CNN was validated in an external dataset of volunteers injected with lipopolysaccharide (LPS).Results: In the external validation set, the four individual feature models distinguished acutely ill patients with sensitivities ranging from 10.5% (95% CI, 1.3–33.1% for the skin model) to 89.4% (66.9–98.7%, for the nose model). Specificity ranged from 42.1% (20.3–66.5%) for the nose model and 94.7% (73.9–99.9%) for skin. The stacked model combining all four facial features achieved an area under the receiver characteristic operating curve (AUROC) of 0.67 (0.62–0.71) and distinguished acutely ill patients with a sensitivity of 100% (82.35–100.00%) and specificity of 42.11% (20.25–66.50%).Conclusion: A deep learning algorithm trained on a synthetic, augmented dataset of facial photographs distinguished between healthy and simulated acutely ill individuals, demonstrating that synthetically generated data can be used to develop algorithms for health conditions in which large datasets are difficult to obtain. These results support the potential of facial feature analysis algorithms to support the diagnosis of acute illness.https://www.frontiersin.org/articles/10.3389/fmed.2021.661309/fullgestaltdeep learningfacial analysissynthetic dataacute illness |
| spellingShingle | Castela Forte Castela Forte Castela Forte Andrei Voinea Malina Chichirau Galiya Yeshmagambetova Lea M. Albrecht Chiara Erfurt Liliane A. Freundt Luisa Oliveira e Carmo Robert H. Henning Iwan C. C. van der Horst Tina Sundelin Tina Sundelin Marco A. Wiering John Axelsson John Axelsson Anne H. Epema Deep Learning for Identification of Acute Illness and Facial Cues of Illness Frontiers in Medicine gestalt deep learning facial analysis synthetic data acute illness |
| title | Deep Learning for Identification of Acute Illness and Facial Cues of Illness |
| title_full | Deep Learning for Identification of Acute Illness and Facial Cues of Illness |
| title_fullStr | Deep Learning for Identification of Acute Illness and Facial Cues of Illness |
| title_full_unstemmed | Deep Learning for Identification of Acute Illness and Facial Cues of Illness |
| title_short | Deep Learning for Identification of Acute Illness and Facial Cues of Illness |
| title_sort | deep learning for identification of acute illness and facial cues of illness |
| topic | gestalt deep learning facial analysis synthetic data acute illness |
| url | https://www.frontiersin.org/articles/10.3389/fmed.2021.661309/full |
| work_keys_str_mv | AT castelaforte deeplearningforidentificationofacuteillnessandfacialcuesofillness AT castelaforte deeplearningforidentificationofacuteillnessandfacialcuesofillness AT castelaforte deeplearningforidentificationofacuteillnessandfacialcuesofillness AT andreivoinea deeplearningforidentificationofacuteillnessandfacialcuesofillness AT malinachichirau deeplearningforidentificationofacuteillnessandfacialcuesofillness AT galiyayeshmagambetova deeplearningforidentificationofacuteillnessandfacialcuesofillness AT leamalbrecht deeplearningforidentificationofacuteillnessandfacialcuesofillness AT chiaraerfurt deeplearningforidentificationofacuteillnessandfacialcuesofillness AT lilianeafreundt deeplearningforidentificationofacuteillnessandfacialcuesofillness AT luisaoliveiraecarmo deeplearningforidentificationofacuteillnessandfacialcuesofillness AT roberthhenning deeplearningforidentificationofacuteillnessandfacialcuesofillness AT iwanccvanderhorst deeplearningforidentificationofacuteillnessandfacialcuesofillness AT tinasundelin deeplearningforidentificationofacuteillnessandfacialcuesofillness AT tinasundelin deeplearningforidentificationofacuteillnessandfacialcuesofillness AT marcoawiering deeplearningforidentificationofacuteillnessandfacialcuesofillness AT johnaxelsson deeplearningforidentificationofacuteillnessandfacialcuesofillness AT johnaxelsson deeplearningforidentificationofacuteillnessandfacialcuesofillness AT annehepema deeplearningforidentificationofacuteillnessandfacialcuesofillness |