Evaluation of electrocardiogram: numerical vs. image data for emotion recognition system [version 2; peer review: 2 approved, 1 approved with reservations]
Background: The electrocardiogram (ECG) is a physiological signal used to diagnose and monitor cardiovascular disease, usually using 2- D ECG. Numerous studies have proven that ECG can be used to detect human emotions using 1-D ECG; however, ECG is typically captured as 2-D images rather than as 1-D...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
F1000 Research Ltd
2022-05-01
|
| Series: | F1000Research |
| Subjects: | |
| Online Access: | https://f1000research.com/articles/10-1114/v2 |
| _version_ | 1811322508900564992 |
|---|---|
| author | Siti Zainab Ibrahim Sophan Wahyudi Nawawi Lee Chia Chun Sharifah Noor Masidayu Sayed Ismail Nor Azlina Ab. Aziz Salem Alelyani Mohamed Mohana |
| author_facet | Siti Zainab Ibrahim Sophan Wahyudi Nawawi Lee Chia Chun Sharifah Noor Masidayu Sayed Ismail Nor Azlina Ab. Aziz Salem Alelyani Mohamed Mohana |
| author_sort | Siti Zainab Ibrahim |
| collection | DOAJ |
| description | Background: The electrocardiogram (ECG) is a physiological signal used to diagnose and monitor cardiovascular disease, usually using 2- D ECG. Numerous studies have proven that ECG can be used to detect human emotions using 1-D ECG; however, ECG is typically captured as 2-D images rather than as 1-D data. There is still no consensus on the effect of the ECG input format on the accuracy of the emotion recognition system (ERS). The ERS using 2-D ECG is still inadequately studied. Therefore, this study compared ERS performance using 1-D and 2-D ECG data to investigate the effect of the ECG input format on the ERS. Methods: This study employed the DREAMER dataset, which contains 23 ECG recordings obtained during audio-visual emotional elicitation. Numerical data was converted to ECG images for the comparison. Numerous approaches were used to obtain ECG features. The Augsburg BioSignal Toolbox (AUBT) and the Toolbox for Emotional feature extraction from Physiological signals (TEAP) extracted features from numerical data. Meanwhile, features were extracted from image data using Oriented FAST and rotated BRIEF (ORB), Scale Invariant Feature Transform (SIFT), KAZE, Accelerated-KAZE (AKAZE), Binary Robust Invariant Scalable Keypoints (BRISK), and Histogram of Oriented Gradients (HOG). Dimension reduction was accomplished using linear discriminant analysis (LDA), and valence and arousal were classified using the Support Vector Machine (SVM). Results: The experimental results show 1-D ECG-based ERS achieved 65.06% of accuracy and 75.63% of F1 score for valence, and 57.83% of accuracy and 44.44% of F1-score for arousal. For 2-D ECG-based ERS, the highest accuracy and F1-score for valence were 62.35% and 49.57%; whereas, the arousal was 59.64% and 59.71%. Conclusions: The results indicate that both inputs work comparably well in classifying emotions, which demonstrates the potential of 1-D and 2-D as input modalities for the ERS. |
| first_indexed | 2024-04-13T13:36:11Z |
| format | Article |
| id | doaj.art-bad6e336de514ba7bda4ea68bcd16ce2 |
| institution | Directory Open Access Journal |
| issn | 2046-1402 |
| language | English |
| last_indexed | 2024-04-13T13:36:11Z |
| publishDate | 2022-05-01 |
| publisher | F1000 Research Ltd |
| record_format | Article |
| series | F1000Research |
| spelling | doaj.art-bad6e336de514ba7bda4ea68bcd16ce22022-12-22T02:44:46ZengF1000 Research LtdF1000Research2046-14022022-05-0110134293Evaluation of electrocardiogram: numerical vs. image data for emotion recognition system [version 2; peer review: 2 approved, 1 approved with reservations]Siti Zainab Ibrahim0Sophan Wahyudi Nawawi1Lee Chia Chun2Sharifah Noor Masidayu Sayed Ismail3https://orcid.org/0000-0002-4667-6415Nor Azlina Ab. Aziz4https://orcid.org/0000-0002-2119-6191Salem Alelyani5Mohamed Mohana6https://orcid.org/0000-0002-9485-8731Faculty of Information Science & Technology, Multimedia University, Bukit Beruang,, Melaka, 75450, MalaysiaSchool of Electrical Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, Skudai, Johor Bahru, 81310, MalaysiaHexon Data Sdn Bhd, Kuala Lumpur, 59200, MalaysiaFaculty of Information Science & Technology, Multimedia University, Bukit Beruang,, Melaka, 75450, MalaysiaFaculty of Engineering, Multimedia University, Bukit Beruang, Melaka, 75450, MalaysiaCenter for Artificial Intelligence, King Khalid University, Abha, 61421, Saudi ArabiaCenter for Artificial Intelligence, King Khalid University, Abha, 61421, Saudi ArabiaBackground: The electrocardiogram (ECG) is a physiological signal used to diagnose and monitor cardiovascular disease, usually using 2- D ECG. Numerous studies have proven that ECG can be used to detect human emotions using 1-D ECG; however, ECG is typically captured as 2-D images rather than as 1-D data. There is still no consensus on the effect of the ECG input format on the accuracy of the emotion recognition system (ERS). The ERS using 2-D ECG is still inadequately studied. Therefore, this study compared ERS performance using 1-D and 2-D ECG data to investigate the effect of the ECG input format on the ERS. Methods: This study employed the DREAMER dataset, which contains 23 ECG recordings obtained during audio-visual emotional elicitation. Numerical data was converted to ECG images for the comparison. Numerous approaches were used to obtain ECG features. The Augsburg BioSignal Toolbox (AUBT) and the Toolbox for Emotional feature extraction from Physiological signals (TEAP) extracted features from numerical data. Meanwhile, features were extracted from image data using Oriented FAST and rotated BRIEF (ORB), Scale Invariant Feature Transform (SIFT), KAZE, Accelerated-KAZE (AKAZE), Binary Robust Invariant Scalable Keypoints (BRISK), and Histogram of Oriented Gradients (HOG). Dimension reduction was accomplished using linear discriminant analysis (LDA), and valence and arousal were classified using the Support Vector Machine (SVM). Results: The experimental results show 1-D ECG-based ERS achieved 65.06% of accuracy and 75.63% of F1 score for valence, and 57.83% of accuracy and 44.44% of F1-score for arousal. For 2-D ECG-based ERS, the highest accuracy and F1-score for valence were 62.35% and 49.57%; whereas, the arousal was 59.64% and 59.71%. Conclusions: The results indicate that both inputs work comparably well in classifying emotions, which demonstrates the potential of 1-D and 2-D as input modalities for the ERS.https://f1000research.com/articles/10-1114/v2Emotion recognition electrocardiogram numerical ECG image ECG DREAMEReng |
| spellingShingle | Siti Zainab Ibrahim Sophan Wahyudi Nawawi Lee Chia Chun Sharifah Noor Masidayu Sayed Ismail Nor Azlina Ab. Aziz Salem Alelyani Mohamed Mohana Evaluation of electrocardiogram: numerical vs. image data for emotion recognition system [version 2; peer review: 2 approved, 1 approved with reservations] F1000Research Emotion recognition electrocardiogram numerical ECG image ECG DREAMER eng |
| title | Evaluation of electrocardiogram: numerical vs. image data for emotion recognition system [version 2; peer review: 2 approved, 1 approved with reservations] |
| title_full | Evaluation of electrocardiogram: numerical vs. image data for emotion recognition system [version 2; peer review: 2 approved, 1 approved with reservations] |
| title_fullStr | Evaluation of electrocardiogram: numerical vs. image data for emotion recognition system [version 2; peer review: 2 approved, 1 approved with reservations] |
| title_full_unstemmed | Evaluation of electrocardiogram: numerical vs. image data for emotion recognition system [version 2; peer review: 2 approved, 1 approved with reservations] |
| title_short | Evaluation of electrocardiogram: numerical vs. image data for emotion recognition system [version 2; peer review: 2 approved, 1 approved with reservations] |
| title_sort | evaluation of electrocardiogram numerical vs image data for emotion recognition system version 2 peer review 2 approved 1 approved with reservations |
| topic | Emotion recognition electrocardiogram numerical ECG image ECG DREAMER eng |
| url | https://f1000research.com/articles/10-1114/v2 |
| work_keys_str_mv | AT sitizainabibrahim evaluationofelectrocardiogramnumericalvsimagedataforemotionrecognitionsystemversion2peerreview2approved1approvedwithreservations AT sophanwahyudinawawi evaluationofelectrocardiogramnumericalvsimagedataforemotionrecognitionsystemversion2peerreview2approved1approvedwithreservations AT leechiachun evaluationofelectrocardiogramnumericalvsimagedataforemotionrecognitionsystemversion2peerreview2approved1approvedwithreservations AT sharifahnoormasidayusayedismail evaluationofelectrocardiogramnumericalvsimagedataforemotionrecognitionsystemversion2peerreview2approved1approvedwithreservations AT norazlinaabaziz evaluationofelectrocardiogramnumericalvsimagedataforemotionrecognitionsystemversion2peerreview2approved1approvedwithreservations AT salemalelyani evaluationofelectrocardiogramnumericalvsimagedataforemotionrecognitionsystemversion2peerreview2approved1approvedwithreservations AT mohamedmohana evaluationofelectrocardiogramnumericalvsimagedataforemotionrecognitionsystemversion2peerreview2approved1approvedwithreservations |