A Deep Learning-Based Semantic Segmentation Model Using MCNN and Attention Layer for Human Activity Recognition
With the development of wearable devices such as smartwatches, several studies have been conducted on the recognition of various human activities. Various types of data are used, e.g., acceleration data collected using an inertial measurement unit sensor. Most scholars segmented the entire timeserie...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/4/2278 |
| _version_ | 1827755508779450368 |
|---|---|
| author | Sang-hyub Lee Deok-Won Lee Mun Sang Kim |
| author_facet | Sang-hyub Lee Deok-Won Lee Mun Sang Kim |
| author_sort | Sang-hyub Lee |
| collection | DOAJ |
| description | With the development of wearable devices such as smartwatches, several studies have been conducted on the recognition of various human activities. Various types of data are used, e.g., acceleration data collected using an inertial measurement unit sensor. Most scholars segmented the entire timeseries data with a fixed window size before performing recognition. However, this approach has limitations in performance because the execution time of the human activity is usually unknown. Therefore, there have been many attempts to solve this problem through the method of activity recognition by sliding the classification window along the time axis. In this study, we propose a method for classifying all frames rather than a window-based recognition method. For implementation, features extracted using multiple convolutional neural networks with different kernel sizes were fused and used. In addition, similar to the convolutional block attention module, an attention layer to each channel and spatial level is applied to improve the model recognition performance. To verify the performance of the proposed model and prove the effectiveness of the proposed method on human activity recognition, evaluation experiments were performed. For comparison, models using various basic deep learning modules and models, in which all frames were classified for recognizing a specific wave in electrocardiography data were applied. As a result, the proposed model reported the best F1-score (over 0.9) for all kinds of target activities compared to other deep learning-based recognition models. Further, for the improvement verification of the proposed CEF method, the proposed method was compared with three types of SW method. As a result, the proposed method reported the 0.154 higher F1-score than SW. In the case of the designed model, the F1-score was higher as much as 0.184. |
| first_indexed | 2024-03-11T08:10:21Z |
| format | Article |
| id | doaj.art-f09ff6f729a648b6ae9ab8551e89a843 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T08:10:21Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-f09ff6f729a648b6ae9ab8551e89a8432023-11-16T23:12:41ZengMDPI AGSensors1424-82202023-02-01234227810.3390/s23042278A Deep Learning-Based Semantic Segmentation Model Using MCNN and Attention Layer for Human Activity RecognitionSang-hyub Lee0Deok-Won Lee1Mun Sang Kim2School of Integrated Technology, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of KoreaSchool of Integrated Technology, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of KoreaSchool of Integrated Technology, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of KoreaWith the development of wearable devices such as smartwatches, several studies have been conducted on the recognition of various human activities. Various types of data are used, e.g., acceleration data collected using an inertial measurement unit sensor. Most scholars segmented the entire timeseries data with a fixed window size before performing recognition. However, this approach has limitations in performance because the execution time of the human activity is usually unknown. Therefore, there have been many attempts to solve this problem through the method of activity recognition by sliding the classification window along the time axis. In this study, we propose a method for classifying all frames rather than a window-based recognition method. For implementation, features extracted using multiple convolutional neural networks with different kernel sizes were fused and used. In addition, similar to the convolutional block attention module, an attention layer to each channel and spatial level is applied to improve the model recognition performance. To verify the performance of the proposed model and prove the effectiveness of the proposed method on human activity recognition, evaluation experiments were performed. For comparison, models using various basic deep learning modules and models, in which all frames were classified for recognizing a specific wave in electrocardiography data were applied. As a result, the proposed model reported the best F1-score (over 0.9) for all kinds of target activities compared to other deep learning-based recognition models. Further, for the improvement verification of the proposed CEF method, the proposed method was compared with three types of SW method. As a result, the proposed method reported the 0.154 higher F1-score than SW. In the case of the designed model, the F1-score was higher as much as 0.184.https://www.mdpi.com/1424-8220/23/4/2278human activity recognitiontransitional activitiesdeep learningaccelerometer sensorattention layersemantic segmentation |
| spellingShingle | Sang-hyub Lee Deok-Won Lee Mun Sang Kim A Deep Learning-Based Semantic Segmentation Model Using MCNN and Attention Layer for Human Activity Recognition Sensors human activity recognition transitional activities deep learning accelerometer sensor attention layer semantic segmentation |
| title | A Deep Learning-Based Semantic Segmentation Model Using MCNN and Attention Layer for Human Activity Recognition |
| title_full | A Deep Learning-Based Semantic Segmentation Model Using MCNN and Attention Layer for Human Activity Recognition |
| title_fullStr | A Deep Learning-Based Semantic Segmentation Model Using MCNN and Attention Layer for Human Activity Recognition |
| title_full_unstemmed | A Deep Learning-Based Semantic Segmentation Model Using MCNN and Attention Layer for Human Activity Recognition |
| title_short | A Deep Learning-Based Semantic Segmentation Model Using MCNN and Attention Layer for Human Activity Recognition |
| title_sort | deep learning based semantic segmentation model using mcnn and attention layer for human activity recognition |
| topic | human activity recognition transitional activities deep learning accelerometer sensor attention layer semantic segmentation |
| url | https://www.mdpi.com/1424-8220/23/4/2278 |
| work_keys_str_mv | AT sanghyublee adeeplearningbasedsemanticsegmentationmodelusingmcnnandattentionlayerforhumanactivityrecognition AT deokwonlee adeeplearningbasedsemanticsegmentationmodelusingmcnnandattentionlayerforhumanactivityrecognition AT munsangkim adeeplearningbasedsemanticsegmentationmodelusingmcnnandattentionlayerforhumanactivityrecognition AT sanghyublee deeplearningbasedsemanticsegmentationmodelusingmcnnandattentionlayerforhumanactivityrecognition AT deokwonlee deeplearningbasedsemanticsegmentationmodelusingmcnnandattentionlayerforhumanactivityrecognition AT munsangkim deeplearningbasedsemanticsegmentationmodelusingmcnnandattentionlayerforhumanactivityrecognition |