PAR-Net: An Enhanced Dual-Stream CNN–ESN Architecture for Human Physical Activity Recognition
Physical exercise affects many facets of life, including mental health, social interaction, physical fitness, and illness prevention, among many others. Therefore, several AI-driven techniques have been developed in the literature to recognize human physical activities. However, these techniques fai...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/6/1908 |
| _version_ | 1797239297109131264 |
|---|---|
| author | Imran Ullah Khan Jong Weon Lee |
| author_facet | Imran Ullah Khan Jong Weon Lee |
| author_sort | Imran Ullah Khan |
| collection | DOAJ |
| description | Physical exercise affects many facets of life, including mental health, social interaction, physical fitness, and illness prevention, among many others. Therefore, several AI-driven techniques have been developed in the literature to recognize human physical activities. However, these techniques fail to adequately learn the temporal and spatial features of the data patterns. Additionally, these techniques are unable to fully comprehend complex activity patterns over different periods, emphasizing the need for enhanced architectures to further increase accuracy by learning spatiotemporal dependencies in the data individually. Therefore, in this work, we develop an attention-enhanced dual-stream network (PAR-Net) for physical activity recognition with the ability to extract both spatial and temporal features simultaneously. The PAR-Net integrates convolutional neural networks (CNNs) and echo state networks (ESNs), followed by a self-attention mechanism for optimal feature selection. The dual-stream feature extraction mechanism enables the PAR-Net to learn spatiotemporal dependencies from actual data. Furthermore, the incorporation of a self-attention mechanism makes a substantial contribution by facilitating targeted attention on significant features, hence enhancing the identification of nuanced activity patterns. The PAR-Net was evaluated on two benchmark physical activity recognition datasets and achieved higher performance by surpassing the baselines comparatively. Additionally, a thorough ablation study was conducted to determine the best optimal model for human physical activity recognition. |
| first_indexed | 2024-04-24T17:49:18Z |
| format | Article |
| id | doaj.art-4956731384d74b7c8635abb1a081c7f9 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-24T17:49:18Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-4956731384d74b7c8635abb1a081c7f92024-03-27T14:04:07ZengMDPI AGSensors1424-82202024-03-01246190810.3390/s24061908PAR-Net: An Enhanced Dual-Stream CNN–ESN Architecture for Human Physical Activity RecognitionImran Ullah Khan0Jong Weon Lee1Mixed Reality and Interaction Lab, Department of Software, Sejong University, Seoul 05006, Republic of KoreaMixed Reality and Interaction Lab, Department of Software, Sejong University, Seoul 05006, Republic of KoreaPhysical exercise affects many facets of life, including mental health, social interaction, physical fitness, and illness prevention, among many others. Therefore, several AI-driven techniques have been developed in the literature to recognize human physical activities. However, these techniques fail to adequately learn the temporal and spatial features of the data patterns. Additionally, these techniques are unable to fully comprehend complex activity patterns over different periods, emphasizing the need for enhanced architectures to further increase accuracy by learning spatiotemporal dependencies in the data individually. Therefore, in this work, we develop an attention-enhanced dual-stream network (PAR-Net) for physical activity recognition with the ability to extract both spatial and temporal features simultaneously. The PAR-Net integrates convolutional neural networks (CNNs) and echo state networks (ESNs), followed by a self-attention mechanism for optimal feature selection. The dual-stream feature extraction mechanism enables the PAR-Net to learn spatiotemporal dependencies from actual data. Furthermore, the incorporation of a self-attention mechanism makes a substantial contribution by facilitating targeted attention on significant features, hence enhancing the identification of nuanced activity patterns. The PAR-Net was evaluated on two benchmark physical activity recognition datasets and achieved higher performance by surpassing the baselines comparatively. Additionally, a thorough ablation study was conducted to determine the best optimal model for human physical activity recognition.https://www.mdpi.com/1424-8220/24/6/1908physical activity recognitiondeep learningmachine learningskeleton dataecho state networks |
| spellingShingle | Imran Ullah Khan Jong Weon Lee PAR-Net: An Enhanced Dual-Stream CNN–ESN Architecture for Human Physical Activity Recognition Sensors physical activity recognition deep learning machine learning skeleton data echo state networks |
| title | PAR-Net: An Enhanced Dual-Stream CNN–ESN Architecture for Human Physical Activity Recognition |
| title_full | PAR-Net: An Enhanced Dual-Stream CNN–ESN Architecture for Human Physical Activity Recognition |
| title_fullStr | PAR-Net: An Enhanced Dual-Stream CNN–ESN Architecture for Human Physical Activity Recognition |
| title_full_unstemmed | PAR-Net: An Enhanced Dual-Stream CNN–ESN Architecture for Human Physical Activity Recognition |
| title_short | PAR-Net: An Enhanced Dual-Stream CNN–ESN Architecture for Human Physical Activity Recognition |
| title_sort | par net an enhanced dual stream cnn esn architecture for human physical activity recognition |
| topic | physical activity recognition deep learning machine learning skeleton data echo state networks |
| url | https://www.mdpi.com/1424-8220/24/6/1908 |
| work_keys_str_mv | AT imranullahkhan parnetanenhanceddualstreamcnnesnarchitectureforhumanphysicalactivityrecognition AT jongweonlee parnetanenhanceddualstreamcnnesnarchitectureforhumanphysicalactivityrecognition |