Infusing a Convolutional Neural Network with Encoded Joint Node Image Data to Recognize 25 Daily Human Activities
Human activity recognition (HAR) has gained popularity in the field of computer vision such as video surveillance, security, and virtual reality. However, traditional methods are limited in terms of computations and holistic learning of human skeletal sequences. In this article, a new time‐series sk...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-11-01
|
| Series: | Advanced Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/aisy.202300266 |
| _version_ | 1827632237210763264 |
|---|---|
| author | Yuliang Zhao Tianang Sun Zhongjie Ju Fanghecong Dong Le Yang Xiaoyong Lv Chao Lian Meng Chen Wen Jung Li |
| author_facet | Yuliang Zhao Tianang Sun Zhongjie Ju Fanghecong Dong Le Yang Xiaoyong Lv Chao Lian Meng Chen Wen Jung Li |
| author_sort | Yuliang Zhao |
| collection | DOAJ |
| description | Human activity recognition (HAR) has gained popularity in the field of computer vision such as video surveillance, security, and virtual reality. However, traditional methods are limited in terms of computations and holistic learning of human skeletal sequences. In this article, a new time‐series skeleton joint data imaging method is infused into an improved convolutional neural network to handle these problems. First, the raw time‐series data of 33 body nodes are transformed to red–green–blue images by encoding the 3D positional information to one pixel. Second, the LeNet‐5 network is enhanced by expanding the receptive field, introducing coordinate attention and the smooth maximum unit to improve smoothness and feature extraction. Third, the ability of coded images to express human activities is studied in various environments. It is shown in the experimental results that the method achieves an impressive accuracy of 98.02% in recognizing 25 daily human activities, such as running, writing, and walking. In addition, it is shown that the number of floating point operations, parameters, and inference time of the method are 0.08%, 0.47%, and 3.05%, respectively, of the average values for six other networks (including AlexNet, GoogLeNet, and MobileNet). The proposed method is thus a novel, lightweight, and high‐precision solution for HAR. |
| first_indexed | 2024-03-09T14:33:15Z |
| format | Article |
| id | doaj.art-f8e0188cb0c144a29bc68065a152df7a |
| institution | Directory Open Access Journal |
| issn | 2640-4567 |
| language | English |
| last_indexed | 2024-03-09T14:33:15Z |
| publishDate | 2023-11-01 |
| publisher | Wiley |
| record_format | Article |
| series | Advanced Intelligent Systems |
| spelling | doaj.art-f8e0188cb0c144a29bc68065a152df7a2023-11-27T21:14:08ZengWileyAdvanced Intelligent Systems2640-45672023-11-01511n/an/a10.1002/aisy.202300266Infusing a Convolutional Neural Network with Encoded Joint Node Image Data to Recognize 25 Daily Human ActivitiesYuliang Zhao0Tianang Sun1Zhongjie Ju2Fanghecong Dong3Le Yang4Xiaoyong Lv5Chao Lian6Meng Chen7Wen Jung Li8School of Information Science and Engineering Northeastern University Shenyang 110819 ChinaSchool of Information Science and Engineering Northeastern University Shenyang 110819 ChinaSchool of Information Science and Engineering Northeastern University Shenyang 110819 ChinaSchool of Information Science and Engineering Northeastern University Shenyang 110819 ChinaSchool of Information Science and Engineering Northeastern University Shenyang 110819 ChinaSchool of Information Science and Engineering Northeastern University Shenyang 110819 ChinaSchool of Information Science and Engineering Northeastern University Shenyang 110819 ChinaDepartment of Mechanical Engineering City University of Hong Kong Hong Kong SAR 999077 ChinaDepartment of Mechanical Engineering City University of Hong Kong Hong Kong SAR 999077 ChinaHuman activity recognition (HAR) has gained popularity in the field of computer vision such as video surveillance, security, and virtual reality. However, traditional methods are limited in terms of computations and holistic learning of human skeletal sequences. In this article, a new time‐series skeleton joint data imaging method is infused into an improved convolutional neural network to handle these problems. First, the raw time‐series data of 33 body nodes are transformed to red–green–blue images by encoding the 3D positional information to one pixel. Second, the LeNet‐5 network is enhanced by expanding the receptive field, introducing coordinate attention and the smooth maximum unit to improve smoothness and feature extraction. Third, the ability of coded images to express human activities is studied in various environments. It is shown in the experimental results that the method achieves an impressive accuracy of 98.02% in recognizing 25 daily human activities, such as running, writing, and walking. In addition, it is shown that the number of floating point operations, parameters, and inference time of the method are 0.08%, 0.47%, and 3.05%, respectively, of the average values for six other networks (including AlexNet, GoogLeNet, and MobileNet). The proposed method is thus a novel, lightweight, and high‐precision solution for HAR.https://doi.org/10.1002/aisy.202300266convolutional neural networkhuman activity recognitionimage processing |
| spellingShingle | Yuliang Zhao Tianang Sun Zhongjie Ju Fanghecong Dong Le Yang Xiaoyong Lv Chao Lian Meng Chen Wen Jung Li Infusing a Convolutional Neural Network with Encoded Joint Node Image Data to Recognize 25 Daily Human Activities Advanced Intelligent Systems convolutional neural network human activity recognition image processing |
| title | Infusing a Convolutional Neural Network with Encoded Joint Node Image Data to Recognize 25 Daily Human Activities |
| title_full | Infusing a Convolutional Neural Network with Encoded Joint Node Image Data to Recognize 25 Daily Human Activities |
| title_fullStr | Infusing a Convolutional Neural Network with Encoded Joint Node Image Data to Recognize 25 Daily Human Activities |
| title_full_unstemmed | Infusing a Convolutional Neural Network with Encoded Joint Node Image Data to Recognize 25 Daily Human Activities |
| title_short | Infusing a Convolutional Neural Network with Encoded Joint Node Image Data to Recognize 25 Daily Human Activities |
| title_sort | infusing a convolutional neural network with encoded joint node image data to recognize 25 daily human activities |
| topic | convolutional neural network human activity recognition image processing |
| url | https://doi.org/10.1002/aisy.202300266 |
| work_keys_str_mv | AT yuliangzhao infusingaconvolutionalneuralnetworkwithencodedjointnodeimagedatatorecognize25dailyhumanactivities AT tianangsun infusingaconvolutionalneuralnetworkwithencodedjointnodeimagedatatorecognize25dailyhumanactivities AT zhongjieju infusingaconvolutionalneuralnetworkwithencodedjointnodeimagedatatorecognize25dailyhumanactivities AT fanghecongdong infusingaconvolutionalneuralnetworkwithencodedjointnodeimagedatatorecognize25dailyhumanactivities AT leyang infusingaconvolutionalneuralnetworkwithencodedjointnodeimagedatatorecognize25dailyhumanactivities AT xiaoyonglv infusingaconvolutionalneuralnetworkwithencodedjointnodeimagedatatorecognize25dailyhumanactivities AT chaolian infusingaconvolutionalneuralnetworkwithencodedjointnodeimagedatatorecognize25dailyhumanactivities AT mengchen infusingaconvolutionalneuralnetworkwithencodedjointnodeimagedatatorecognize25dailyhumanactivities AT wenjungli infusingaconvolutionalneuralnetworkwithencodedjointnodeimagedatatorecognize25dailyhumanactivities |