PointMapNet: Point Cloud Feature Map Network for 3D Human Action Recognition
3D human action recognition is crucial in broad industrial application scenarios such as robotics, video surveillance, autonomous driving, or intellectual education, etc. In this paper, we present a new point cloud sequence network called PointMapNet for 3D human action recognition. In PointMapNet,...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Symmetry |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-8994/15/2/363 |
| _version_ | 1797618099241877504 |
|---|---|
| author | Xing Li Qian Huang Yunfei Zhang Tianjin Yang Zhijian Wang |
| author_facet | Xing Li Qian Huang Yunfei Zhang Tianjin Yang Zhijian Wang |
| author_sort | Xing Li |
| collection | DOAJ |
| description | 3D human action recognition is crucial in broad industrial application scenarios such as robotics, video surveillance, autonomous driving, or intellectual education, etc. In this paper, we present a new point cloud sequence network called PointMapNet for 3D human action recognition. In PointMapNet, two point cloud feature maps symmetrical to depth feature maps are proposed to summarize appearance and motion representations from point cloud sequences. Specifically, we first convert the point cloud frames to virtual action frames using static point cloud techniques. The virtual action frame is a 1D vector used to characterize the structural details in the point cloud frame. Then, inspired by feature map-based human action recognition on depth sequences, two point cloud feature maps are symmetrically constructed to recognize human action from the point cloud sequence, i.e., Point Cloud Appearance Map (PCAM) and Point Cloud Motion Map (PCMM). To construct PCAM, an MLP-like network architecture is designed and used to capture the spatio-temporal appearance feature of the human action in a virtual action sequence. To construct PCMM, the MLP-like network architecture is used to capture the motion feature of the human action in a virtual action difference sequence. Finally, the two point cloud feature map descriptors are concatenated and fed to a fully connected classifier for human action recognition. In order to evaluate the performance of the proposed approach, extensive experiments are conducted. The proposed method achieves impressive results on three benchmark datasets, namely NTU RGB+D 60 (89.4% cross-subject and 96.7% cross-view), UTD-MHAD (91.61%), and MSR Action3D (91.91%). The experimental results outperform existing state-of-the-art point cloud sequence classification networks, demonstrating the effectiveness of our method. |
| first_indexed | 2024-03-11T08:05:23Z |
| format | Article |
| id | doaj.art-f499269bfb644de39cdc78ed976463d1 |
| institution | Directory Open Access Journal |
| issn | 2073-8994 |
| language | English |
| last_indexed | 2024-03-11T08:05:23Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Symmetry |
| spelling | doaj.art-f499269bfb644de39cdc78ed976463d12023-11-16T23:32:19ZengMDPI AGSymmetry2073-89942023-01-0115236310.3390/sym15020363PointMapNet: Point Cloud Feature Map Network for 3D Human Action RecognitionXing Li0Qian Huang1Yunfei Zhang2Tianjin Yang3Zhijian Wang4The Key Laboratory of Water Big Data Technology of Ministry of Water Resources, Hohai University, Nanjing 211100, ChinaThe Key Laboratory of Water Big Data Technology of Ministry of Water Resources, Hohai University, Nanjing 211100, ChinaThe Key Laboratory of Water Big Data Technology of Ministry of Water Resources, Hohai University, Nanjing 211100, ChinaThe Key Laboratory of Water Big Data Technology of Ministry of Water Resources, Hohai University, Nanjing 211100, ChinaThe Key Laboratory of Water Big Data Technology of Ministry of Water Resources, Hohai University, Nanjing 211100, China3D human action recognition is crucial in broad industrial application scenarios such as robotics, video surveillance, autonomous driving, or intellectual education, etc. In this paper, we present a new point cloud sequence network called PointMapNet for 3D human action recognition. In PointMapNet, two point cloud feature maps symmetrical to depth feature maps are proposed to summarize appearance and motion representations from point cloud sequences. Specifically, we first convert the point cloud frames to virtual action frames using static point cloud techniques. The virtual action frame is a 1D vector used to characterize the structural details in the point cloud frame. Then, inspired by feature map-based human action recognition on depth sequences, two point cloud feature maps are symmetrically constructed to recognize human action from the point cloud sequence, i.e., Point Cloud Appearance Map (PCAM) and Point Cloud Motion Map (PCMM). To construct PCAM, an MLP-like network architecture is designed and used to capture the spatio-temporal appearance feature of the human action in a virtual action sequence. To construct PCMM, the MLP-like network architecture is used to capture the motion feature of the human action in a virtual action difference sequence. Finally, the two point cloud feature map descriptors are concatenated and fed to a fully connected classifier for human action recognition. In order to evaluate the performance of the proposed approach, extensive experiments are conducted. The proposed method achieves impressive results on three benchmark datasets, namely NTU RGB+D 60 (89.4% cross-subject and 96.7% cross-view), UTD-MHAD (91.61%), and MSR Action3D (91.91%). The experimental results outperform existing state-of-the-art point cloud sequence classification networks, demonstrating the effectiveness of our method.https://www.mdpi.com/2073-8994/15/2/3633D human action recognitionpoint cloud sequencepoint cloud feature map |
| spellingShingle | Xing Li Qian Huang Yunfei Zhang Tianjin Yang Zhijian Wang PointMapNet: Point Cloud Feature Map Network for 3D Human Action Recognition Symmetry 3D human action recognition point cloud sequence point cloud feature map |
| title | PointMapNet: Point Cloud Feature Map Network for 3D Human Action Recognition |
| title_full | PointMapNet: Point Cloud Feature Map Network for 3D Human Action Recognition |
| title_fullStr | PointMapNet: Point Cloud Feature Map Network for 3D Human Action Recognition |
| title_full_unstemmed | PointMapNet: Point Cloud Feature Map Network for 3D Human Action Recognition |
| title_short | PointMapNet: Point Cloud Feature Map Network for 3D Human Action Recognition |
| title_sort | pointmapnet point cloud feature map network for 3d human action recognition |
| topic | 3D human action recognition point cloud sequence point cloud feature map |
| url | https://www.mdpi.com/2073-8994/15/2/363 |
| work_keys_str_mv | AT xingli pointmapnetpointcloudfeaturemapnetworkfor3dhumanactionrecognition AT qianhuang pointmapnetpointcloudfeaturemapnetworkfor3dhumanactionrecognition AT yunfeizhang pointmapnetpointcloudfeaturemapnetworkfor3dhumanactionrecognition AT tianjinyang pointmapnetpointcloudfeaturemapnetworkfor3dhumanactionrecognition AT zhijianwang pointmapnetpointcloudfeaturemapnetworkfor3dhumanactionrecognition |