Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos
Compared to traditional dance, intangible cultural heritage dance often involves the isotropic extension of choreographic actions, utilizing both upper and lower limbs. This characteristic choreography style makes the remote joints lack interaction, consequently reducing accuracy in existing human m...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/12/23/4830 |
| _version_ | 1797400226588262400 |
|---|---|
| author | Xingquan Cai Pengyan Cheng Shike Liu Haoyu Zhang Haiyan Sun |
| author_facet | Xingquan Cai Pengyan Cheng Shike Liu Haoyu Zhang Haiyan Sun |
| author_sort | Xingquan Cai |
| collection | DOAJ |
| description | Compared to traditional dance, intangible cultural heritage dance often involves the isotropic extension of choreographic actions, utilizing both upper and lower limbs. This characteristic choreography style makes the remote joints lack interaction, consequently reducing accuracy in existing human motion prediction methods. Therefore, we propose a human motion prediction method based on the multi-scale hypergraph convolutional network of the intangible cultural heritage dance video. Firstly, this method inputs the 3D human posture sequence from intangible cultural heritage dance videos. The hypergraph is designed according to the synergistic relationship of the human joints in the intangible cultural heritage dance video, which is used to represent the spatial correlation of the 3D human posture. Then, a multi-scale hypergraph convolutional network is constructed, utilizing multi-scale transformation operators to segment the human skeleton into different scales. This network adopts a graph structure to represent the 3D human posture at different scales, which is then used by the single-scalar fusion operator to spatial features in the 3D human posture sequence are extracted by fusing the feature information of the hypergraph and the multi-scale graph. Finally, the Temporal Graph Transformer network is introduced to capture the temporal dependence among adjacent frames within the time domain. This facilitates the extraction of temporal features from the 3D human posture sequence, ultimately enabling the prediction of future 3D human posture sequences. Experiments show that we achieve the best performance in both short-term and long-term human motion prediction when compared to Motion-Mixer and Motion-Attention algorithms on Human3.6M and 3DPW datasets. In addition, ablation experiments show that our method can predict more precise 3D human pose sequences, even in the presence of isotropic extensions of upper and lower limbs in intangible cultural heritage dance videos. This approach effectively addresses the issue of missing segments in intangible cultural heritage dance videos. |
| first_indexed | 2024-03-09T01:52:31Z |
| format | Article |
| id | doaj.art-0ee88f5471f6468383f065e8f937603e |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-09T01:52:31Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-0ee88f5471f6468383f065e8f937603e2023-12-08T15:14:10ZengMDPI AGElectronics2079-92922023-11-011223483010.3390/electronics12234830Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance VideosXingquan Cai0Pengyan Cheng1Shike Liu2Haoyu Zhang3Haiyan Sun4School of Information Science and Technology, North China University of Technology, Beijing 100144, ChinaSchool of Information Science and Technology, North China University of Technology, Beijing 100144, ChinaSchool of Information Science and Technology, North China University of Technology, Beijing 100144, ChinaSchool of Information Science and Technology, North China University of Technology, Beijing 100144, ChinaSchool of Information Science and Technology, North China University of Technology, Beijing 100144, ChinaCompared to traditional dance, intangible cultural heritage dance often involves the isotropic extension of choreographic actions, utilizing both upper and lower limbs. This characteristic choreography style makes the remote joints lack interaction, consequently reducing accuracy in existing human motion prediction methods. Therefore, we propose a human motion prediction method based on the multi-scale hypergraph convolutional network of the intangible cultural heritage dance video. Firstly, this method inputs the 3D human posture sequence from intangible cultural heritage dance videos. The hypergraph is designed according to the synergistic relationship of the human joints in the intangible cultural heritage dance video, which is used to represent the spatial correlation of the 3D human posture. Then, a multi-scale hypergraph convolutional network is constructed, utilizing multi-scale transformation operators to segment the human skeleton into different scales. This network adopts a graph structure to represent the 3D human posture at different scales, which is then used by the single-scalar fusion operator to spatial features in the 3D human posture sequence are extracted by fusing the feature information of the hypergraph and the multi-scale graph. Finally, the Temporal Graph Transformer network is introduced to capture the temporal dependence among adjacent frames within the time domain. This facilitates the extraction of temporal features from the 3D human posture sequence, ultimately enabling the prediction of future 3D human posture sequences. Experiments show that we achieve the best performance in both short-term and long-term human motion prediction when compared to Motion-Mixer and Motion-Attention algorithms on Human3.6M and 3DPW datasets. In addition, ablation experiments show that our method can predict more precise 3D human pose sequences, even in the presence of isotropic extensions of upper and lower limbs in intangible cultural heritage dance videos. This approach effectively addresses the issue of missing segments in intangible cultural heritage dance videos.https://www.mdpi.com/2079-9292/12/23/4830human motion predictionhypergraphmulti-scale hypergraph convolutional networktransformer |
| spellingShingle | Xingquan Cai Pengyan Cheng Shike Liu Haoyu Zhang Haiyan Sun Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos Electronics human motion prediction hypergraph multi-scale hypergraph convolutional network transformer |
| title | Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos |
| title_full | Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos |
| title_fullStr | Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos |
| title_full_unstemmed | Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos |
| title_short | Human Motion Prediction Based on a Multi-Scale Hypergraph for Intangible Cultural Heritage Dance Videos |
| title_sort | human motion prediction based on a multi scale hypergraph for intangible cultural heritage dance videos |
| topic | human motion prediction hypergraph multi-scale hypergraph convolutional network transformer |
| url | https://www.mdpi.com/2079-9292/12/23/4830 |
| work_keys_str_mv | AT xingquancai humanmotionpredictionbasedonamultiscalehypergraphforintangibleculturalheritagedancevideos AT pengyancheng humanmotionpredictionbasedonamultiscalehypergraphforintangibleculturalheritagedancevideos AT shikeliu humanmotionpredictionbasedonamultiscalehypergraphforintangibleculturalheritagedancevideos AT haoyuzhang humanmotionpredictionbasedonamultiscalehypergraphforintangibleculturalheritagedancevideos AT haiyansun humanmotionpredictionbasedonamultiscalehypergraphforintangibleculturalheritagedancevideos |