Learning progressive joint propagation for human motion prediction
Despite the great progress in human motion prediction, it remains a challenging task due to the complicated structural dynamics of human behaviors. In this paper, we address this problem in three aspects. First, to capture the long-range spatial correlations and temporal dependencies, we apply a tra...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Conference Paper |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/144139 |
| _version_ | 1824456634437468160 |
|---|---|
| author | Cai, Yujun Huang, Lin Wang, Yiwei Cham, Tat-Jen Cai, Jianfei Yuan, Junsong Liu, Jun Yang, Xu Zhu, Yiheng Shen, Xiaohui Liu, Ding Liu, Jing Thalmann, Nadia Magnenat |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Cai, Yujun Huang, Lin Wang, Yiwei Cham, Tat-Jen Cai, Jianfei Yuan, Junsong Liu, Jun Yang, Xu Zhu, Yiheng Shen, Xiaohui Liu, Ding Liu, Jing Thalmann, Nadia Magnenat |
| author_sort | Cai, Yujun |
| collection | NTU |
| description | Despite the great progress in human motion prediction, it remains a challenging task due to the complicated structural dynamics of human behaviors. In this paper, we address this problem in three aspects. First, to capture the long-range spatial correlations and temporal dependencies, we apply a transformer-based architecture with the global attention mechanism. Speci cally, we feed the network with the sequential joints encoded with the temporal information for spatial and temporal explorations. Second, to further exploit the inherent kinematic chains for better 3D structures, we apply a progressive-decoding strategy, which performs in a central-to-peripheral extension according to the structural connectivity. Last, in order to incorporate a general motion space for high-quality prediction, we build a memory-based dictionary, which aims to preserve the global motion patterns in training data to guide the predictions.We evaluate the proposed method on two challenging benchmark datasets (Human3.6M and CMU-Mocap). Experimental results show our superior performance compared with the state-of-the-art approaches. |
| first_indexed | 2025-02-19T03:57:13Z |
| format | Conference Paper |
| id | ntu-10356/144139 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2025-02-19T03:57:13Z |
| publishDate | 2020 |
| record_format | dspace |
| spelling | ntu-10356/1441392020-10-17T20:11:42Z Learning progressive joint propagation for human motion prediction Cai, Yujun Huang, Lin Wang, Yiwei Cham, Tat-Jen Cai, Jianfei Yuan, Junsong Liu, Jun Yang, Xu Zhu, Yiheng Shen, Xiaohui Liu, Ding Liu, Jing Thalmann, Nadia Magnenat School of Computer Science and Engineering European Conference on Computer Vision (ECCV) Institute for Media Innovation (IMI) Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence 3D Motion Prediction Transformer Network Despite the great progress in human motion prediction, it remains a challenging task due to the complicated structural dynamics of human behaviors. In this paper, we address this problem in three aspects. First, to capture the long-range spatial correlations and temporal dependencies, we apply a transformer-based architecture with the global attention mechanism. Speci cally, we feed the network with the sequential joints encoded with the temporal information for spatial and temporal explorations. Second, to further exploit the inherent kinematic chains for better 3D structures, we apply a progressive-decoding strategy, which performs in a central-to-peripheral extension according to the structural connectivity. Last, in order to incorporate a general motion space for high-quality prediction, we build a memory-based dictionary, which aims to preserve the global motion patterns in training data to guide the predictions.We evaluate the proposed method on two challenging benchmark datasets (Human3.6M and CMU-Mocap). Experimental results show our superior performance compared with the state-of-the-art approaches. National Research Foundation (NRF) Accepted version This research / project is supported by the National Research Foundation, Singapore under its International Research Centres in Singapore Funding Initiative. Any opinions, fndings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. This research is partially supported by the Monash FIT Start-up Grant, start-up funds from University at Buffalo and SUTD project PIE-SGP-Al-2020-02. 2020-10-15T06:20:12Z 2020-10-15T06:20:12Z 2020 Conference Paper Cai, Y., Huang, L., Wang, Y., Cham, T.-J., Cai, J., Yuan, J., ... Thalmann, N. M. (2020). Learning progressive joint propagation for human motion prediction. European Conference on Computer Vision (ECCV). https://hdl.handle.net/10356/144139 en © 2020 Springer Nature Switzerland AG. This is a post-peer-review, pre-copyedit version of a conference paper published in European Conference on Computer Vision (ECCV). application/pdf |
| spellingShingle | Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence 3D Motion Prediction Transformer Network Cai, Yujun Huang, Lin Wang, Yiwei Cham, Tat-Jen Cai, Jianfei Yuan, Junsong Liu, Jun Yang, Xu Zhu, Yiheng Shen, Xiaohui Liu, Ding Liu, Jing Thalmann, Nadia Magnenat Learning progressive joint propagation for human motion prediction |
| title | Learning progressive joint propagation for human motion prediction |
| title_full | Learning progressive joint propagation for human motion prediction |
| title_fullStr | Learning progressive joint propagation for human motion prediction |
| title_full_unstemmed | Learning progressive joint propagation for human motion prediction |
| title_short | Learning progressive joint propagation for human motion prediction |
| title_sort | learning progressive joint propagation for human motion prediction |
| topic | Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence 3D Motion Prediction Transformer Network |
| url | https://hdl.handle.net/10356/144139 |
| work_keys_str_mv | AT caiyujun learningprogressivejointpropagationforhumanmotionprediction AT huanglin learningprogressivejointpropagationforhumanmotionprediction AT wangyiwei learningprogressivejointpropagationforhumanmotionprediction AT chamtatjen learningprogressivejointpropagationforhumanmotionprediction AT caijianfei learningprogressivejointpropagationforhumanmotionprediction AT yuanjunsong learningprogressivejointpropagationforhumanmotionprediction AT liujun learningprogressivejointpropagationforhumanmotionprediction AT yangxu learningprogressivejointpropagationforhumanmotionprediction AT zhuyiheng learningprogressivejointpropagationforhumanmotionprediction AT shenxiaohui learningprogressivejointpropagationforhumanmotionprediction AT liuding learningprogressivejointpropagationforhumanmotionprediction AT liujing learningprogressivejointpropagationforhumanmotionprediction AT thalmannnadiamagnenat learningprogressivejointpropagationforhumanmotionprediction |