Investigating pose representations and motion contexts modeling for 3D motion prediction
Predicting human motion from historical pose sequence is crucial for a machine to succeed in intelligent interactions with humans. One aspect that has been obviated so far, is the fact that how we represent the skeletal pose has a critical impact on the prediction results. Yet there is no effort tha...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/162631 |
| _version_ | 1811676635407056896 |
|---|---|
| author | Liu, Zhenguang Wu, Shuang Jin, Shuyuan Liu, Qi Ji, Shouling Lu, Shijian Cheng, Li |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Liu, Zhenguang Wu, Shuang Jin, Shuyuan Liu, Qi Ji, Shouling Lu, Shijian Cheng, Li |
| author_sort | Liu, Zhenguang |
| collection | NTU |
| description | Predicting human motion from historical pose sequence is crucial for a machine to succeed in intelligent interactions with humans. One aspect that has been obviated so far, is the fact that how we represent the skeletal pose has a critical impact on the prediction results. Yet there is no effort that investigates across different pose representation schemes. We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. Moreover, recent approaches build upon off-the-shelf RNN units for motion prediction. These approaches process input pose sequence sequentially and inherently have difficulties in capturing long-term dependencies. In this paper, we propose a novel RNN architecture termed AHMR for motion prediction which simultaneously models local motion contexts and a global context. We further explore a geodesic loss and a forward kinematics loss, which have more geometric significance than the widely employed L2 loss. Interestingly, we applied our method to a range of articulate objects including human, fish, and mouse. Empirical results show that our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency, such as retaining natural human-like motions over 50 seconds predictions. Our codes are released. |
| first_indexed | 2024-10-01T02:24:36Z |
| format | Journal Article |
| id | ntu-10356/162631 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T02:24:36Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | ntu-10356/1626312022-11-02T00:42:18Z Investigating pose representations and motion contexts modeling for 3D motion prediction Liu, Zhenguang Wu, Shuang Jin, Shuyuan Liu, Qi Ji, Shouling Lu, Shijian Cheng, Li School of Computer Science and Engineering Engineering::Computer science and engineering Motion Prediction Motion Context Predicting human motion from historical pose sequence is crucial for a machine to succeed in intelligent interactions with humans. One aspect that has been obviated so far, is the fact that how we represent the skeletal pose has a critical impact on the prediction results. Yet there is no effort that investigates across different pose representation schemes. We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. Moreover, recent approaches build upon off-the-shelf RNN units for motion prediction. These approaches process input pose sequence sequentially and inherently have difficulties in capturing long-term dependencies. In this paper, we propose a novel RNN architecture termed AHMR for motion prediction which simultaneously models local motion contexts and a global context. We further explore a geodesic loss and a forward kinematics loss, which have more geometric significance than the widely employed L2 loss. Interestingly, we applied our method to a range of articulate objects including human, fish, and mouse. Empirical results show that our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency, such as retaining natural human-like motions over 50 seconds predictions. Our codes are released. 2022-11-02T00:42:18Z 2022-11-02T00:42:18Z 2022 Journal Article Liu, Z., Wu, S., Jin, S., Liu, Q., Ji, S., Lu, S. & Cheng, L. (2022). Investigating pose representations and motion contexts modeling for 3D motion prediction. IEEE Transactions On Pattern Analysis and Machine Intelligence, 3139918-. https://dx.doi.org/10.1109/TPAMI.2021.3139918 0162-8828 https://hdl.handle.net/10356/162631 10.1109/TPAMI.2021.3139918 34982672 2-s2.0-85122574743 3139918 en IEEE Transactions on Pattern Analysis and Machine Intelligence © 2021 IEEE. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering Motion Prediction Motion Context Liu, Zhenguang Wu, Shuang Jin, Shuyuan Liu, Qi Ji, Shouling Lu, Shijian Cheng, Li Investigating pose representations and motion contexts modeling for 3D motion prediction |
| title | Investigating pose representations and motion contexts modeling for 3D motion prediction |
| title_full | Investigating pose representations and motion contexts modeling for 3D motion prediction |
| title_fullStr | Investigating pose representations and motion contexts modeling for 3D motion prediction |
| title_full_unstemmed | Investigating pose representations and motion contexts modeling for 3D motion prediction |
| title_short | Investigating pose representations and motion contexts modeling for 3D motion prediction |
| title_sort | investigating pose representations and motion contexts modeling for 3d motion prediction |
| topic | Engineering::Computer science and engineering Motion Prediction Motion Context |
| url | https://hdl.handle.net/10356/162631 |
| work_keys_str_mv | AT liuzhenguang investigatingposerepresentationsandmotioncontextsmodelingfor3dmotionprediction AT wushuang investigatingposerepresentationsandmotioncontextsmodelingfor3dmotionprediction AT jinshuyuan investigatingposerepresentationsandmotioncontextsmodelingfor3dmotionprediction AT liuqi investigatingposerepresentationsandmotioncontextsmodelingfor3dmotionprediction AT jishouling investigatingposerepresentationsandmotioncontextsmodelingfor3dmotionprediction AT lushijian investigatingposerepresentationsandmotioncontextsmodelingfor3dmotionprediction AT chengli investigatingposerepresentationsandmotioncontextsmodelingfor3dmotionprediction |