A causal convolutional neural network for multi-subject motion modeling and generation
Abstract Inspired by the success of WaveNet in multi-subject speech synthesis, we propose a novel neural network based on causal convolutions for multi-subject motion modeling and generation. The network can capture the intrinsic characteristics of the motion of different subjects, such as the influ...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2023-11-01
|
| Series: | Computational Visual Media |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s41095-022-0307-3 |
| _version_ | 1797362988960710656 |
|---|---|
| author | Shuaiying Hou Congyi Wang Wenlin Zhuang Yu Chen Yangang Wang Hujun Bao Jinxiang Chai Weiwei Xu |
| author_facet | Shuaiying Hou Congyi Wang Wenlin Zhuang Yu Chen Yangang Wang Hujun Bao Jinxiang Chai Weiwei Xu |
| author_sort | Shuaiying Hou |
| collection | DOAJ |
| description | Abstract Inspired by the success of WaveNet in multi-subject speech synthesis, we propose a novel neural network based on causal convolutions for multi-subject motion modeling and generation. The network can capture the intrinsic characteristics of the motion of different subjects, such as the influence of skeleton scale variation on motion style. Moreover, after fine-tuning the network using a small motion dataset for a novel skeleton that is not included in the training dataset, it is able to synthesize high-quality motions with a personalized style for the novel skeleton. The experimental results demonstrate that our network can model the intrinsic characteristics of motions well and can be applied to various motion modeling and synthesis tasks. |
| first_indexed | 2024-03-08T16:15:07Z |
| format | Article |
| id | doaj.art-3bd893f996924e05af59539a2860b8f2 |
| institution | Directory Open Access Journal |
| issn | 2096-0433 2096-0662 |
| language | English |
| last_indexed | 2024-03-08T16:15:07Z |
| publishDate | 2023-11-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Computational Visual Media |
| spelling | doaj.art-3bd893f996924e05af59539a2860b8f22024-01-07T12:39:02ZengSpringerOpenComputational Visual Media2096-04332096-06622023-11-01101455910.1007/s41095-022-0307-3A causal convolutional neural network for multi-subject motion modeling and generationShuaiying Hou0Congyi Wang1Wenlin Zhuang2Yu Chen3Yangang Wang4Hujun Bao5Jinxiang Chai6Weiwei Xu7State Key Lab of CAD&CG, Zhejiang UniversityXmovSchool of Automation, Southeast UniversityState Key Lab of CAD&CG, Zhejiang UniversitySchool of Automation, Southeast UniversityState Key Lab of CAD&CG, Zhejiang UniversityXmovState Key Lab of CAD&CG, Zhejiang UniversityAbstract Inspired by the success of WaveNet in multi-subject speech synthesis, we propose a novel neural network based on causal convolutions for multi-subject motion modeling and generation. The network can capture the intrinsic characteristics of the motion of different subjects, such as the influence of skeleton scale variation on motion style. Moreover, after fine-tuning the network using a small motion dataset for a novel skeleton that is not included in the training dataset, it is able to synthesize high-quality motions with a personalized style for the novel skeleton. The experimental results demonstrate that our network can model the intrinsic characteristics of motions well and can be applied to various motion modeling and synthesis tasks.https://doi.org/10.1007/s41095-022-0307-3deep learningoptimizationmotion generationmotion denoisingmotion control |
| spellingShingle | Shuaiying Hou Congyi Wang Wenlin Zhuang Yu Chen Yangang Wang Hujun Bao Jinxiang Chai Weiwei Xu A causal convolutional neural network for multi-subject motion modeling and generation Computational Visual Media deep learning optimization motion generation motion denoising motion control |
| title | A causal convolutional neural network for multi-subject motion modeling and generation |
| title_full | A causal convolutional neural network for multi-subject motion modeling and generation |
| title_fullStr | A causal convolutional neural network for multi-subject motion modeling and generation |
| title_full_unstemmed | A causal convolutional neural network for multi-subject motion modeling and generation |
| title_short | A causal convolutional neural network for multi-subject motion modeling and generation |
| title_sort | causal convolutional neural network for multi subject motion modeling and generation |
| topic | deep learning optimization motion generation motion denoising motion control |
| url | https://doi.org/10.1007/s41095-022-0307-3 |
| work_keys_str_mv | AT shuaiyinghou acausalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT congyiwang acausalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT wenlinzhuang acausalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT yuchen acausalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT yangangwang acausalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT hujunbao acausalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT jinxiangchai acausalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT weiweixu acausalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT shuaiyinghou causalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT congyiwang causalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT wenlinzhuang causalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT yuchen causalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT yangangwang causalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT hujunbao causalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT jinxiangchai causalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration AT weiweixu causalconvolutionalneuralnetworkformultisubjectmotionmodelingandgeneration |