Building a large-scale dataset for audio-conditioned dance motion synthesis
Generative models for audio-conditioned dance motion synthesis map music features to dance movements. Models are trained with a few assumptions such as strong music-dance correlation, controlled motion data and relatively simple poses. These characteristics are found in all existing datasets for dan...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Master by Research |
| Language: | English |
| Published: |
Nanyang Technological University
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/160410 |
| _version_ | 1811694914733342720 |
|---|---|
| author | Wu, Jinyi |
| author2 | Chen Change Loy |
| author_facet | Chen Change Loy Wu, Jinyi |
| author_sort | Wu, Jinyi |
| collection | NTU |
| description | Generative models for audio-conditioned dance motion synthesis map music features to dance movements. Models are trained with a few assumptions such as strong music-dance correlation, controlled motion data and relatively simple poses. These characteristics are found in all existing datasets for dance motion synthesis, and indeed recent methods can achieve good results.
We introduce a new dataset aiming to challenge these common assumptions. We focus on breakdancing which features acrobatic moves and tangled postures. We source our data from the Red Bull BC One competition videos and adopt a hybrid labelling pipeline leveraging deep estimation models as well as manual annotations to obtain good quality keypoint sequences at a reduced cost. Our dataset can readily foster advance in dance motion synthesis. With intri- cate poses and swift movements, models are forced to go beyond learning a mapping between modalities and reason more effectively about body structure and movements. |
| first_indexed | 2024-10-01T07:15:09Z |
| format | Thesis-Master by Research |
| id | ntu-10356/160410 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T07:15:09Z |
| publishDate | 2022 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1604102022-08-01T05:07:19Z Building a large-scale dataset for audio-conditioned dance motion synthesis Wu, Jinyi Chen Change Loy School of Computer Science and Engineering ccloy@ntu.edu.sg Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Generative models for audio-conditioned dance motion synthesis map music features to dance movements. Models are trained with a few assumptions such as strong music-dance correlation, controlled motion data and relatively simple poses. These characteristics are found in all existing datasets for dance motion synthesis, and indeed recent methods can achieve good results. We introduce a new dataset aiming to challenge these common assumptions. We focus on breakdancing which features acrobatic moves and tangled postures. We source our data from the Red Bull BC One competition videos and adopt a hybrid labelling pipeline leveraging deep estimation models as well as manual annotations to obtain good quality keypoint sequences at a reduced cost. Our dataset can readily foster advance in dance motion synthesis. With intri- cate poses and swift movements, models are forced to go beyond learning a mapping between modalities and reason more effectively about body structure and movements. Master of Engineering 2022-07-21T06:51:46Z 2022-07-21T06:51:46Z 2022 Thesis-Master by Research Wu, J. (2022). Building a large-scale dataset for audio-conditioned dance motion synthesis. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/160410 https://hdl.handle.net/10356/160410 10.32657/10356/160410 en This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Wu, Jinyi Building a large-scale dataset for audio-conditioned dance motion synthesis |
| title | Building a large-scale dataset for audio-conditioned dance motion synthesis |
| title_full | Building a large-scale dataset for audio-conditioned dance motion synthesis |
| title_fullStr | Building a large-scale dataset for audio-conditioned dance motion synthesis |
| title_full_unstemmed | Building a large-scale dataset for audio-conditioned dance motion synthesis |
| title_short | Building a large-scale dataset for audio-conditioned dance motion synthesis |
| title_sort | building a large scale dataset for audio conditioned dance motion synthesis |
| topic | Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision |
| url | https://hdl.handle.net/10356/160410 |
| work_keys_str_mv | AT wujinyi buildingalargescaledatasetforaudioconditioneddancemotionsynthesis |