MRBERT: Pre-Training of Melody and Rhythm for Automatic Music Generation
Deep learning technology has been extensively studied for its potential in music, notably for creative music generation research. Traditional music generation approaches based on recurrent neural networks cannot provide satisfactory long-distance dependencies. These approaches are typically designed...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/4/798 |
| _version_ | 1797619510031679488 |
|---|---|
| author | Shuyu Li Yunsick Sung |
| author_facet | Shuyu Li Yunsick Sung |
| author_sort | Shuyu Li |
| collection | DOAJ |
| description | Deep learning technology has been extensively studied for its potential in music, notably for creative music generation research. Traditional music generation approaches based on recurrent neural networks cannot provide satisfactory long-distance dependencies. These approaches are typically designed for specific tasks, such as melody and chord generation, and cannot generate diverse music simultaneously. Pre-training is used in natural language processing to accomplish various tasks and overcome the limitation of long-distance dependencies. However, pre-training is not yet widely used in automatic music generation. Because of the differences in the attributes of language and music, traditional pre-trained models utilized in language modeling cannot be directly applied to music fields. This paper proposes a pre-trained model, MRBERT, for multitask-based music generation to learn melody and rhythm representation. The pre-trained model can be applied to music generation applications such as web-based music composers that includes the functions of melody and rhythm generation, modification, completion, and chord matching after being fine-tuned. The results of ablation experiments performed on the proposed model revealed that under the evaluation metrics of HITS@k, the pre-trained MRBERT considerably improved the performance of the generation tasks by 0.09–13.10% and 0.02–7.37%, compared to the usage of RNNs and the original BERT, respectively. |
| first_indexed | 2024-03-11T08:28:02Z |
| format | Article |
| id | doaj.art-1031932209884e2cad4e715f446afe2f |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-11T08:28:02Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-1031932209884e2cad4e715f446afe2f2023-11-16T21:54:12ZengMDPI AGMathematics2227-73902023-02-0111479810.3390/math11040798MRBERT: Pre-Training of Melody and Rhythm for Automatic Music GenerationShuyu Li0Yunsick Sung1Department of Multimedia Engineering, Graduate School, Dongguk University–Seoul, Seoul 04620, Republic of KoreaDepartment of Multimedia Engineering, Dongguk University–Seoul, Seoul 04620, Republic of KoreaDeep learning technology has been extensively studied for its potential in music, notably for creative music generation research. Traditional music generation approaches based on recurrent neural networks cannot provide satisfactory long-distance dependencies. These approaches are typically designed for specific tasks, such as melody and chord generation, and cannot generate diverse music simultaneously. Pre-training is used in natural language processing to accomplish various tasks and overcome the limitation of long-distance dependencies. However, pre-training is not yet widely used in automatic music generation. Because of the differences in the attributes of language and music, traditional pre-trained models utilized in language modeling cannot be directly applied to music fields. This paper proposes a pre-trained model, MRBERT, for multitask-based music generation to learn melody and rhythm representation. The pre-trained model can be applied to music generation applications such as web-based music composers that includes the functions of melody and rhythm generation, modification, completion, and chord matching after being fine-tuned. The results of ablation experiments performed on the proposed model revealed that under the evaluation metrics of HITS@k, the pre-trained MRBERT considerably improved the performance of the generation tasks by 0.09–13.10% and 0.02–7.37%, compared to the usage of RNNs and the original BERT, respectively.https://www.mdpi.com/2227-7390/11/4/798automatic music generationgenerative pre-trainingembeddingrepresentation learning |
| spellingShingle | Shuyu Li Yunsick Sung MRBERT: Pre-Training of Melody and Rhythm for Automatic Music Generation Mathematics automatic music generation generative pre-training embedding representation learning |
| title | MRBERT: Pre-Training of Melody and Rhythm for Automatic Music Generation |
| title_full | MRBERT: Pre-Training of Melody and Rhythm for Automatic Music Generation |
| title_fullStr | MRBERT: Pre-Training of Melody and Rhythm for Automatic Music Generation |
| title_full_unstemmed | MRBERT: Pre-Training of Melody and Rhythm for Automatic Music Generation |
| title_short | MRBERT: Pre-Training of Melody and Rhythm for Automatic Music Generation |
| title_sort | mrbert pre training of melody and rhythm for automatic music generation |
| topic | automatic music generation generative pre-training embedding representation learning |
| url | https://www.mdpi.com/2227-7390/11/4/798 |
| work_keys_str_mv | AT shuyuli mrbertpretrainingofmelodyandrhythmforautomaticmusicgeneration AT yunsicksung mrbertpretrainingofmelodyandrhythmforautomaticmusicgeneration |