Transformer-Based Seq2Seq Model for Chord Progression Generation
Machine learning is widely used in various practical applications with deep learning models demonstrating advantages in handling huge data. Treating music as a special language and using deep learning models to accomplish melody recognition, music generation, and music analysis has proven feasible....
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/5/1111 |
| _version_ | 1797614881192542208 |
|---|---|
| author | Shuyu Li Yunsick Sung |
| author_facet | Shuyu Li Yunsick Sung |
| author_sort | Shuyu Li |
| collection | DOAJ |
| description | Machine learning is widely used in various practical applications with deep learning models demonstrating advantages in handling huge data. Treating music as a special language and using deep learning models to accomplish melody recognition, music generation, and music analysis has proven feasible. In certain music-related deep learning research, recurrent neural networks have been replaced with transformers. This has achieved significant results. In traditional approaches with recurrent neural networks, input sequences are limited in length. This paper proposes a method to generate chord progressions for melodies using a transformer-based sequence-to-sequence model, which is divided into a pre-trained encoder and decoder. A pre-trained encoder extracts contextual information from melodies, whereas a decoder uses this information to produce chords asynchronously and finally outputs chord progressions. The proposed method addresses length limitation issues while considering the harmony between chord progressions and melodies. Chord progressions can be generated for melodies in practical music composition applications. Evaluation experiments are conducted using the proposed method and three baseline models. The baseline models included the bidirectional long short-term memory (BLSTM), bidirectional encoder representation from transformers (BERT), and generative pre-trained transformer (GPT2). The proposed method outperformed the baseline models in <i>Hits@k</i> (<i>k</i> = 1) by 25.89, 1.54, and 2.13 %, respectively. |
| first_indexed | 2024-03-11T07:18:33Z |
| format | Article |
| id | doaj.art-683868acba4e40939eaa3daa2df41d9d |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-11T07:18:33Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-683868acba4e40939eaa3daa2df41d9d2023-11-17T08:08:22ZengMDPI AGMathematics2227-73902023-02-01115111110.3390/math11051111Transformer-Based Seq2Seq Model for Chord Progression GenerationShuyu Li0Yunsick Sung1Department of Multimedia Engineering, Graduate School, Dongguk University–Seoul, Seoul 04620, Republic of KoreaDepartment of Multimedia Engineering, Dongguk University–Seoul, Seoul 04620, Republic of KoreaMachine learning is widely used in various practical applications with deep learning models demonstrating advantages in handling huge data. Treating music as a special language and using deep learning models to accomplish melody recognition, music generation, and music analysis has proven feasible. In certain music-related deep learning research, recurrent neural networks have been replaced with transformers. This has achieved significant results. In traditional approaches with recurrent neural networks, input sequences are limited in length. This paper proposes a method to generate chord progressions for melodies using a transformer-based sequence-to-sequence model, which is divided into a pre-trained encoder and decoder. A pre-trained encoder extracts contextual information from melodies, whereas a decoder uses this information to produce chords asynchronously and finally outputs chord progressions. The proposed method addresses length limitation issues while considering the harmony between chord progressions and melodies. Chord progressions can be generated for melodies in practical music composition applications. Evaluation experiments are conducted using the proposed method and three baseline models. The baseline models included the bidirectional long short-term memory (BLSTM), bidirectional encoder representation from transformers (BERT), and generative pre-trained transformer (GPT2). The proposed method outperformed the baseline models in <i>Hits@k</i> (<i>k</i> = 1) by 25.89, 1.54, and 2.13 %, respectively.https://www.mdpi.com/2227-7390/11/5/1111chord progression generationtransformersequence-to-sequencepre-training |
| spellingShingle | Shuyu Li Yunsick Sung Transformer-Based Seq2Seq Model for Chord Progression Generation Mathematics chord progression generation transformer sequence-to-sequence pre-training |
| title | Transformer-Based Seq2Seq Model for Chord Progression Generation |
| title_full | Transformer-Based Seq2Seq Model for Chord Progression Generation |
| title_fullStr | Transformer-Based Seq2Seq Model for Chord Progression Generation |
| title_full_unstemmed | Transformer-Based Seq2Seq Model for Chord Progression Generation |
| title_short | Transformer-Based Seq2Seq Model for Chord Progression Generation |
| title_sort | transformer based seq2seq model for chord progression generation |
| topic | chord progression generation transformer sequence-to-sequence pre-training |
| url | https://www.mdpi.com/2227-7390/11/5/1111 |
| work_keys_str_mv | AT shuyuli transformerbasedseq2seqmodelforchordprogressiongeneration AT yunsicksung transformerbasedseq2seqmodelforchordprogressiongeneration |