Double Linear Transformer for Background Music Generation from Videos
Many music generation research works have achieved effective performance, while rarely combining music with given videos. We propose a model with two linear Transformers to generate background music according to a given video. To enhance the melodic quality of the generated music, we firstly input n...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-05-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/10/5050 |
| _version_ | 1827670534252396544 |
|---|---|
| author | Xueting Yang Ying Yu Xiaoyu Wu |
| author_facet | Xueting Yang Ying Yu Xiaoyu Wu |
| author_sort | Xueting Yang |
| collection | DOAJ |
| description | Many music generation research works have achieved effective performance, while rarely combining music with given videos. We propose a model with two linear Transformers to generate background music according to a given video. To enhance the melodic quality of the generated music, we firstly input note-related and rhythm-related music features separately into each Transformer network. In particular, we pay attention to the connection and the independence of music features. Then, in order to generate the music that matches the given video, the current state-of-the-art cross-modal inference method is set up to establish the relationship between visual mode and sound mode. Subjective and objective experiment indicate that the generated background music matches the video well and is also melodious. |
| first_indexed | 2024-03-10T03:23:25Z |
| format | Article |
| id | doaj.art-6f88e849375e400da0dd0440092eeeb9 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T03:23:25Z |
| publishDate | 2022-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-6f88e849375e400da0dd0440092eeeb92023-11-23T09:57:00ZengMDPI AGApplied Sciences2076-34172022-05-011210505010.3390/app12105050Double Linear Transformer for Background Music Generation from VideosXueting Yang0Ying Yu1Xiaoyu Wu2Faculty of Information and Communication Engineering, Communication University of China, Beijing 100024, ChinaFaculty of Information and Communication Engineering, Communication University of China, Beijing 100024, ChinaFaculty of Information and Communication Engineering, Communication University of China, Beijing 100024, ChinaMany music generation research works have achieved effective performance, while rarely combining music with given videos. We propose a model with two linear Transformers to generate background music according to a given video. To enhance the melodic quality of the generated music, we firstly input note-related and rhythm-related music features separately into each Transformer network. In particular, we pay attention to the connection and the independence of music features. Then, in order to generate the music that matches the given video, the current state-of-the-art cross-modal inference method is set up to establish the relationship between visual mode and sound mode. Subjective and objective experiment indicate that the generated background music matches the video well and is also melodious.https://www.mdpi.com/2076-3417/12/10/5050video background music generationmusic feature extractionlinear Transformer |
| spellingShingle | Xueting Yang Ying Yu Xiaoyu Wu Double Linear Transformer for Background Music Generation from Videos Applied Sciences video background music generation music feature extraction linear Transformer |
| title | Double Linear Transformer for Background Music Generation from Videos |
| title_full | Double Linear Transformer for Background Music Generation from Videos |
| title_fullStr | Double Linear Transformer for Background Music Generation from Videos |
| title_full_unstemmed | Double Linear Transformer for Background Music Generation from Videos |
| title_short | Double Linear Transformer for Background Music Generation from Videos |
| title_sort | double linear transformer for background music generation from videos |
| topic | video background music generation music feature extraction linear Transformer |
| url | https://www.mdpi.com/2076-3417/12/10/5050 |
| work_keys_str_mv | AT xuetingyang doublelineartransformerforbackgroundmusicgenerationfromvideos AT yingyu doublelineartransformerforbackgroundmusicgenerationfromvideos AT xiaoyuwu doublelineartransformerforbackgroundmusicgenerationfromvideos |