LSTM-Based Framework for the Synthesis of Original Soundtracks
Recently, significant developments have been made in Long Short-Term Memory (LSTM) networks within the realm of synthesis music. Notwithstanding these advancements, several challenges persist warranting further research. Primarily, there exists an absence of dedicated research on the application of...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10458114/ |
| _version_ | 1797269098673995776 |
|---|---|
| author | Yuanzhi Huo Mengjie Jin Sicong You |
| author_facet | Yuanzhi Huo Mengjie Jin Sicong You |
| author_sort | Yuanzhi Huo |
| collection | DOAJ |
| description | Recently, significant developments have been made in Long Short-Term Memory (LSTM) networks within the realm of synthesis music. Notwithstanding these advancements, several challenges persist warranting further research. Primarily, there exists an absence of dedicated research on the application of LSTM networks for the synthesis of Original Sound Tracks (OST). Secondly, in general, people can only judge whether the synthesized music meets their expectations based on the model output. However, due to the time-consuming of training the model may need to try multiple times to obtain successful training results. Moreover, the subjective of music quality evaluation relying on human perception, not only the result of model training. To address these multifaceted challenges, this paper concentrates specifically on OST and proposes a framework termed the OST Synthesis Framework (OSTSF) utilizing LSTM. This framework accepts various OST types as input, processed through LSTM to yield innovative OST. Additionally, a novel preprocessing algorithm is proposed to screen input OST elements such as notes and chords, enabling control over music type and quality before the training phase. This algorithm serves to mitigate training uncertainties and reduce situations that require repeated training. Besides, a postprocessing approach, leveraging mathematical formulations facilitates the evaluation of synthesis OST also proposed. This approach aims to quantify subjective evaluations, providing a more intuitive representation through scoring metrics. Experiment results reveal that the OSTSF synthesized OST received favorable rate among a cohort of 100 surveyed respondents attaining 78.8%, demonstrating the efficacy of the proposed framework in the realm of music synthesis utilizing LSTM. |
| first_indexed | 2024-04-25T01:42:59Z |
| format | Article |
| id | doaj.art-2f7e28d61031405dad0f657765291829 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-25T01:42:59Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-2f7e28d61031405dad0f6577652918292024-03-08T00:00:27ZengIEEEIEEE Access2169-35362024-01-0112338323384210.1109/ACCESS.2024.337258110458114LSTM-Based Framework for the Synthesis of Original SoundtracksYuanzhi Huo0https://orcid.org/0009-0009-1695-8903Mengjie Jin1Sicong You2College of Information Engineering, Henan University of Science and Technology, Luoyang, ChinaSchool of Mathematics and Statistics, Henan University of Science and Technology, Luoyang, ChinaCollege of Food Science and Technology, Nanjing Agricultural University, Nanjing, ChinaRecently, significant developments have been made in Long Short-Term Memory (LSTM) networks within the realm of synthesis music. Notwithstanding these advancements, several challenges persist warranting further research. Primarily, there exists an absence of dedicated research on the application of LSTM networks for the synthesis of Original Sound Tracks (OST). Secondly, in general, people can only judge whether the synthesized music meets their expectations based on the model output. However, due to the time-consuming of training the model may need to try multiple times to obtain successful training results. Moreover, the subjective of music quality evaluation relying on human perception, not only the result of model training. To address these multifaceted challenges, this paper concentrates specifically on OST and proposes a framework termed the OST Synthesis Framework (OSTSF) utilizing LSTM. This framework accepts various OST types as input, processed through LSTM to yield innovative OST. Additionally, a novel preprocessing algorithm is proposed to screen input OST elements such as notes and chords, enabling control over music type and quality before the training phase. This algorithm serves to mitigate training uncertainties and reduce situations that require repeated training. Besides, a postprocessing approach, leveraging mathematical formulations facilitates the evaluation of synthesis OST also proposed. This approach aims to quantify subjective evaluations, providing a more intuitive representation through scoring metrics. Experiment results reveal that the OSTSF synthesized OST received favorable rate among a cohort of 100 surveyed respondents attaining 78.8%, demonstrating the efficacy of the proposed framework in the realm of music synthesis utilizing LSTM.https://ieeexplore.ieee.org/document/10458114/Deep learningLSTMmachine learningmusic synthesisRNNsequence prediction |
| spellingShingle | Yuanzhi Huo Mengjie Jin Sicong You LSTM-Based Framework for the Synthesis of Original Soundtracks IEEE Access Deep learning LSTM machine learning music synthesis RNN sequence prediction |
| title | LSTM-Based Framework for the Synthesis of Original Soundtracks |
| title_full | LSTM-Based Framework for the Synthesis of Original Soundtracks |
| title_fullStr | LSTM-Based Framework for the Synthesis of Original Soundtracks |
| title_full_unstemmed | LSTM-Based Framework for the Synthesis of Original Soundtracks |
| title_short | LSTM-Based Framework for the Synthesis of Original Soundtracks |
| title_sort | lstm based framework for the synthesis of original soundtracks |
| topic | Deep learning LSTM machine learning music synthesis RNN sequence prediction |
| url | https://ieeexplore.ieee.org/document/10458114/ |
| work_keys_str_mv | AT yuanzhihuo lstmbasedframeworkforthesynthesisoforiginalsoundtracks AT mengjiejin lstmbasedframeworkforthesynthesisoforiginalsoundtracks AT sicongyou lstmbasedframeworkforthesynthesisoforiginalsoundtracks |