Foley Music: Learning to Generate Music from Videos
In this paper, we introduce Foley Music, a system that can synthesize plausible music for a silent video clip about people playing musical instruments. We first identify two key intermediate representations for a successful video to music generator: body keypoints from videos and MIDI events from au...
Main Authors: | Gan, Chuang, Huang, Deng, Chen, Peihao, Tenenbaum, Joshua B, Torralba, Antonio |
---|---|
Other Authors: | MIT-IBM Watson AI Lab |
Format: | Book |
Language: | English |
Published: |
Springer International Publishing
2021
|
Online Access: | https://hdl.handle.net/1721.1/130350 |
Similar Items
-
Music Gesture for Visual Sound Separation
by: Gan, Chuang, et al.
Published: (2021) -
Dance2Music-Diffusion: leveraging latent diffusion models for music generation from dance videos
by: Chaoyang Zhang, et al.
Published: (2024-09-01) -
Hyperbolic Music Transformer for Structured Music Generation
by: Wenkai Huang, et al.
Published: (2023-01-01) -
Controllable music : supervised learning of disentangled representations for music generation
by: Watcharasupat, Karn N.
Published: (2021) -
Multimodal Music Genre Classification of Sotho-Tswana Musical Videos
by: Osondu E. Oguike, et al.
Published: (2025-01-01)