Music Style Transfer: A Position Paper, proposed by Prof. Gus. Xia, etc.
https://arxiv.org/abs/1803.06841

University of Michigan - Ann Arbor

Shortly music style transfer means combining two different styles of music into new one. 

Music Style Transfer 

1. Score Representation
2. Performance Control Representation
3. Sound Representation

Different Representations of Music

1. Timbre Style Transfer (sound)
2. Performance Style Transfer (control)
3. Composition Style Transfer (score)

Music Style Transfer For Different Music Representations

1. Create covers for songs
2. Bring pleasure to humans 
3. Understand the key characteristics of the music. 
4. Create new music that is both human like and creative.
5. "One can only imagine the amount of effort that goes into arranging an entire symphony based on a comparatively simple rock song. Domain
transfer systems could significantly accelerate this process, or
even automate it completely, which would let us enjoy music
that generally has not been feasible to create on a large scale." 

Purpose 
(Music Style Transfer)

1. The term "music style" is difficult to be defined. 
2. Music has multi-level, multi-modal representation. 
3. Existing studies can still hardly achieve multi-modal and non-deterministic mapping between different domains. However, when we transfer a piano solo piece into guitar solo, we often expect the outcome of the guitar solo to be adjustable, perhaps with various ﬁngering styles, brightness, musical texture, or other sound quality.

Challenges
(Music Style Transfer)

There are three basic music model constructing foundations:
1. Conditions or labels could be added so as to realize explicit control. And we could try to maximize the "mutual information" introduced by infoGAN to train our model.
2. In Deep Variational Model or Seq2seq models, the high-dimensional representation of music and the styles are decoupled, and the styles are replaced with other style representations so as to realize the music style tranfer.
3. It's actually a subset of intuition 2. However, it's widely used in timbre transfer models. As a result, it's separately introduced here. We could transfer our music to pics and use methods based on neural style transfer to realize the music style transfer.

Basic Intuitions of Music Transfer Model

A Universal Music Translation Network (Facebook AI Research, 2018) is a deep-learning model for music style transfer. It uses a shared encoder $$E$$ together with per-domain WaveNet decoders $$D_j$$ to transfer timbre and playing style (e.g., between jazz, traditional, and other genres) while preserving the underlying musical content. Training combines a classification loss, which pushes the encoder toward a domain-confused (style-invariant) representation via a domain classifier $$C$$, with a reconstruction loss that keeps each decoder's output faithful to the original input. This combination showed that musical style can be disentangled from content, and the model is widely cited as an influential baseline for music style transfer.

A Universal Music Translation Network

Style transfer is a computational technique that applies the stylistic characteristics of one medium—such as an image, video, voice, or musical piece—to a different input content. While image style transfer has achieved widespread success, adapting these techniques to other modalities often presents additional challenges.

Style Transfer

This node organizes papers, software, and other resources related to music style transfer.

1. https://arxiv.org/pdf/1803.06841.pdf
2. https://arxiv.org/pdf/1809.07575.pdf
3. https://github.com/imalikshake/StyleNet
4. https://arxiv.org/pdf/1910.03713v2.pdf
5. https://arxiv.org/pdf/1708.03535.pdf
6. https://lib.ugent.be/fulltxt/RUG01/002/785/848/RUG01-002785848_2019_0001_AC.pdf
7. https://software.intel.com/content/www/us/en/develop/articles/neural-style-transfer-on-audio-signals.html
8. https://ieeexplore.ieee.org/document/9178446

Learn Before

Related