Machine Learning for Creativity and Design Workshop (NeurIPS2018), and +@

Following their first workshop last year, there was the second ML4 Creativity and Design workshop on 8th Dec 2018 at Neurips2018 (=one of the biggest machine learning conferences), Montreal (=one of the coldest area I’ve ever been). It was great! And even greater for those who are interested in music. I missed the last year’s one but seems like there were more musical stuff this year than before. Here’s my summary for the workshop, a non-exhaustive and mostly musical one, but please treat yourself with other papers too. Ok, here we go.

1. Music-related works

Screen Shot 2018-12-10 at 9.12.35 AM.png

  • “Neural wavetable: a playable wavetable synthesizer using neural networks”
    • By Lamtharn Hantrakul and Li-Chia Yang (Google Brain residency and Georgia Tech)
    • To generate an wavetable, which is a (data)base for a certain type of synthesizer (wavetable synthesizer, obviously), they used WaveNet + AutoEncoder so that by controlling the latent space (hidden representation of AutoEncoder) the waveforms of the table can be manipulated continuously.




  • (continued)
    • Compared to MusicVAE, multitrack VAE is..
      • still with a global z over time, but this time z has multitrack information encoded
      • and with chord conditioned for each bar.

Screen Shot 2018-12-10 at 12.56.36 PM

2018-12-08 13.28.582018-12-08 13.28.422018-12-08 13.28.47


2018-12-08 13.42.30

  • “Transformer-NADE for piano performances”
    • by Curtis Hawthorne et al. (Google Magenta)
    • proposed to use NADE (neural autoregressive distribution estimator) to predict the following note and the dimension is an element of note tuple — the elements are properties of note (onset timing, duration, ..).
    • FYI “Transformer” is a purely attention-based sequence-to-sequence model, originally proposed for language translation, recently used for symbolic music generation

2018-12-08 14.08.462018-12-08 14.11.372018-12-08 14.11.43


  • Piano Ginie


2. Some others

  • “Artistic Influence GAN
    • by Eric Chu, MIT Media Lab
    • “What if Banksy had met Jackson Pollock during his formative years, or if David Hockney had missed out on the Tate Gallery’s famous 1960 Picasso exhibition?”
    • Similar to one thing that I’ve always thought about — to simulate the history of music, maybe with RL though.

2018-12-08 13.34.482018-12-08 13.34.58

3. An awesome talk

  • Michael Levin’s Keynote titled “What bodies think about: Bioelectric computation outside the nervous system, primitive cognition, and synthetic morphology” totally blew many’s minds, I think it could be one that excited the NeurIPS 2018 participants the most — and you don’t need to be a deep learning research to get impressed.

4. Neither musical or talk (i.e. the usual NeurIPS stuff)

  • Best GAN ever

  • VAE + GAN

Ok this is it. Thanks for the great works everyone!

One thought on “Machine Learning for Creativity and Design Workshop (NeurIPS2018), and +@

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s