San Francisco research lab OpenAI has introduced Jukebox, a neural net that synthesizes music in a variety of artists’ styles, including rudimentary singing.
They’ve trained the AI with a wide range of artists’ music, including Elvis, Frank Sinatra, Katy Perry and others. The AI directly synthesizes audio, with instruments and faux-vocals.
The results are terrible and awesome, sounding like music from another universe, captured via a vintage tube radio.
Some of the tracks seem like plausible bad songwriting, like faux-Katy Perry singing:
“I count every moment, every hour since I said goodbye.
I count every minute every hour, since your lips were touching mine.
I count every minute, every hour, hoping I’m the one you want.”
Pop, in the style of Katy Perry:
Other tracks seem like something out of a David Lynch dream sequence, like a drunken Frank Sinatra singing “It’s Christmas time, and you know what that means. It’s hot tub time!”
All this needs to be a warped holiday classic is a deep-fake video of Sinatra crooning “It’s hot tub time!” with a martini in his hand.
In addition to synthesizing tracks with original lyrics, the AI can create disturbing new music based on classic lyrics.
The results are nowhere near achieving uncanny-valley near-realism. But listening to ‘faux Katy Perry’ or ‘drunk Frank Sinatra’, it’s clear that the AI is capturing some characteristic aspects of their performance styles.
In addition to doing imitative synthesis of artists’ performances, Jukebox also features recordings that take the AI’s training and apply it in unusual ways.
For example, you may know Marni Nixon’s version of I Whistle a Happy Tune, a feel-good song from the Rodgers & Hammerstein musical The King & I.
The upbeat lyrics sound completely different, though, when paired with AI-generated music and vocals.
Here’s faux drunk Sinatra, with his take on I Whistle a Happy Tune:
The researchers are also exploring creating tracks that combine models, so 0.25 Ella Fitzgerald + 0.75 Frank Sinatra can sing an original song, based on the lyrics to La La Land’s City Of Stars:
Jukebox is a bizarre rabbit hole to fall down – but also a preview of the insane possibilities of a new type of synthesis.
Check out the examples and share your thoughts on AI song synthesis in the comments!