If There Was A Turing Test For Music Artificial Intelligence, ‘Kulitta’ Might Pass It


A key concept in the world of artificial intelligence is the Turing Test – the idea that a computer exhibits intelligence when a natural language text discussion with it becomes indistinguishable from a text discussion with a human.

That idea, introduced by computer scientist Alan Turing, has sparked the imagination of many, been a recurring topic of fiction and film (like the recent Ex Machina, above) and led to alternative Turing tests that attempt to identify intelligence in different ways, ranging from chess playing ability to problem solving to creativity.

If there was a Turing Test for musical artificial intelligence, a new music composition application, Kulitta, might pass it.


Kulitta is a framework for automated composition, created by Donya Quick at Yale, that can be configured to run as a standalone artificial intelligence for generating music in a particular style.

A central idea to Kulitta’s approach is the notion of abstraction: the idea that something can be described at many different levels of detail. Music has many levels of abstraction, ranging from the sound we hear to a paper score and large-scale structural patterns.

Music is also very multidimensional and prone to tractability problems. Kulitta works at many of levels of abstraction in stages as a way to mitigate these inherent complexity problems. Dealing with these layers of musical abstraction depends on creating grammars for modeling and working with music in new ways.

All this would be irrelevant, though, if the results weren’t musical. Here’s an example of one of Kulitta’s compositions, Etude for piano:

Kulitta’s Etude is certainly musical – it sounds like music we’ve heard before and has immediately recognizable forms that make it clear that this isn’t just random notes or chord progressions.

But are the results really the result of a artificial musical intelligence – or just a computer program that can follow instructions well?

That’s where the idea of a ‘musical turing test’ comes in.

Here’s some additional background on Kulitta:

A summary of Kulitta by Yale News suggests that Quick’s application has already passed a basic musical Turing test:

In two separate tests, each involving more than 100 human subjects of varied musical experience, participants listened to 40 short musical phrases, some written by humans, others by computer programs, including Quick’s, which she calls Kulitta.

The subjects were asked to rate the musical phrases on a seven-point scale ranging from “absolutely human” to “absolutely computer.”

In both tests, Kulitta’s compositions rated, on average, on the human side of the scale.

This test, which rated the ‘humanness’ of short musical phrases, is setting a low bar for musical intelligence. Any programmer with a musical background should be able to create a program that can do basic musical tasks efficiently, like generating melodic phrases that are usable, if not inspired. And Quick has clearly gone far beyond that, creating an application that can generate musical works.

A true Turing Test for music artificial intelligence needs to look further – at whether a program can independently create music that can compete with human music, in the ways that human music has to compete for your attention:

  • Can it marry words and music together to create a basic pop song that can hit the top 40?
  • Can it create compositions that win traditional classical composition contests?
  • Can it get ‘liked’ on Spotify or similar services similarly to human music?

Kulitta is far from passing these sort of tests – but is still a very impressive example of the progress that is being made in understanding how musical creation works. You can find out more about Kulitta at Donya Quick’s site.

What do you think a valid Turing Test for musical artificial intelligence would be? Leave a comment and share your thoughts!

27 thoughts on “If There Was A Turing Test For Music Artificial Intelligence, ‘Kulitta’ Might Pass It

  1. is this actually praise for AI, or is this a critique on the level we’ve allowed popular music to stoop to? We’ve gotten to a point where we accept that nobody can actually sing anymore, since we auto-tune everything. nobody can play a song all the way through so we record a few bars and then loop them. programmed beats, preset settings, EDM’s tedious influence on everything ( i heard a rap track the other day that was basically an EDM song with different sound settings)

    no, i say that artificial intelligence hasnt actually gotten good at writing music. i say we actually have just come to accept what we’re hearing as “human” since we have no viable alternatives to refute the argument.

  2. The dynamics of the phrases were pretty sensitive, but it wasn’t clear if we were hearing a human performance of a Kulitta composition.

    I thought it was harmonically pretty disjointed, but not jarringly so. I’d call it clunky- a bit like how conversation with Siri is. Ok, that’s not fair. It’s way more human feeling than Siri– who most will admit is a pretty stupid excuse for a person. Good thing she doesn’t have an emotional subroutine, yet.

    There was some person who had made a thing that generated Bach-like material that was a little more impressive.

    As to the topic of whether or not this is useful, worthwhile, interesting, or inherently good/evil; programming a robot to make music is a thing we humans seem to like to do. This is just that. Dance, robot, dance!

  3. Awesome! I love technology and this is just incredible. Can’t wait until the day that I can finally simply upload my brain into a virtual room full of vintage synths 🙂

  4. A program will never have a physical body that can move around in space in time interact with others, get shaped by an environment, thus will in my current opinion never be able to be intelligent. It may simulate to such a certain degree to which we will not be able to distinguish the program from the intelligent but it always will be only a mechanism.

    So my idea is we don’t need a test at all.

    AI is Sci-Fi

    My opinion is based on the works of David Cope’s EMI(Experiments in Musical intelligence) ; D. Hofstadter’s Gödel Escher Bach; Ghost in the Shell 1

    1. i agree that sensory information is needed for cognition. in a lot of ways autonomic nervous system is more ‘intelligent’ anyway, rational brain lets us do stupid things.

      a program is written by a human that does all of those things. no program of any sophistication is made in one go without testing and feedback and bugs and discovering new things as you go and changing it. programs get user feedback and are changed by humans constantly.

      saying this is ai is ridiculous to me, it’s like saying people that wear glasses are cyborgs because they’re using technology to augment vision.

      ai would be closer a program that can write programs that it wasn’t programed to write. we’re so far away from that. qualities like taste and introspection, intuition, that we don’t even understand are needed. we can’t program what we don’t know, although we get use computers to test out guess to further our understanding.

      what computer technology teaches us is that the popular group think idea of intelligence is stupid and depends on context and history.

      algorithmic music as a starting point i do a lot. computer lets you make more mistakes more quickly and learn more, allows you to manage more interactions at once and at least give you a rough idea in closer to real time or let you pause and come back to ideas after letting your brain work on it for a while.

      computer is augmentation of human memory useful offering different point of view of entirely human ideas.

      until plants program computers there will never be ‘artificial’ intelligence.

      1. The key issue at play here is a confusion with ‘intelligence’ and ‘artificial intelligence’. If it were free thinking and composing tunes on a whim then that isn’t ‘artificial intelligence’ that is ‘intelligence’ – if it be man made or not.

        So you are really saying, the day a device proves to me it is truly intelligent is the day I will believe in ‘artificial intelligence’, but on that day that device has proved it is more that a mere AI box – it has free thinking and is truly intelligent, and not artificially so. An AI device just gives out an illusion designed to trick and fool the user, an intelligent device is not playing those tricks – it is just being.

        The Turning test is a poor test designed for AI boxes, to see if they can fake it – which ultimately isn’t anyone’s goal – and an easy goal at that. The real goal being to design a box that will devise a test to work out if the users are real or not – that would be truly intelligent. And success or failure for such a machine isn’t in the execution of the plan, but the fact it will devise such a plan, and more importantly, that this machine is concerned enough to be driven to do so.

        Do a voice search on Google for ‘Cats with hats?’ that is serious AI, and works amazingly well, like with self-driven cars. Next year Qualcomm will be putting Neural Processing Units (NPU) in mobile phones, AI works and is all around us, it doesn’t need to be proving itself to anyone. Turning tests should be left to the 1940’s, we need to devise an ‘intelligence test’, and not an AI test. I don’t see the point of testing an artificially intelligent machine if it can fool you that it is intelligent? Are we saying the ultimate intelligence test is to get away with being dishonest? Why set such a goal for AI? What do we do once it is achieved? Teach it to lie better? Well, that’s just asking for a machine overlord.

  5. Before install Kulitta, please run a virus scan, and be sure the software won’t take over your host machine and enslave human kind.

  6. My browser just prompted this msg after the download: “Kulitta_2.0.exe” is not commonly downloaded and could be dangerous. COULD BE DANGEROUS!!

  7. It’s apparent that, in the near future, common music will be generated automatically and endlessly from end user devices.

  8. The topic of AI is overblown. The basic insect level intelligence we are currently capable of manufacturing is many years away from even the most basic mammalian level intelligence capabilities. The rest is P.R., smoke and mirrors. You’re not going to get anywhere near an advanced mammalian intelligence without major advances in quantum computing so you might as well wait. So called AI developers want your money to develop algorithms in a software patent land rush.

    Q: What’s the first job you submit to a quantum computer?
    A: “Design a better version of yourself.”
    That’s when the real barriers start falling like dominoes and the world changes so fast it’ll make all our heads spin and the patent lawyers start to go broke.

    1. While AI may be “overblown”, it is pretty indisputable that it has made a lot of progress. It was far too unrealistic to expect human like assistance in a 10-25 year time frame. However, AI is being used thousands of decision making processes every day. AI will soon be a major part of medical diagnosis and will be superior on some levels to doctors simply because it can wade through huge databases and statistical samples a human cannot.

      Sometimes it is hard to understand the magnitude of change when you are living inside it. PCs are only 40 years old. Cars are about 100. Electrical distribution and powered flight less than 150. Those are huge advances in a relatively short time.

      AI has always had a fantasy component to it. The one where everyone imagines it creating human replacements. But for those in the community it isn’t primarily about replacements but for finding the things that an intelligent machine could do better than an intelligent human while adjusting for outside factors and learning to get better. We ARE at that stage or at least on the precipice of it..

      1. I think in the case of medical expertise we shouldn’t confuse the term AI with Expert System. AI has great marketing value, but Expert System is far closer to the reality of what the software is actually doing.

    2. “The topic of AI is overblown.”

      Meanwhile, there are whole areas of science and computing that rely on artificial intelligence, most people interact with artificial intelligence on a daily basis and artificial intelligent systems can be the smartest humans at a huge range of tasks.

      We’ve gone from mobile phones being phones to being virtual assistants in less than 10 years. Explain to me why you think this type of rapid progress is overblown.

  9. The music is OK, harmony a bit disjointed in places but nothing terrible. A good deal of the human expressive quality seems to be injected by the playing. The important point here though is that this is an imitation of the compositional process based on analysing a ‘corpus’ of pre existing human composed phrases and musical syntax. When computers decide for themselves they want to compose music then we will be closer to what I would consider AI. I wonder though if we would even recognise it as music even if computers did do that one day? The Turing test is based on recognising ourselves and so doesn’t really tell us if a computer is initiating anything at all. The experience of a computer would be so different from ours that anything they expressed through music and the way they chose to express it might be incomprehensible to us.

  10. Not much new here. Stylistic modelling and corpus-based systems have been around for a bit. David Cope’s systems and others have been looked at many times. There are also real-time improvisational systems that listen and react live to players like George Lewis’ Voyager. What we’re entering into is an age of compositionally assistive or interactive music systems and meta-creation. Music has many theoretical avenues and systems which determine how specific compositional styles work. These rules can be programmed to imitate existing styles. Thus, compositions can become corpora for analysis and algorithmic use. It is still music even if it is considered bad. We are still the critics. Our aesthetic values and biases come from many places. In future, what a machine may like may be less appealing to some humans, but it’s still music. Another side to this is that we can learn a lot about what music is by creating generative systems for research. The discussion hasn’t always been about replacing human creativity or creating the singularity. There is much to be learned about AI inside compositional systems. Also, a turing test only goes so far in research and misses a lot of qualitative questions that we have about creativity and aesthetics. That’s partly why recent academic research in this area isn’t always trying to fool the listener and can be more about creative exploration. Some interesting things here. http://metacreation.net

  11. While sleeping last night, every single person I encountered in my dream, essentially, passed the Turing test. Unfortunately I can’t say the same for every person I met today.

    In conclusion, artificial intelligence is indeed a dream.

  12. David Cope made his EMI when he had a composer’s block. I think these tools would be great when used moderately like for supplementing or kickstarting composition. Adding human inputted accents or substituting variations on top of a machine generated base (however boring it may initially sound) could go a long way.

  13. The real test is if the computer simply decides one day to put it’s other tasks on hold and writes some music because it feels like it. Conversely, when instructed to write a composition, it develops writers block or gets distracted by cat videos on the internet.

    1. I think that is a test of inefficiency rather than intelligence.

      The real test would be that you give some machine 20 minutes to stack boxes, and when you come back it has stacked the boxes, sold the goods, and arranged dispatch, while holding an aggressive takeover, as to sack the workforce to then replace every working role with itself – while managing to compose a beautiful song for each individual worker as a farewell gift.

  14. The demo sound-track has that same slightly disjointed feel to it that every automatically composed piece of music has that I’ve heard, it sort of nearly makes sense as a cohesive whole, but not quite. But it’s definitely very good and a big step in the right direction.

Leave a Reply

Your email address will not be published. Required fields are marked *