Nagual Dance Uses Kinect To Translate Movement Into Music

Berlin-based Nagual Sounds has developed a new gestural musical instrument, Nagual Dance, based on the Microsoft Kinect.

The system is based on two main components:

  • Control data – from sensors, cameras, files and all kinds of sources – is used to turn gestures and movement into sounds; and
  • The Soundscape -an interactive piece of music that doesn’t consist of pre-recorded musical sequences, but of musical possibilities. The music decides the behavior of the used data source.

In the example above, a Kinect camera is used by Nagual Dance to control the Soundscape ‘Firedance’.

Here’s how it works:

The Kinect tracks your movements and sends it in form of data to a computer. The software, Nagual Dance, translates this data into music. The process happens in real time!.

The software works with one or two players. Specific sound elements (instruments) are assigned to certain limbs of your body. What you play with it is totally up to you.

The music is created in Soundscapes (interactive music pieces), a new music format. This Soundscape is called Firedance. The Soundscapes are not a set of pre-recorded loops, but are generative music rules that outline the musical possibilities of the composition.


In the video demo, the left player plays the drum elements.

He’s got Percussions on his feet, Kick + Snare on his right hand and Hi-Hat + Shaker on his right hand.

The right player plays the melodies. She’s got chords on her right foot+percussive synths on her left foot, the bass on her right hand and a Pad + Leadon her left hand.

Nagual Dance is still in development, but  Nagual Sounds’ Head Of Design, Georgios Delkos, told us that the company is working on an IndieGoGo campaign to crowdfund the project.

One of the more interesting aspects of this project is the Soundscape format for generative compositions. We’ve followed up to Nagual Sounds to see if we can get more details.

More examples and details are available at the Nagual Sounds site.

22 thoughts on “Nagual Dance Uses Kinect To Translate Movement Into Music

  1. Amusing, but it comes across like a solution looking for a problem. Its only a novelty if there is too little connection between the movement and the WHY of it. Just triggering patterns doesn’t wow me in hardware and likewise with Kinect. The simple fact of the system’s existence doesn’t automatically lead to any creative emotional connection. If someone truly integrated it into a stage play, as one example, then I’d be impressed. If it was plotted out to back “The Tempest” with key effects, it would suddenly have a clear shape and goal. Context matters. If its going to be a dance enhancement, the sounds and changes thereof need to be choreographed to some extent, just like the dance steps. “Free-form” only goes so far. With a little more planning, I CAN imagine several ways in which this could add some pizzazz to a show. Its has potential, even at this starting stage.

  2. It doesn’t make much sense that the guy is triggering the drum beats, which means the dancers are essentially trying to keep time with themselves. Underlying concept is interesting, maybe they went with this simplistic implementation to make the link between movement and sound as clear as possible.

  3. This looks like so much fun.
    How much control of the music is possible with the Kinect?
    Even if you don’t have that much control it seems really fun.

    Why all the Hate?!
    Some commentators are just miserable haters…

  4. Looks good!
    Although it doesn’t make sense. Cause music gets made when you dance but in real life we dance when music is turned on. Still if they get it to microsoft they might get really rich.

  5. The WOMAN is a dancer… the guy is just… a guy who can’t really dance.
    He just flops around and does the new “fist pump” dance move that seems to be all the rage these days.

    She looks a little embarrassed for him, actually. His “pants’ don’t help, what with the penile poke vs. saggy bottom effect going on, lol.

    1. Damn, just went to their site, and the guy is “Chief Dance Officer”!

      How in the hell did he get that job? He has no grace!

      1. Ah, come on. He’s not that bad.

        It is another way to have music be more interactive– without it necessarily being a tedious focus. Will it generate great music? Probably not. But it clearly has to potential to generate fun experiences.

  6. hi there
    just a few words to this software Nagual Dance.
    Im the guy dancing in the video.
    you are right, im no dancer, im just a guy who loves music and moving. the girl is a professionell dancer. this video should show that the software is for everybody. dancer and people who just like music. it´s about the feeling you have when you controll the music. it´s just fun and it keeps you fit.
    i´m the music producer in the team and i´m the chief dance officer cause i make all the settings in the software for the dancer and not cause im the greates dancer in town. i produce the music and check if everything feels and sounds good.
    To answer the question why: it´s a lot of fun and it gives people without music knowledge the possibility to be musically creative. It´s also fun for Music Producer cause it´s a new way of producing possibilities. The software has note generation moduls and synths. you can assign this to different parts of the bodies.
    we startet this project because it was just our dream to make music out of movement. but after many installations we have seen that also a lot of other people and specially a lot of kids really like it.
    so for us it´s a really fun way of interacting with music, to get lost in sound and dancing.
    we want to share this feeling with all the people who like it. If you don´t like, it´s also ok 🙂

  7. Hi Artur,
    I totally dig what you are doing. Kudos! Do you have any issue with the latency on the Kinect? I’m so curious as to if this could be played “real-time” with a band and still maintain “feel” and “groove”…

    1. Hey Ian.
      Thank you.
      We don´t have problems with the latency. It can be play in real time and you can use it with a band.
      We are planing to use it for performances in combination with “real” instruments.

  8. Hello Artur and Nagual Dance,

    First of all, this the best video Nagual has ever created and serves you best for promotional purpose. If you can, you should invite your companion dancer more often, because she is great and best dancer in all of your videos.

    Also, the best video, where I actually like music composition as well. Reminds me of Robert Folk’s mid 80’s electronic scores. Is this original composition created for video? Can you name composer?

    Nice work, best regards for your future plans

    1. Thank you very much for your kinds words.
      For sure we are working with Malin (she is the dancer) more often. We are also working with
      more professionell dancers now.
      The Composition in this soundscape is our own composition. Mark Moebius our Co-Founder is a Composer and he is composing the Songstructure for the Soundscapes. Im responsible for production and sounddesign.
      We are also starting with the integration of other composers and producers to have a variety of songs.

Leave a Reply