Controlling Ableton Live via Oculus Rift & Pensato

This video captures a demonstration by Byron Mallett of using Pensato – a virtual reality interface for controlling music software and devices, using an Oculus Rift and custom VR gloves.

Here’s what Mallet has to say about the technology he’s using:

Tech Discussion:

Pensato is a virtual reality interface I’ve developed for controlling music software and devices using an Oculus Rift and custom VR gloves. Using this system, I can control instruments, trigger sound clips and manipulate effects by interacting with virtual representations of multiple sound devices. The gloves are made from modified Razer Hydra game controllers and are paired with two Teensy microcontrollers equipped with bend sensors for detecting hand my hand gestures.

This performance features a master version of Pensato running in Unity on the computer monitor to my right, which creates the visuals for the Rift and controls Ableton Live. The projector computer runs another copy of Pensato that listens to the position and orientation of the gloves, as well as the hand gesture and displays and displays a copy of my VR interface on 3 projectors. The laptop runs Ableton Live 9 and is connected to each copy of Pensato using Showtime, a framework that I developed for connecting together realtime performance software and hardware.

This work is the final compositional output for my Masters of Design Innovation: Computer Graphics at Victoria University of Wellington. All audio was captured from the performance and used to create the edited version of the vide.”

An interview with Byron Mallett about his work with Pensato is available at the Ableton site.

12 thoughts on “Controlling Ableton Live via Oculus Rift & Pensato

  1. so how exactly does it make use of the third dimension? i mean it is techy and certainly cool to play around with it, but it doesnt use any interface/interaction metaphors that cant be also used on a simple multitouch interface, you could even control it entirely (from what i have seen) with the mouse – so no higher order gestures. also gorilla arm… *sigh*
    it feels like the first wave of interfaces (based on virtual reality metaphors) that are supposedly adding more expression based on the new interaction medium. the last wave was leapmotion and then there are the millions of ipad/tablet apps that seriously *LACK* any understanding of musical HCI or post-wimp/embodied interaction concept.

    other than that it is quite a nice tech demo 🙂

  2. This is a really interesting way to utilize virtual reality I guess. I would love to see an artist release an album that has an accompanying virtual reality game type thing with it. I suppose it wouldn’t be too difficult to create.

    1. you can play music in Fract OSC (studio)
      it is a game out of Montreal ,
      i got a chance to do some beta testing (2012) at that time there was no midi support .
      you could do allot with it and there was also a game section
      , it worked work well with a VR interface (HMD and Data gloves)

      if you search you can find many VR interfaces for audio

      personally beyond the (this is COOL factor !!!) i never found anything fast or practical enough .

  3. I think the point here is the evolution of the next generation of performance and gear. These are the early steps in a fantastic and wondrous direction that has yet to be defined, but can let the imagination run wild. Fantastic demo and I look forward to what these bright minds bring to reality, or VR, in the future!

  4. I don’t wqnt to sound disrespectful or like a troll. Sure this is not my cup of tea, but new technology should produce new music. I think in this sexample it fails. Any Kraftwerk record can beat this.
    And of course I need to quote Herbert Marcuse:
    “The extent to which this civilization transforms the object world into an extension of man’s mind and body makes the very notion of alienation questionable. The people recognize themselves in their commodities; they find their soul in their automobile, hi-fi set, split-level home, kitchen equipment. The very mechanism which ties the individual to his society has changed, and social control is anchored in the new needs which it has produced.”. Hey good luck…

Leave a Reply