This video, via robotmakers, demonstrates using a Microsoft Kinect as a gestural controller for modular synthesizers.
Here are the technical details:
Custom software enables gestural control using Kinect and PC.
Hand movements in the X and Y axes are translated into MIDI signals which generate control voltages, allowing 2 dimensional morphing of waveforms in a “Morphing Terrarium” module, processed through modular moog.
Recorded in the superterranian lair of the Robotmakers.
The latency and resolution of the Kinect are barriers to using it as a music controller. The approach demonstrated in the video avoids this issue, though, using a traditional keyboard for playing the melody and using a virtual x/y pad, via the Kinect, to control the sound expressively.