Dan Lloyd, Brownell Professor of Philosophy at Trinity College, is using fMRI data from scans of the brain to synthesize “music”, and has found that different brains make very different melodies.

Lloyd created software which translates data from a brain scan to Musical Instrument Digital Interface (MIDI) data. This lets him use the information to control a synthesizer, resulting in brain-driven synthesis. According to Lloyd, this technology that may give new insights into the differences and similarities between normal and dysfunctional brains.

“The sounds work in unison and create melodies,” he said. “Different parts harmonize with each other to make not just sounds, but music.”

Lloyd used this technology to compare brain scans from people with dementia and schizophrenia to healthy subjects and found a noticeable difference in the music they created.

For more videos on Lloyd’s research, check his YouTube channel.

Do you think this qualifies as music, or is it just translating information from one timeline based form to another? Leave a comment with your thoughts!

via the BBC: The Multimodal Brain Orchestra performed its world premiere on Thursday.

Led by an “emotional conductor” and a traditional one, music and video change in time with the performers’ brain waves and heart rate. According to the work’s producer, the orchestra aims to “see what the brain can do without the body”.

The orchestra’s premiere performance closed the Science Beyond Fiction conference in Prague.

“Only recently we have come to appreciate more the tight coupling between mind, brain and body,” Paul Verschure, head of the project, told the audience. “But we can wonder what the mind and brain would be capable of if it would be directly interfaced to the world, bypassing the body.”

What do you think of the Multimodal Brain Orchestra’s brain music? Is it time to junk the old gear?

More on the Multimodal Brain Orchestra below.


About The Multimodal Brain Orchestra

The Multimodal Brain Orchestra – MBO – shows the raw creative abilities of the unmediated brain. It explores the question what the creative content is that a collection of brains can generate that are directly interfaced to the world, bypassing their bodies.

The Brain Orchestra members play virtual musical instruments through Brain Computer Interface – BCI – technology alone. In addition, an emotional conductor drives the affective content of a multi-modal composition by means of her physiological state.

For FET09, SPECS developed a piece called XMotion that will be performed by the Brain Orchestra.

Where Emotions refer to the passions that drive our actions in the real world, Xmotions are those affective states that can be generated and experienced by the unmediated brain that is immersed in and takes charge of the multi modal jungle in which it find itself.

In Xmotion the audience will witness how experience can be created through direct interfaces to the brain and the body that bypass the volitional body centred control that defines the common modus operandi of human existence. Where the brain requires a body to exist and operate in the real world, the body can become a burden when we want to transcend the borders of the omniverse of all possible physical realities and enter the noosphere of all possible mental states and subjective experiences. The brain can make this transition as exemplified in the creative content of dreams and art or the delusional states of psychosis. We can incorporate these states in the physical world and externalize them through direct brain interfaces, synthetic creative systems and interactive media.

MBO will demonstrate new concepts and technologies of interactive, affect-based self-generated media content, brain computer interfaces and autonomous synthetic music composition. The MBO XMotion performance is unique in a number of respects including:

1: the use of a multi-person orchestra to control virtual instruments through brain states
2: the integration of multi-person physiological states in an evolving synthetic audio-visual composition
3: the real-time control of the affective expression of a performance by the real-time physiological state of a human, the, so called, emotional conductor

In this video, Okudaira-san from MI7 Japan, is using a brainwave analyzer from IBVA to get signals from his brain into the computer, which control the red, green and blue color channels of a Percussa AudioCube.

Percussa’s Bert Schiettecatte is looking for cool AudioCubes demos, so if you’re using them in a unique way, let him know at cubes (at) percussa (dot) com.

Let me know in the comments, too!

It’s hard to believe, but it looks like there may be a commercial future for brain music medicine.

According to a Fox News report, Dr. Galina Mindlin, a psychiatrist at Columbia University in New York City has been using brain music therapy for three years to treat a variety of ills, including insomnia, anxiety, attention deficit disorder, depression, substance abuse and stress.

A complete treatment, which includes a brief medical evaluation, a recording of the brain waves and a personalized music file, costs $550.

According to Mindlin, the therapy has an 80-85 percent success rate. That’s where my BS detector goes off.

Drug treatment programs have abysmally low completion rates – one study reports that about 3.5% of those that start a drug treatment program complete it.

Call me skeptical on brain music therapy, even if there’s a market for it.

Let me know what you think!

Got a brain? Then you can make music!

At the UK Aldeburgh Festival, set for June 13-29, Norwegian composer Rolf Wallin is going to be demonstrating the latest brain music action.

This isn’t like controlling modular synthesizers with mind bullets, though.

According to the Telegraph, you simply “let a skilled assistant fasten electrodes to your head, relax and enjoy a short film specially designed to stimulate your brain activity. A computer converts your brainwaves into your own very own music […] The music is performed real-time on a self-playing piano, and you can purchase a CD of your work to take away.”

Fragmented Orchestra

If that’s not enough brain music for you, the winning entry for the recent PRS New Music Award may interest you.

The Fragmented Orchestra is a music installation that aims to let us hear the human brain at work.

Digital “neuron units” will gather and process the sounds of 24 locations round the UK, ranging from a cathedral to a school playground to a dairy farm. These units will link together into a kind of virtual brain, processing the sounds and feeding the results out at the locations themselves and at FACT, Liverpool.

Gnaural is a multi-platform programmable binaural-beat generator, implementing the principle of binaural beats as described in the October 1973 Scientific American article Auditory Beats in the Brain (Gerald Oster).

It can be used to synthesize sounds that affect the brain. Here’s a demo. It should be listened to using headphones:

The central theme of Oster’s article is that processing of auditory binaural beats within the brain bears distinct differences from that done for normal sound, emphasizing different neural pathways and highlighting different parameters of the sound stimulus.

Oster’s observations inspired a wave of research in to the ways in which binaural beats could affect the brain. One area of research explored how binaural beats could evoke a “frequency-following response” (also known as “brainwave entrainment”) in EEG measures. My personal interest in binaural beats has centered almost exclusively around exploring this entrainment potential as a means of facilitating meditative states. However, Gnaural was designed to be neutral with regard to any hypothesis or application, relying strictly on the fundamental findings as described in Oster’s 1973 overview.

Gnaural is an open source project. Versions are currently available for Windows & Linux, but not yet for Mac OS X.

A limited brower-based version of Gnaural is available. A Mac developer is needed to help port this to OS X.

What are auditory binaural beats?

In 1839, German experimenter Heinrich Wilhelm Dove discovered that illusory “beats” are perceived when pure tones of slightly different frequency are separately and simultaneously presented to each ear. Dove’s insight was to realize that since there is no acoustic mixing of the tones, the perceived beats must exist soley within the auditory system, specifically that part which processes binaural (e.g., “stereo”) sound.

While research in to binaural beats continued after that, the subject was largely viewed as no more than a scientific curiosity. Oster’s paper was landmark not so much for its laboratory findings, but in how it tied-together the isolated islands of research since Dove, in a way that gave the subject a renewed relevance to modern scientific questions.

In particular, Oster viewed binaural beats as a tool with cognitive and medical applications. Cognitively, he felt it could be used to explore neural pathways, and also to address higher-level questions, including how we spatially locate sounds in our environment, and our auditory system’s propensity for selective attention (e.g., the “cocktail party effect”).

Medically, Oster saw potential for binaural beats as a diagnostic tool, both for auditory impairments, and for a broad range of non-auditory subjects. Most notable was evidence Oster showed that a decreased ability to perceive binaural beats appeared to be a pre-onset indicator of Parkinson’s disease. He also presented corroborating data correlating subtle cyclical fluctuations of estrogen in women and their ability to perceive binaural beats.

Notable for Oster’s thesis (that binaural beats are processed in ways fundamentally different from normal hearing) was the fact that binaural beats are percieved even when one of the two frequencies is below the human frequency threshold, and also when both frequencies are below the human volume threshold. This, combined with the brain data available at that time, suggested to Oster that the processing of binaural beats followed different neural pathways in the brain from other auditory processing.

Broadly, the rhythmic influencing of brainwave activity is known as “driving.” And while binaural beats have no monopoly as an auditory driving stimulus, they do have the unusual property of being able to deliver direct auditory stimuli at sub-audible frequencies (below the range of human hearing), by virtue of the heterodyning being simulated within the auditory system.

The reason this is interesting in regard to FFR is that (generally speaking) the spectrum of perceivable acoustic frequencies is well above the frequency spectrum of brainwave activity. Thus, aside from binaural beats, the only means of presenting acoustic driving stimuli is by externally modulating sound (in to waves or pulses whose periodicity falls within the spectrum of brainwave frequencies).

Binaural beats, on the other hand, provide a direct means by which pure acoustic tones can be delivered to directly produce a driving stimulus within the range of brainwave activity. Perhaps even more important (in regard to driving) is that with binaural beats, the driving stimulus arises internally (within the auditory system). This suggests that binaural beats may more effectively induce driving than simple monaural modulation, if only for the fact that the resulting stimuli arises directly within neural pathways that can be measured in the course of gauging brainwave activity.

Gnaural is officially open source, released under the GNU General Public License.

Here’s the weirdest thing you’re likely to see today – Sleep Waking – a robot that does interpretive dancing based on your brainwaves.

Here’s what creator Fernando Orellana has to say about the project:

In the next thirty years we will see more robotic technology integrated into our society, furthering our experience of reality through agency. Robots already go beyond the limitations of our bodies. They build things that we find too difficult or tedious, assist us in our medical care, and are beginning to substitute as our sexual partners. Just as the computer and the automobile have augmented our communication and transportation capabilities, the robot is destined to be essential to our society. What will humanity’s relationship be to the robot in the future?

Using recorded brainwave activity and eye movements during REM sleep to determine robot behaviors and head positions, “Sleep Waking” acts as a way to “play-back” dreams. Through this piece we hope to investigate one of the possible human-robot relationships.

“This video shows me demoing my initial Brain Computer Music Interface. There are many ways that I can improve this system (it’s still a bit slow at the moment), but I think it demonstrates the potential of the idea well enough. This video was difficult to make because someone was playing unbelievably loud Keane-esque power ballads on the frickin’ piano next door. Those of you who have experience with EEG will no doubt find this most amusing.”

The Houston Chronicle has an interesting article that looks at Brain Music Therapy, the medical practice of using brain waves to generate music to help people relax.

“Your brain is actually listening to the best of itself,” said William Wade, a psychotherapist who offers Brain Music Therapy at his practice, the Institute for Family Psychology, in West University Place. “It models itself after the brain music.”

Developed overseas in the early 1990s, the therapy made its way to Houston only about a year ago. Since then, Wade and his partner, Carol Kershaw, who believe they are the only psychologists offering the procedure locally, have used it for nearly 100 patients. It has worked for most, they say.

But there’s still doubt in the medical community about the therapy’s legitimacy.

“When I put on my science hat, I’m skeptical,” said Max Hirshkowitz, director of the sleep center at the Michael E. DeBakey VA Medical Center. “When I put on my clinical hat, I’ll do anything that works.”

Hirshkowitz, who also is a professor of medicine at the Baylor College of Medicine, said he’s not convinced, partly because it’s unclear exactly how the brain waves are made into music. That information is proprietary.

A relatively small study conducted in 2002 showed that patients who listened to their own brain music suffered fewer symptoms of insomnia and anxiety than those who listened to someone else’s brain music.

The technique isn’t cheap, and it’s not covered by insurance. The first recording costs about $550 and is effective for about three months for most patients because the brain adapts to the music. Patients then, for the same price, are encouraged to get a second recording, which usually lasts about four years, Kershaw says.

Krischall demonstrates how to use brain power to control a modular synthesizer.

By connecting a biofeedback helmet to a 1970s E-mu Matrix synth, the device is able to generate sounds just by thinking about them.

Actually – although this looks well within the realm of possibility – Krischall’s mind bullet experiment owes more to Photoshop than neurology.

Let me know if you’re really controlling your synth with mind bullets!

via Technobob