Artistic Considerations in the use of the EyeCon Motion Tracking System

The Nature of Interaction

So what is this thing "interaction"? Its what people do with each other. Its what we are not doing now. If you were to write me back a letter, I would get your reaction, but we don't really start interacting until we sit down together and hash it out. It belongs to the most primitive of human instincts that when we are together with one another, exchanging on almost any level, we become animated and excited. We share this, of course, with most other animals.

Human beings have been dancing and making music for 10,000 years. During most of this long history performances were highly interactive, much more so than they are today. The distinctions of "performer" and "audience", and even those of "musician", "dancer", etc. were far less clear than they are today. There was no, "think I'll sit this one out". Everyone was part of the event. There are still today examples in African of traditions for which the same word is used for both dance and music. Participants feed off of each other's energy in a way which is seen today only in such settings as dance clubs (the good ones) and music/dance improv jams. Jazz music provides perhaps a last bastion of highly interactive performing in the West, which is also codified and highly disciplined.

Beginning with the predominance of the bourgeoisie, theater in Europe saw a closing off of interaction between performer and audience. With bright lights on one side of a proscenium, and a darken area with seats on the other, the audience's role was pretty much reduced to sitting quietly and then clapping before going home. This has not changed much in the last two hundred years.

Between artists, interactivity has also seen a great decline. Ironically, modern technology is a major culprit. Recording and sampling techniques have meant that musicians, for example, often work separately. Pop music relies heavily on sampling ("stealing with respect" I have heard it called) rather than creating from scratch or getting together and jamming. Dancers and musicians, meanwhile, rarely work directly with one another anymore. Only a tiny fraction of dances performed today (including those by big name companies) directly involve musicians in any part of the actual stage production: creation, rehearsal or performance.

But the biggest interactivity-buster of all is surely the projection screen. Not only did video further reduce the need for dancers and composers to work together creatively, but of course we don't even need to be part of an audience today to watch a performance: we just turn on the television.

And then there is the "group energy" thing. From disco dancing to Ashtanga yoga, we generally pump up better when we are part of a group. As much as we may admire the individualist, in our hearts (and in our genes) we're basically a pretty social bunch of monkeys.

Palindrome has a piece called "Publikumsstück" ("audience piece") in which 10 audience members are brought backstage during the intermission. We teach them ingredients for a structured improvisation as well as give them a crash-course in interactive performing. After the intermission the piece is performed within an interactive stage environment so that different audience members control different sounds with their movements. A woman once came up to me after the show and commented that she liked the part that "we were involved in". I looked at her for I did not recognize her as one of the ten. She wasn't. She meant "we" the audience.

This woman's reaction points out something quite fundamental about how interaction works. It is very much a "feeling thing" -- a subjective, rather than objective phenomenon. Small amounts of participation can have an enormous effect. On the other side, giving the audience many things to do may have little effect on their "interactivity" -- their sense that they were part of the piece. It depends heavily on how it is done; the issue is the context, getting that group-feeling thing going and the choices of mapping.

The same principle applies to performer-based interactive pieces. Sometimes a minor, or short-lived interactive event, with a painfully simple mapping scheme, may completely alter the nature of the piece.

You will notice, we can talk today of two kinds of interactivity: interaction between artists and interaction involving the audience and artists together. To my way of thinking, the same basic principles apply to both; they share the same psychological roots and in practice will function in a similar way. In both cases, it is highly dependent on the performer being loose and easy in their role so that they can respond honestly to the media and people around them.

Which is not the same as improvisation. Palindrome's work is probably 90% choreographed. Still, within a basic structure there must be a certain critical "play-room". If a piece is truly fixed, it can never be interactive.

When you first experience a system like Eyecon, there is the "aha!" effect. Oh, I can trigger sounds with movements, cool. I imagine this must have been something like what people experienced when the light switch was invented. People probably did shows where they turned lights on and off -- automatically! Notice: neither switching on a light, nor triggering sounds with movement has anything to do with interaction. Let's not get confused. Interaction entails a back-and-forth, not just a forth.

If you are dancing in a (now I have to say) "so-called" interactive system, you may choose to "interact" with it or not. If your movement controls, say, the sound you are dancing to, and your dancing is modulated by the real-time changes in the music, then there is a kind of interaction; a circle is closed, so to speak. You will notice, this does not strictly fit the definition I gave you earlier since there is no one else there! You might say the computer has taken over this role. But let's not split hairs, the point is that a particular spontaneous quality can be achieved -- a sense of liveliness -- in a performance of this kind. The dancer, through her movement, is empowered by having control over the music to which she is dancing. Whether you choose to call this "interactive" or not, the process by which it is created generally involves much more collaboration -- and a more intimate kind of collaboration -- than in a traditional dance work. The exchanges between dancer, composer, choreographer and perhaps engineer are highly interactive. There are not only many more interdisciplinary issues involved, but there may be more than one secondary media (Eyecon may also control stage lights, for example).

Now, there is one way to work with systems like EyeCon which holds "interactive" to its word. These are pieces which use technology as a kind of mediator between live performers. My partner, Frieder Weiß is a champion of this school of thought. Such a piece might would function like this: Dancer's movements are tracked and, instead of playing music electronically, the data are fed to live musicians (for example notes may appear on their electronic note stands). The musicians then interpret this information (i.e. play the notes as can, or see fit). The sound of their instruments is then fed again to a computer where it is translated to information which can in turn inform the dancer's movement. The system we have used, for example, involves projecting a light field onto the stage from above. The size and shape of this field is altered according to the pitch and dynamics of the music. The dancer must try to fit her movements into the size and shape of the projected field -- thus the dancer's stage (and thus her movements) are modified by the music. The circle is thus closed.