Artistic Considerations in the use of the EyeCon Motion Tracking System


The human body in motion draws its expressive power in no small measure from the special sensitivity of the human eye to particular movement qualities or parameters. These are felt by the actor or dancer -- in her or his so-called "kinesthetic sense" -- but equally important, they are perceived by the audience. The point is, when it comes to human movement, there is a difference between what the objective eye -- in this case the video camera -- sees, and what the viewer perceives. We are not especially aware of it, but the human brain does quite a bit of "image processing" after the picture is taken. Thus, sometimes even the subtlest of movements can play a crucial role in our perception, influencing our impression of what we have seen and in the end determining what the dance says to us.

Computers, at least those of the current generation, lack this sensitivity. To them, human motion is just so many pixels changing light or color value. We are thus left with a huge discrepancy in data between what we humans experience as "dance" and what a computer, any computer, measures and has available to process. Understanding this discrepancy and learning to compensate for it is the key to using Eyecon effectively. One needs to find "work-arounds" -- ways to make the computer seem smarter than it actually is.

To put it another way: Sometimes a key quality or moment in a dance involves a relatively small, or localized change in the image (what to a computer means "few pixels have changes in light value"). Meanwhile, at the same moment, artistically unimportant movement may be large or fast (large pixel value changes) and yet these larger changes may not even be noteworthy or noticeable to an audience. Thus, we, as interactive piece designers, are faced with a situation in which we must "teach" the system what to pay attention to, otherwise we may end up with a piece which is technically interactive, and yet fails to seem this way for anyone other than perhaps the technicians and performers. For the audience, such a use of technology is a red herring; something the program notes tell them about, but which has no real relevance to the experience of the piece.

It is remarkable how often this occurs, even at performances of quite renowned companies! To me, this always seems perplexing; it seems so avoidable. There are different reasons why this happens:

One is simply that the artist wanted it that way. My taste in these things is not everyone's! One hears it argued that since the performers are aware of the interaction, and they report that it affects the way they dance, then the use of technology is justified for it has, if indirectly, altered the final outcome. So we can all go home having seen an "interactive performance". Artists, remember, do love to be obtuse. The last thing we want is for our work to be obvious or pedantic. Our tendency is to err on side of obscurity, then at least we can hide behind the ruse of artistic license and eccentricity. No one can accuse us of having used technology as a "special effect" just to wow an audience.

My reaction is: Come on guys -- we can do better than that! There was a time when contact improv (perhaps the most interactive of contemporary dance forms) was used on stage with the same justification. The problem in either case is that the audience is marginalized in the event. To me this seems to fly in face of what interaction is all about.

Interactive or not, art is ultimately something which should engage an audience -- challenge them, surprise them, touch them. Systems such as EyeCon allow artists possibilities for doing this in ways which were not available in the past and it seems a shame to miss out on the very quality that these systems are so well-suited to achieve just because of poor planning or poor understanding of how interaction functions.

And there is another issue which has made for slow going in this field. The possibilities of interactive art are simply too poorly understood by all of us. Collaboration of the sort required, between artists and engineers, can be slow going. Engineers present us with flashy state-of-the-art devices and choreographers (along with the rest of society) get unrealistic ideas of what they may be good for; where their true potential lies. Artists may lack the technical knowledge necessary to suggest modifications to the system -- the things which can be altered to "bend" the system to their special, sometimes "illogical" purposes. Thus, in the creation process a crucial play-ground phase is often missing, and pieces result which employ technical systems basically straight off the engineer's table without really having undergone an "application-oriented" developmental process.

I have two lines of thought that I hope might be helpful to those wishing to do interactive performing. One concerns the nature of interaction, i.e. that special quality -- what is really a psychological phenomenon -- one can achieve using systems like Eyecon. The other is the more technical issue of how you go about designing pieces to capitalize on a system's capabilities while hiding its weaknesses. This second point is referred to as mapping, i.e. the choices of input and output -- what characteristics of the body in motion the system should be focussed on, and then how it is we want to use the data. This second point, as we shall see, also concerns the "way" performers do their performing.