Kids in (Sound) Space: Process and Performance

The "Kids in (Sound) Space" project involved three students from the Computer School in Manhattan, Seweryn Nehring, Mariyam Salley and Hope Allen-Kahn, myself (douglas repetto) and my programming assistant Karl Ward and a rotating cast of teaching assistants from Teachers College: Kate Hofstetter, Tanya Papayannopoulou and Sun-Ho Joo.

We had about five two-hour sessions with the kids during the winter leading up to the presentation in Greece in March. Although the project and technology may seem a bit complex from this description, the kids understood it all almost immediately.

We started off by talking about sound environments. We took some walks around our building and the neighborhood, and practiced listening to the environment. We stood on the street with our eyes closed and tried to describe the scenes taking place in front of us. We discussed the types and qualities of the sounds we heard and thought about the ways those sounds were related, how they influenced each other and what kinds of stories they told individually and as a whole. We talked about the idea that music is not necessarily just about notes and rhythms, and about how speech, environmental, abstract and computer generated sounds can all be used to generate sound-based art. Finally we took another walk around the neighborhood and a nearby park, but this time we brought along a mini-disc recorder, which the kids used to record various found and kid-made sounds. These sounds would ultimately form the core of our performance in Greece.

A side note: at this point one of our students, Hope, went on a brief trip with her mom. When she returned the first thing she said to me was, "You won't believe some of the sounds I heard in Mexico!" Having spent a couple weeks talking about and listening to her environment, it was clear that she had begun thinking about and experiencing the world in some new ways.

In the next few sessions the kids started learning how to use the software and controllers. It takes awhile to get used to controlling the balls on screen with the joystick and accellerometers, so we started off with very simple ball-to-sound mappings. One very effective setup is to map a ball's x-position (left-right) to the pitch of a sinewave and the ball's y-position to the volume of the sinewave. This makes the following mapping: bottom left = quiet, low; top left = loud, low; top right = loud, high; bottom right = quiet, high. This simple mapping makes the relationship between the ball's position and the sound's features immediately obvious. The kids mastered this mapping very quickly, so we proceeded to explore the potential of more complicated mappings and more interesting sounds.

The next step was to begin using the sounds that the kids had recorded. We had a large library of sounds and spent a fair amount of time listening to them, talking about them and deciding which ones would work best in our performance. We were concerned with how well the sounds would go together, how easily their parameters could be controlled, and how interesting they would be for an audience to listen to. For example, we decided that the subway sound would work well as a background sound, but that it didn't have enough variety to work well in the foreground. Even if we mangled the subway sound in all different ways it was still fairly static, making it less suitable as a "lead voice" than some of the other sounds that could change character from moment to moment. For instance, the sound of a cash register printer worked very well as a foreground sound, since its character changed drastically when we slowed it down or changed its pitch.

We continued trying out different sounds throughout the rest of our sessions. We made the software very flexible so that it was easy to experiment with different sounds and sound combinations. As we all got comfortable with the setup the piece we would perform in Greece began to take shape. Somewhere along the way we all agreed that the piece would take the form of a journey from one sound environment to another. We weren't satisfied with just using the sounds we had recorded, so we went on the web and started looking for additional sounds. We found a lot of interesting animal and insect sounds and we started to use them in our rehearsals. Ultimately we ended up with a three part piece, a sonic journey that took us from the city (city sounds) to the country (animal and insect sounds) and finally to outer space (warped city sounds and sound effects).

Each kid had their own sound in each part of the piece. So Mariyam might have gone from controlling the cash register to the frog to the alien voice in the course of a performance. As we rehearsed the piece the kids kept changing their sounds until finally they settled into a combination that was interesting for both the performers and the audience. Once the sound selection was fairly stable we focused on creating a dynamic performance using the sequence of sounds we had settled on. Each sound had to be performed in its own way, depending on its qualities. For instance, the cow sound was very entertaining when used subtly, but quickly became annoying if it was loud and repetitive. We talked a lot about which sounds worked well in which situations and how the sounds could be used to complement one another.

The Pongserver software was purposely designed to be difficult to control precisely. This means that it was not possible for the kids to perform the piece the same way twice, and that there was no point in their memorizing a particular set of moves or sequence of events to be used in performance. Instead each time they rehearsed the piece the kids had to be fully engaged in listening to and controlling the sounds they had selected. Because the sounds we used were fixed, the overall character of each performance was similar, while the details of the piece changed every time.

The last few rehearsals were spent working out stage logistics (entering and exiting stage, bowing, etc.), stage presence and developing the visual aspects of the piece. We encouraged the kids to look at each other and at the audience while performing, rather than fixating on the screen in front of them. They also began talking to one another during performances, often making suggestions and jokes about the way the piece was progressing. We encouraged them to continue that sort of in-performance communication.

In the performance there was a large computer projection behind the kids and a computer monitor off stage in front of them. The kids stood in the middle of the stage facing the audience. This made the performance seem less like three kids watching TV or playing a video game and more like three kids performing on a stage for an audience. They still had to watch the smaller screen in order to perform, but it wasn't the focus of everyone's attention.

For the visual side of the performance we wrote a number of client programs that the kids could use to control the creation of graphics using same data that they were using to control the sounds. The kids helped configure these graphics programs, selecting colors, shapes, locations and other details of the visual design. Once we got to Greece the kids were very confident in their ability to perform the piece, so we spent a lot of our rehearsal time working with them on these visual elements.

We ended up with a very dynamic visual display that changed along with the sections of the piece. In one corner of the screen was the field with the balls and paddles in it. In the other three corners were a variety of visual interpretations of the control data. One interpretation was a simple tracer that drew a line behind each ball as it moved. As the piece progressed these lines developed into a tangled map of the piece's progress. Another simulated colored raindrops falling into a pond whenever two balls collided. The raindrops fell at the point of collision and the waves they made propagated out across the screen. Another made a three-dimensional wireframe model using the location of each ball on screen as a vertex.

At one point Seweryn said that the image on the tracer screen was getting too dense, making it hard to follow the path of the balls as the piece progressed, so that night we reprogrammed it to erase the screen at the beginning of each segment of the piece. We kept going back and forth on these and other issues (colors, sounds, who enters the stage from where and when, etc.) until everyone was satisfied with both the aural and the visual aspects of the piece. We had one final dress rehearsal and then the big performance.

The performance went very well; it was probably the best version of the piece the kids had done. They listened to one another, controlled the sounds and graphics in subtle and creative ways, and paced the piece to keep it interesting and exciting for themselves and for the audience. We were all very proud of the work that we had done together.

Check out three video clips from the performance:
hands, closeup -- everyone -- video screen

And these audio clips:
city -- city to country -- space