For some time now, Golan Levin has taken an interest in producing different types of images and sounds in an unconventional way. During the summer of 2002, Levin and Zachary Lieberman were artists in residence at the Ars Electronica Futurelab in Linz, Austria, where they created and produced RE:MARK
(2002) and Hidden Worlds of Noise and Voice
(2002). These can be considered as the groundwork leading to the creation of Messa di Voce
. The two installations are being presented for a two-year period at the Ars Electronica Museum of the Future in Linz.
“If we could see our speech, what might it look like?” (Levin, Lieberman)
The first two interactive audiovisual installations share the same theme: the users are able to see their voices. In fact, the voices are made visible in the form of animated graphic figures that seem to emerge from the mouths of the participants. In the installation Hidden Worlds
, the user wears special see-through data glasses, which register and superimpose the images in 3D graphics. The sound of the user's voice creates abstract and coloured forms that are amplified and modified according to the tone and volume. A broad range of audiovisual variations are then made possible. RE:MARK
is an installation for two participants that also presents an interactive visualization of speech... Unlike Hidden Worlds
, which primarily focuses on voice as a projection of noise, RE:MARK
shifts the emphasis to the more symbolic domain of the spoken and written word. In this installation, sounds spoken into a pair of microphones are analyzed and classified by a phoneme recognition system. When a phoneme is clearly recognized, the written name of the phoneme is projected on the user's installation display. If the user's sound is not recognized by the system's classifier, then an abstract shape is generated instead, based on the tonal characteristics of the vocalization. Furthermore, visitors become active participants through a computer-vision system that allows them to conceal these forms with the movement of their shadows projected on a screen. The artists describe these images and this experience as a form of consensual hallucination.
The performance Messa di Voce
explores this theme further using two virtuoso vocalists, Joan La Barbara and Jaap Blonk. (1)
This new work is rooted in phonesthesia, or phonetic symbolism, where to a certain extent and on a connotative level, the sounds of the words are depicted through forms and textures. If you compare it with the two previously mentioned installations, Messa di Voce
represents technological progress, since it demands the development of a system capable of a very high level of expression. In other words, the system has to reflect the virtuosity of the vocalists. Using the latest technological voice recognition and motion detection technologies, Messa di Voce
remains a profoundly human work in that it explores the meaning and the effects produced by the voice. Not a single word is uttered in this performance, which focuses on the purely formal and sonic aspects of singing, poetry and performance. Despite its technological complexity, the device is simple: a video camera hooked up to a computer detects the position of the performers and analyses their voices. In response, various shapes appear on the screen behind the duo. The visual representations of their voices, cries and singing — and you can imagine the range of possibilities with such virtuosos — also serve as an acoustic playback. In return, the performers' manipulation of these representations can modify their voices. Thanks to the motion detection system and what Levin calls the "fiction that speech can be truly visible," these apparitions appear to emerge from La Barbara and Blonk's mouths.
The performance is comprised of a series of 12 vignettes and features a narrative progression ranging from pure sound to speech and song. With names such as Fluid, Ripple, Bodystamp
, the vignette titles are clearly both literal and evocative.
Following the world premiere of Messa di Voce
at the Ars Electronica Festival
on September 7, 2003, performances were held at the Ultrasound Festival
in Huddersfield, England, and the ICA in London, England, in November 2003. Certain vignettes from Messa di Voce
are available as a two-player interactive installation. This version was presented during Beta Launch ’03: Eyebeam Artist-in-Residence
, organized by the Eyebeam Atelier Gallery Space in New York in October and November of 2003.