1. Overview

Figure 1 - Overview Picture of the Installation

@wake is a digital-physical installation which explores several themes. The issues I wanted to address with this work are the creation of a very separate and distinct physical space within a gallery space, the creation of a sense of connection between the current participant and previous participants, the use of foreground vs. ambient information, and the development of artwork-specifi c forms of interaction.

@wake is presented as a coffin. By using a dark, largely soundproof box, the participant is physically isolated from the gallery space. The coffi n metaphor further enhances that sense of separation by making use of the participantʼs strong feelings about death and the afterlife. Behavior is heavily affected by environment: just as visitors to a church do not behave in the same way they might behave at a football game, participants feel able to behave inside the coffi n differently than they would in the gallery space.

2. Description of Installation

The @wake installation consists of a purpose-built wooden coffi n which incorporates speakers, a microphone and a video camera directed at the face of the occupant. The speakers and microphone are connected to a hidden computer while the video cameraʼs output (an infrared view of the occupantʼs face) is displayed on a visible monitor in the main gallery space. As the participant enters the coffi n and closes the coffi n lid, a concealed button is triggered, beginning the experience. The participant is in complete darkness.

After being given a brief spoken introduction, the participant is asked a series of questions refl ecting on her life – some yes/no, some open-ended. Answers to the yes/no questions are interpreted by speech recognition software. Based on these answers, the system selects openended follow-up questions. For instance, if the participant answers ʻyesʼ to the question, ʻdo you have children?ʼ she might be asked a question about her relationship with her children. This open-ended question can be answered in as much depth as the participant wishes. Once the system detects silence for a certain length of time, it assumes that the participant is fi nished answering. That answer is then saved as digital audio fi les for later use and a new question is asked. Each participant is asked approximately 8-10 questions and the experience generally takes about three to four minutes.

Like the classic ʻpsychiatristʼ program ELIZA [B7], @wakeʼs questions are chosen based on the userʼs input, although ELIZA uses keyword and grammatical structure to determine its responses while @wake uses a more linear branching structure. An installation piece titled Sex, Lies and Binary Logic, created by Mark Winstanley & Michael Guida, uses a similar technique of question-and-response, but it takes a much more aggressive tone, playing on the userʼs fears. ʻYou bring in your own fears, in this case fear of lying.ʼ [B3] The fear of technology is also addressed as the installation toys with its user. ʻIts aggressive gamesmanship becomes deeply disturbing.ʼ [B2] @wakeʼs aim is not to antagonize, though. Its questions invite introspection but are not confrontational.

The answers which are recorded and saved will be played back during subsequent sessions, allowing participants to hear fragments of what previous people have said in the coffi n. These replayed answers are played back with digital echo and reverb effects to give them an ethereal quality.

Figure 2 – Diagram of @wakeʼs Processes

@wakeʼs software was written using Macromedia Director and its scripting language, Lingo. This was used to detect the closing and opening of the coffi n, to integrate two pieces of audio software – Game Commander and CPS, and to control the recording and playback of audio. After experimenting with two standard commercial speech recognition packages, I was unable to achieve reliable results. After a short period of training, which enables the system to recognize a wide range of words when spoken by a particular user, both packages are very effective for a single user. They are not, however, practical for use with a variety of users issuing single-syllable commands, as they depend heavily on training as well as sentence structure and context. The most effective package for recognizing simple ʻyesʼ and ʻnoʼ commands was a piece of software called Game Commander designed primarily to add simple voice command shortcuts to computer games. Game Commanderʼs vocabulary is completely confi gurable, which allowed me to reduce it to only two words – ʻyesʼ and ʻnoʼ. Given such a limited vocabulary, it had very little trouble distinguishing between the two very dissimilar-sounding words even when spoken by a variety of different voices.

The audio effects applied to the playback of previously recorded answers are provided by CPS, an audio ʻplug-inʼ for Director. By using CPS, I was able to keep all of the processing on a single laptop computer rather than having to use multiple computers or hardware audio effects units. CPSʼs Director integration and low processing demands made it ideal, as it allowed the opportunity to manipulate audio effects in realtime via Directorʼs scripting language.

During the experience, anyone standing near the installation is able to see a live video feed of the participantʼs face from an infrared video camera mounted in the coffi nʼs lid. This video is the only connection between the coffi n and the outside world. The coffi nʼs interior is padded, effectively preventing sound from entering or leaving the coffi n. The interior of the coffi n is completely dark.

3. Coffin as Metaphor

I chose a coffin and death metaphor as a means of emphasizing the distinction between the space in the coffi n and the space outside the coffi n. I wanted to create a very separate and distinct place for the participant in order to give a feeling of isolation. Although the gallery space is on the other side of 2 inches of fabric, foam and wood, I hoped the participant would temporarily feel transported to another place. I felt that by creating this distinct location and mood, participants would be more willing to give honest, thoughtful answers. It would not have been as effective to simply use a plain wooden box or booth. Although a plain box would have been functionally almost identical, the participants would be less likely to answer personal questions. By utilizing the ideas of death and the afterlife, the participant may unconsciously suspend disbelief and offer answers which she might not otherwise.

4. Participant Reaction

@wake was exhibited for six days at The Loading Bank Gallery in Brick Lane, London along with a variety of digital exhibits by others. I was very pleased by the reaction from participants. Perhaps most impressive were the length of time spent in the coffi n and the quality of answers given. Many users stayed in the coffi n well beyond the conclusion of their participation in order to listen to previously recorded answers. The answers themselves were surprisingly sincere and personal, I think largely due to the mood created by the installation. The questions asked were under the pretense that the participant had recently died, though this fact was implied not explicitly stated. They revolve around regrets, life-evaluation, and so on. Consequently, the answers tended to be of a confessional nature, often going well beyond what I expect someone would tell a complete stranger under other circumstances.

Allowing participants to leave something behind for future visitors creates a sense of community amongst those who have entered (and will enter) the coffi n. Unlike participatory works such as Yoko Onoʼs Wish Trees [B9], in which visitors are encouraged to write wishes on bits of paper and hang them from a tree, the thoughts left behind in @wake are not accessible to casual observers. Because the previouslyrecorded answers are only audible from inside the coffi n, participants can feel a sense of exclusivity – a private connection through time with past and future participants. Both the human desires for voyeurism and self-expression are exploited. Non-participants (those who only watch from outside the coffi n) are given a hint of the experience via the infrared video display, but must enter the coffi n to fully experience the installation. The external video display is primarily a means of tempting observers with a preview of the experience. It is also a tool to encourage crowds to form around the installation. Visitors seemed to be much more likely to enter the coffi n when other visitors (particularly friends and family) were there watching. This gathering of viewers around the coffi n is strangely reminiscent of mourners at a real funeral.

The issue of foreground vs. ambient information arises in relegating the playback of previous usersʼ comments to the sonic background. Much like MITʼs AmbientROOM project [B5] which allows its user to perceive information either as a primary or periphery source, @wakeʼs previously-recorded answers are designed to intensify the experience even when they are not the participantʼs primary focus. Like the ambient displays of AmbientROOM they convey information, but in so doing serve as much to create a mood as to tell the user anything in particular. During the experience, those comments are often unintelligible either due to distortion (echo/reverb) or because they are drowned out by the foreground audio (the questions being asked). Even though the participant is not always able to understand the voices speaking in the background, she is still aware of their presence. With a conscious change of focus, she can generally move away from the foreground information and have a more detailed understanding of the background. Once the user is satisfi ed, the primary audio interface can once again come to the foreground of her attention.

Most computing systems cater only to the userʼs primary focus of attention [B4] – the video display. It is increasingly recognized however that humans are capable of accepting information from periphery sources as well as their primary focus [B6]. The fact that humans can accept this periphery data while simultaneously focusing on a primary data source allows us to create a much richer experience by taking advantage of ʻambient displaysʼ – sources of information which not necessarily meant be the primary focus of attention [B6].

Because I was able to both build the physical components of the installation as well as write all of the necessary code to run it, I had complete control over all aspects of the fi nished work. This control allowed me a great deal of freedom to change and fi ne-tune everything from the physical appearance and functionality of the coffi n to the delay between questions and amount of reverb applied to audio playback. The ability to make subtle changes was crucial to the installationʼs success.

5. Non-Command Interfaces

This installation piece utilizes a type of non-command interface. Rather than having to issue some sort of command to the computer system, the user only needs to close the coffi n lid to begin the experience. By entering the coffi n and closing the lid (both part of the overall experience), the user unknowingly activates a trigger which alerts the computer system to begin. To end the experience, the user needs only to open the lid and leave the coffi n. The process is intuitive and automatic, never requiring the user to consciously ʻstartʼ or ʻstopʼ it. After watching users interact with the project, the only part which seemed to confuse people was realizing that they could enter the coffi n in the fi rst place. I had assumed that the open coffi n would be an adequate invitation, but some visitors did not realize that they were meant to get into the coffi n. When exhibited in a gallery dominated by screen-based exhibits, this was not entirely surprising. The keyboard/mouse/monitor computing paradigm is strongly entrenched. It may be diffi cult for many people to immediately recognize that a different sort of interface is, in fact, an interface at all.

6. Customized Hardware Interfaces

Both personal computers and video game systems have, over decades, settled into standard, multipurpose interaction devices. With few exceptions, computer users are restricted to the mouse and keyboard. Similarly, the vast majority of video game consoles are controlled by generic gamepads for everything from driving games to sports games to role-playing games. A game-specifi c (or application-specifi c) device such as a steering wheel controller is often much more effective [B8]. Following this approach, like Shawʼs The Legible City [B1], @wake uses a non-standard means of interacting with a computer system to increase its effectiveness. Shawʼs work uses the metaphor of bicycling through a city while @wake uses a funeral, tailoring the physical experience to the metaphor. Just as The Legible City would lose much of its appeal if the user were forced to maneuver using a joystick rather than by peddling and steering a bicycle, @wakeʼs appeal would be diminished by anything reminiscent of a standard computer interface.

7. Future Works

For future showings of @wake, further work is needed on three aspects: enticing participants to enter the coffi n; increasing the accuracy of the speech recognition so that the participant is as unaware of the mechanical aspects of this as possible; and developing greater subtlety in the routines by which the participantʼs responses guide the ʻconversationʼ.

Other works now in progress will continue to explore the themes addressed by @wake, combining specialized digital and physical components to create unique interactive spaces.

8. Bibliography

1. Abel, Manuela and Staff, Heike, “Legible City”, http://www. jeffrey-shaw.net/html_main/show_work.php3?record_id=83.
2. Arnaud, Danielle, “Fair Play”, http://www.piffl e.demon.co.uk/ articles/catalogues/cat-fair.htm, Oct. 2001.
3. Bureaud, Annick, Leonardo Electronic Almanac, vol. 5, no. 3, http://mitpress2.mit.edu/e-journals/LEA/TEXT/lea5-3.txt, Mar. 1997.
4. Heiner, Jeremy M. et al., “The Information Percolator: Ambient Information Display in a Decorative Object”, CHI Letters, Vol. 1,1, 1999.
5. Ishii, Hiroshi et al., “ambientROOM: Integrating Ambient Media with Architectural Space”, CHI 98 Papers, 1998.
6. Ishii, Hiroshi and Ullmer, Brygg, “Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms”, CHI 97 Papers, 1997.
7. Jerz, Dennis, “Eliza”, http://jerz.setonhill.edu/if/canon/eliza.htm, Mar. 06, 2001.
8. Laurel, Brenda K., “Interface as Mimesis”, User Centered System Design, Lawrence Erlbaum Associates, New Jersey, 1986.
9. Mieszkowski, Katharine, “culture shocked”, The San Francisco Bay Guardian, Nov. 20, 2002.