1. Introduction

The implementation of real-time distributed interaction between researchers, educators and students has often been limited to “talking head” forms of interaction, even when cutting edge distributed virtual environment and co-presence technologies have been employed [1], [2], [3], [4], [5]. Understandably, educators in specifi c domains are usually concerned primarily with the content and practices of their own domain. So it follows that their approach to the use of new media in educational settings, whether live or virtual, will tend to follow the traditional form prescribed by their core discipline. In the view of the authors, this prescriptive approach can be augmented signifi cantly with new tools and techniques to foster a more effective and engaging environment for learning. A number of experimental media events, techniques and environments have been developed at the University of Floridaʼs Digital Worlds (DW) Institute over the past three years. These developments indicate that both interpersonal and intercontinental collaborations can be effectively facilitated and enhanced with new tools being developed across the diverse disciplines that are converging around the nascent fi eld of Digital Media.

2. Related Prior Work

In the spring of 2001, a computer scientist invited one of the authors to the Physics building at the University of Florida (UF) and gave a brief introduction to an emerging technology called the “Access Grid” (AG) [6]. The author foresaw an immense potential for the AG system beyond its initial adoption by the scientifi c research community. A proposal was subsequently submitted to the 2001 global SuperComputing Conference (SC2001) to showcase real-time collaboration between artists and engineers located across two continents. The resultant project, entitled “Dancing Beyond Boundaries” (DBB) joined master percussionists in Brazil with musicians and dancers at the University of Florida, a choreographer and dancers in Minneapolis and dancers performing inside a three-walled virtual environment on the SC Conference fl oor in Denver. The resultant premiere at SC2001 garnered the award for “Most Creative and Courageous” use of the high-speed network in the High-Bandwidth Challenge. This initial effort received substantial exposure in both the research and commercial world [7], [8].

Building upon the technical success of DBB, Australian media artist Kelli Dipple undertook a residency at the UF Digital Worlds Institute in Fall 2002. Working in the DW NAVE Lab, Dipple and students in the Digital Arts and Sciences program created and premiered a threecontinent collaboration uniting performers and technologists in Australia, Florida and England entitled “Navigating Gravity.” To facilitate these distributed collaborations, a host of specialists from traditionally disparate fi elds were required (i.e., network engineers, digital animators, computer scientists, theater designers, audio and video technicians, television producers and directors, camera operators, composers, musicians, dancers and choreographers).

Although these initial projects focused on collaborations in the performing arts, it became apparent that the underlying distributed collaboration systems and resultant techniques developed at Digital Worlds had tremendous potential for effective real-time group interaction across the variety of domains converging around the fi eld of Digital Media.

3. Underlying and Enabling Technologies

Although educators have been quick to adopt video conferencing as a classroom technology preferable to other communications devices such as the telephone, late twentieth century quality of service issues (including lack of bandwidth and ineffi cient video compression /decompression (CODEC) schemes) initially created usability problems in early deployment scenarios [9]. As bandwidth increased and CODECs have matured, distributed collaboration in the sciences has become more commonplace with the proliferation of high-speed backbone networks like Abilene, STAR TAP, DANTE, GWIN. Centered in the United States, the Internet2 consortium supports various initiatives including the Access Grid (AG) [6], the Virtual Rooms Video Conferencing Service (VRVS) [10] the Virtual Auditorium of Stanford University [11] and the new Global Conference System [12].

Because of its ability to support multi-perspective inter-nodal communication, the Access Grid (AG) has been used by DW as the primary means of linking geographically divergent partners during its initial distributed collaborations. The Access Grid is an ensemble of resources used to support human interaction across the Internet. It consists of multimedia displays, multipoint bi-directional audio, cameras, and shared virtual “venues” for presentations and interactions. See Figure 1.

FIGURE 1. A Basic Access Grid (AG) node confi guration

A distinguishing feature of the Access Grid is its ability to capture up to four video images simultaneously and transmit them to all of the other collaborating locations. This multi-cast capability enables each node to see and talk to each other (as in a standard point-to-point teleconference) with the added feature of multiple perspectives shared across multiple nodes simultaneously. This enables participants (whether across campus or across the world) to interact in a naturalistic fashion, compared with systems that relegate participantsʼ images to avatars or other abstracted iconic representations [13], [14]. The depth of communicative cues offered by real-time realistic video has been incorporated into a number of projects aimed at facilitating more effective distributed collaboration. Indeed, one of the more promising developments is the ability to composite multiple persons into a shared virtual environment simultaneously, while at the same time retaining the individualistic expressions and gestures of each of the participants [4].

Figure 2. P. Pagano in the VPS (top) and at two subsequent virtual “beaches”: Brighton, UK (middle) and Miami, USA (bottom).

An ongoing development at Digital Worlds is Patrick Paganoʼs work on a low-cost tool for compositing live humans (whether teachers, performers or students) into virtual or realistic environments as a means of heightening the effectiveness of the video imagery transmitted across the AG. While there are a number of commercially available real-time video compositing tools, they tend to be very expensive and/or require complicated hardware systems to operate. Paganoʼs solution is based on the initial work of Mark Danks [15 ] with the Graphics Environment for Multimedia (GEM). GEM was originally written to generate real-time computer graphics, especially for audio-visual compositions. Because GEM is a visual programming environment, users do not need any experience in traditional computer languages. Paganoʼs implementation is currently running on Mac OSX, linked into the AG as though it were simply another camera input.

This solution is at once affordable and scalable especially when used in learning environments where immersive visualization can heighten understanding of the lesson (i.e., examining multiple perspectives from within a complex dynamic system or moving around inside ancient structures for an anthropology or history class [16], etc.) . See Figure 2. Additionally, it is easily inserted into standard video conferencing systems (whether point-to-point or multi-cast) for educators working in a variety of domains.

Access to high-speed networks at educational institutions is becoming more prevalent; therefore it is assumed that the use of these enabling technologies can only become more widespread as the availability of bandwidth increases.

4. Design and Implementation of Flexible Digital Media Space: the REVE as prototype

While institutional access to raw bandwidth is on the increase, oftentimes the physical location of video-and virtual-conference systems create lack of access situations for persons not involved in the department that “owns” the facility. For example, high-end virtual environment systems are often located in Computer Science or Engineering units, and may not be readily accessible for educators from other departments. And although there are now nearly 200 Access Grid nodes deployed worldwide [6], these nodes have often been built in rooms that can seat only a dozen or so people at a time, if that many.

Observations such as these were key considerations in the design and implementation of the Research, Education and Visualization Environment (REVE) at the University of Florida. The REVE was purposefully designed to facilitate activities within one contiguous 5,500 sq. ft area with the following features:

1. A large green screen environment for acquisition of live video
2. Integrated production and post-production of digital media content
3. Ready access to Internet2 for real-time distributed collaboration (with live or pre-rendered components)
4. Flexible immersive classroom/studio space with large-scale visualization for up to 50 people
5. Interconnection of all of the above resources
6. Availability to educators and researchers for use across domains
7. Modular design so that the image generation and display systems can be readily and inexpensively upgraded as new technologies emerge
8. Research on Pedagogy, Interaction Design and Learning can be conducted onsite

A number of traditional and emerging models were examined and subsequently utilized in the design of the REVE. Several of the functional components for this facility were found in the broadcast industry (as well as fi lm and television post-production studios) of the twentieth century. Another contributing model for the REVE design was the convergence of high-speed networks and distributed collaboration tools (i.e., virtual and augmented environments) found in computer science and engineering research labs. Additionally, the proven social paradigms of the Classroom (where students and teachers gather face to face to engage in learning) and the Theater (where people gather to view and engage in performing arts – whether “real-time” or “pre-rendered”) were also incorporated into the facility design.

The fi nal design of the REVE, which opened in Fall 2003, sought to integrate the creation, processing, distribution and presentation of digital media into one fl exible physical location, and to make it accessible to practitioners from a variety of domains. Thus the entire community of researchers, teachers, artists and students are able to reserve and utilize the capabilities of the facility.

Figure 3 The Polymodal Immersive Theater (PIT)

The primary functional sections of the REVE are:

1. Polymodal Immersive Theater (PIT)
2. REVE Image Generator (RIG)
3. Virtual Production Studio (VPS)
4. Digital Media Suite (DMS)

The Polymodal Immersive Theater (PIT) in the REVE was designed to not only facilitate large group interaction at the local level (i.e., classroom of up to 50 people) but to also facilitate real-time connection across campus and around the world over the AG and other video conferencing tools.

Researchers in the fi eld of Education have offered a number of observations and subsequent theories on the effectiveness of electronic interaction in the classroom [18], [19], [20]. [21], [22]. A number of these studies have been confi ned to the paradigm of one or two users “shoulder to shoulder” in front of a desktop computer monitor [23], [24]. While the interpersonal skills developed in two-person teams is certainly useful, this model does not allow for advantageous group work [25] often found in the traditional classroom setup. In addition to use as a classroom or conference room, the PIT can be confi gured for live performances, with or without a local audience.

4.2 The REVE Image Generator (RIG) is a secure machine room where all of the computers and devices used in rendering the projected graphical elements are located. Current image generators are an SGI Onyx2 “Reality Monster” as well as a cluster of Dell PCs. A partnership with Trimension/SEOS has resulted in the acquisition and development of a real-time layering and compositing device (code-named SCORPION RT) which allows images from separate sources to be joined together seamlessly across the PITʼs 52 foot wide screen. Thus videoconferencing feeds and other live video sources can be superimposed into animated virtual environments, PowerPoint presentations, DVD playback or any number of sources.

4.3 The Virtual Production Studio (VPS) is a fl exible space that can be used for classes, rehearsals, performances, receptions and live video production. Its feature set includes a two-walled green screen, a video projection system, Internet2 /AG connectivity and a dedicated production entrance.

4.4 The Digital Media Suite (DMS) is a secure space located contiguously with the VPS. The DMS contains a variety of post-production tools, ranging from audio, animation, compositing, editing and DVD mastering tools. Direct network connection to the RIG allows fast transfer of media production elements and fi nished presentations for viewing in the PIT.

The most complex use of this facility to date was an international collaboration created for showcase at the Fall 2003 Internet2 meeting, held in Indianapolis, Indiana (USA). This project was entitled “NON DIVISI” (an Italian musical term meaning “not divided”). South American percussionists performed live with an ensemble of North American musicians located at the REVE, accompanying dancers in Seoul, Korea performing synchronously with dancers from Miami and UF and the dance soloist located at the Indiana State Museum (the Internet2 meeting venue).

“NON DIVISI*” brought into focus a variety of technical, artistic and cultural challenges inherent in real time distributed collaboration across half of the worldʼs time zones (including logistics across the international date line). See Figures 3 and 4.

Figure 3. Multi-window views from Indiana

Figure 4 . Multi-window views from Korea

Thus the REVE provided a functional convergence of the models upon which it was built, and was able to share the resultant international collaboration as both a real-time event and a subsequent multi-channel DM production.

For this DM presentation, all of the spaces of the REVE were linked together to form a synergistic environment incorporating:

• 5 live performance spaces (2 onsite at the REVE and 3 virtual)
• intercontinental multicast distribution
• a live audience at the REVE
• documentation and subsequent post-production of a DM artifact
• See Figure 5.

Figure 5. Images from the REVE spaces

As a result of the partnership forged in “NON DIVISI” a number of new projects and initiatives have been proposed. All of these projects are multi-continental in nature and would not be able to be realized without the technologies and techniques explored at the REVE for the Fall Internet2 meeting 2003.

Another international event facilitated at the REVE was produced for SuperComputing 2003. Entitled “Use of Collaborative Technologies: Artistic and Cultural Instincts,” the event joined researchers and participants from around the world and utilized the PIT projection screens to form a large-scale immersive conference environment. See Figure 6.

Figure 6. Large-scale AG Display at REVE PIT

5. Current Initiatives

We view the REVE as a prototype for a new generation of affordable, functional, scalable and updateable DM classroom/studio spaces. To utilize the fl exible structure of the REVE across domains, a number of current DM initiatives are currently underway.

• Collaborators from BioMedical Engineering, the McKnight Brain Institute, the Center for Pre-Collegiate Education and Training, the Digital Worlds Institute and the University College of Education have formed a team to study the “Development and Assessment of a Polymodal Learning Environment.” This team has proposed using the REVE as a virtual laboratory to study not only the design and presentation of digital media in a classroom setting, but how polymodal input activates various neural mechanisms underlying the learning process itself. This team has submitted a proposal to the National Science Foundation as part of its “Science of Learning” Centers initiative.
• Collaborators from the fi elds of Mechanical and Aerospace Engineering are combining both virtual and augmented reality systems in the DW NAVE and REVE to assist in the design and development of autonomous Micro Air Vehicles (MAVs).
• The REVE is being utilized as part of a study sponsored by the Ronald McNair Foundation, seeking to investigate how color and sound interact in the perception of mood in virtual space.
• Surgeons and researchers have created a proposal to investigate how virtual environments and related digital media techniques can be used to enhance both surgical practice and education. Extensions of this work will enable students to observe remote surgery and integrate virtual models and simulations with the live action footage from the operating room.
• Current DW international distributed collaborations are focussed on creating real-time interactive performances to study the potential for the use of gaming techniques and practices in classroom and virtual education environments. Digital Worldsʼ partners in this developing initiative include researchers at the Korean Advanced Institute of Science and Technology (KAIST), the Doncaster Education City (DEC), the Red Universitaria Nacional (REUNA), and the Royal Melbourne Institute of Technology (RMIT).

6. Future Research Trajectories

As DM systems become more complex, it becomes increasingly important to be able to differentiate whether discrete and aggregate system components are contributing to the desired result, or are creating undesirable “noise” in the system. To this end we are partnering with colleagues with expertise in interaction design and experimental assessment to determine the effectiveness of the various modalities used in DM-enhanced education and training. Parameters to be studied and assessed in relation to polymodal DM systems include:

• perceptual enhancement with localization of audio via multi-channel playback systems
• use of stereoscopic video display to enhance presence, focus and understanding
• relevant social factors in the design and implementation of DM leaning environments

7. Acknowledgments

For their signifi cant contributions to this work, the authors wish to acknowledge the following collaborators, colleagues and Institutions: Dr. Hyun Yang, Korean Advanced Institute of Science and Technology (KAIST), Dr. Florencio Utreras, Red Universitaria Nacional (REUNA) Dr. Richard Ferdig, Andy Quay, Joella Walz, Arturo Sinclair, University of Florida.

References:

1. Gernot Goebbels and Vail Lalioti, 2001. Co-Presence and Co-Working in Distributed Collaborative Virtual Environments. AFRIGRAPH 2001 Capetown South Africa Copyright ACM 2001 1-58113-446 -0/01/11.
2. Benford, Bowers et al. Supporting co-operative work in virtual environments. The Computer Journal, 37(8), 1994.
3. S.A. Bly, S.R. Harrison, and S. Irwin. Media Spaces: Video, Audio and Computing. Communications of the ACM, 36(1):28-47, 1993.
4. P. Kauff and O. Schreer: “An Immersive 3D Video- Conferencing System Based on a Shared Virtual Environment”, Proc. Int. Conference on Media Futures, Florence, May 2001.
5. O. Ståhl: “Meetings for Real - Experiences from a series of VR-based project meetings”, Symposium on Virtual Reality Software and Technology, UCL, London, December 1999.
6. AG Alliance: “Access Grid”, Home Page: http://wwwfp.mcs.anl.gov/ fl /accessgrid/ag-spaces.htm.
7. Oliverio, Quay and Walz, 2002. Facilitating Real-time Intercontinental Collaboration with Emergent Grid Technologies: Dancing Beyond Boundaries. Proceedings of the Virtual Simulation in MultiMedia Conference, Korea.. Design and Implementation of Accessible Digital Media
8. Cascone, 2002. NEXT@CNN. Transcript of television feature broadcast internationally April 27/28, 2002. http://www.cnn.com/TRANSCRIPTS/0204/27/nac.00.html.
9. AN IMMERSIVE 3D VIDEOCONFERENCING SYSTEM USING SHARED VIRTUAL TEAM USER ENVIRONMENTS Peter Kauff and Oliver Schreer (2002) CVEʼ02, September 30–October 2, 2002, Bonn, Germany. Copyright 2002 ACM 1-58113-489-4/02/0009.
10. VRVS: “Virtual Room Video-Conferencing System”, http://www. vrvs.org/About/index.html.
11. M. Chen: “Design of a Virtual Auditorium”, Proceedings ofACM Multimedia 2001, Ottawa, Canada, Sept. 2001 109 111.
12. Fuqua School of Business: “Global Conference System”, Press Release, Duke University, May 2002, http://www.fuqua.duke.edu/admin/extaff/news/global_conf_2002.htm.
13. Lester, J. Zettlemoyer, L, Gregoire, J. & Bares, W. “Explanatory Lifelike Avatars: Performing User Centered Tasks in 3D Learning Environments, in Autonomous Agentsʼ99, ACM Press, 1999.
14. Martinho, C. & Paiva, A. “Pathematic Agents: Rapid Development of Believable Emotional Agents in Intelligent Virtual Environments,” in Autonomous Agentsʼ99, ACM Press, 1999.
15. Danks, Mark (1996) Real-time Image and Video Processing in GEM. Proceedings of the ICMC. http://www.danks.org/mark/GEM/ICMC97.pdf
16. Oliverio and Sompura, 2003. Extending Virtual Heritage Beyond the Local Site: Creating An International Digital Network . Proceedings of the Virtual Simulation in MultiMedia Conference, Montreal, VSMM 2003
17. Roussos, Johnson et al.,The NICE project: Narrative, Immersive, Constructionist/Collaborative Environments for Learning in Virtual Reality. Proceedings of the ED-MEDIA/ED-TELECOM ʼ97, (Calgary, Alberta, Canada, June 14–19, 1997).
18. HILTZ, R. 1990. Evaluating the virtual classroom. In Online Education: Perspectives on a New Environment. Harasim L.M. and Turnoff M. Eds.Praeger, New York, 133-183.
19. Moher, T.: Constructing Collaborative Stories Within Virtual Learning Landscapes. Proceedings of European Conference on A.I. in Education, Lisbon, Portugal, 1996.
20. Druin, Stewart et al.. (1997). KidPad: A Design Collaboration Between Children, Technologists, and Educators. CHI 97.
21. HAZEMI, R., HAILES, S., AND WILBUR, S. (Eds.) 1998. The Digital University: Reinventing the Academy. Springer, London.
22. Benford, Benjamin et al: Designing Storytelling Technologies to Encourage Collaboration Between Young Children CH! 2000 o 1-6 APRIL 2000, 556-563.
23. Light, P., Foot, T., and Colboum, C., (1997) Collaborative interactions at the microcomputer keyboard. Educational Psychology, 7, 1, 13-21.
24. Bier, E. A., & Freeman, S. (1991). MMM: A User Interface Architecture for Shared Editors on a SingleScreen. UIST91, pp. 79-86.
25. Klein, B. 2000. An International Student/Faculty Collaboration: The Runestone Project. SIGCSE Bulletin 32(3), 128-131.