Assessment of the effectiveness of the video game was a critical component of the project and as a result a detailed game assessment process was developed. The assessment results of the player can be analyzed at the end of each game, and the cumulative results of all players are stored on a relational database. Because of the importance of validating that actual learning outcomes have been achieved for such a life threatening situation, the Chicago Fire Department is considering adopting the game as one of its methods of teaching fire safety.

Introduction

Games are not just about entertainment.1 Games that go beyond merely entertainment are referred to as “serious games” and are used for interactive education, communication, and art. The idea of serious games was developed in 2002 by game developers when they realized that the simulations and environments created in their games could be applied to problems in the real world.2 Corporations, government agencies, and educational institutions are using the commercially available software used to create video games as a new way of developing simulations and training applications. This has resulted in a demand for applications that deal with “serious” purposes other then entertainment.3 There is now a growing market for game developers who produce products for both economic and social benefit.4 Games for Change (G4C) is an initiative that uses games to advance organizational mission and societal changes. Some of their recent projects include working with Microsoft in the Xbox 360 Challenge, a worldwide competition for college students to develop the best game based on the theme of global warming. G4C is also part of the PETLab project that will work with teens to develop a social issue web-based game.

Figure 1

Columbia College has recently completed development of the game High rise Evacuation Learning Platform (H.E.L.P.). H.E.L.P. is a game that uses the core software components and real time graphics capabilities of the Unreal game engine to realistically simulate a fire in a high rise. The goal is to successfully exit a building when faced with different real-life situations. Through simulation the player learns the appropriate responses to each life threatening situation. (See Figure 1.)

Figure 2

H.E.L.P. was developed using the Unreal Game Engine and because of the richness of the 3D elements and virtual world capabilities available, (See Figure 2). The resulting game has the appearance and game play elements found in the most popular video games. This helped bridge the gap between entertainment and the effectiveness of serious games. None of the learning outcomes of the game had to be sacrificed to give it the playability demanded by the current video gaming community. Playability can be measured by evaluating the game play, game story, game mechanics, and game usability. Each component contributes to the player’s overall positive experience by increasing the player’s enjoyment, immersion, or challenges of the game.5

H.E.L.P., as an educational tool that can be used to teach high rise occupants what to do in case of fire, has an assessment component that measures the efficacy of the game as a learning method, in contrast to more traditional methods such as video and written instructions.6 The learning outcomes of the H.E.L.P. game were defined by the Chicago Fire Department based on the C.A.L.M. model of fire response: Call 911 if you see or smell smoke, Alert authorities, Listen to authorities and follow instructions, and Move safely and quickly exit the building. The five learning objectives of H.E.L.P are: 1) participants will understand the four steps of the C.A.L.M. model of fire response; 2) they will be able to identify a safe exit route; 3) develop basic escape strategies; 4) know who to alert in case of a fire, and 5) know if they are designated to help others to safety.

H.E.L.P. uses four narrative scenarios to review and reinforce the learning goals identified in the Chicago Fire Department’s C.A.L.M video, one of the teaching tools used by the Chicago Fire Department in addition to fire drills and safety lectures. (See Figure 5 Scenarios)

Scenario 1: It’s late at night and the player is alone working in his cubicle on an important project that is due tomorrow. He smells smoke and then discovers smoke and fire coming from under the door of a locked closet. The door is very hot to the touch.

Scenario 2: It’s a normal workday morning. The player is in his office when the fire alarm sounds. Moments later a co-worker, who is the floor’s designated Fire Marshal, opens the player’s door and instructs the player to follow him to the stairwell.

Scenario 3: It’s Saturday afternoon and the player is alone working in his cubicle on an important project that is due on Monday. He smells smoke. The player is unable to figure out where the smoke is coming from.

Scenario 4: It’s Saturday afternoon and the player is alone working in his cubicle on an important project that is due on Monday. He smells smoke. The player is unable to figure out where the smoke is coming from and both exits are blocked.

Figure 5

Figure 3

THE H.E.L.P WORLD takes place in a typical high rise office building. The player is working on the 6th floor of a 20 story building. The player’s workspace is a cubicle equipped with a phone with a direct line to building security. The player is able to place phone calls, open and close doors, approach fellow coworkers, grab objects to seal doors, walk around the office floor, and run down the stairwells. There are many objects throughout the building that the player can interact with to enable him to safely exit the building. These include the office phone, doors, and the coat rack. (See Figure 3.)

Figure 4

Characters were modeled in 3D using Autodesk’s Maya 3D animation software.7 Reference photos of Chicago firefighters and models were used for texture mapping. (See Figure 4.) Environments were either modeled specifically for the game or purchased online. Models were exported to the Unreal Engine as game assets for the programming team. Animation was created by Chicago firefighters and actors specializing in motion capture performance using Motion Analysis.8 The motion capture data was assigned to the character models using Autodesk’s Motion Builder and Maya’s 3D animation software. Animation files were exported to the Unreal Engine as game assets for the programming team. The game was developed using the Unreal 2 game engine licensed through Epic Games.9

There are two original music scores that are used repeatedly throughout the game. The first entitled VictoryMusic is heard when the player has successfully completed one of the four scenarios. The second entitled DeathMusic is heard when a player fails to complete one of the four scenarios correctly.

Voice dialogues are used throughout the game to provide instruction or to enhance the reality of the experience. Examples of the dialogues are: A male player calling 911; a female player redialing 911 when trapped in office; a panicked coworker and the 911 operator.

In order to create realistic ambient sounds during game play sound effects will be heard depending on the location and game state of the player. Examples include: fire engines, foot steps, phone dialing, fire alarm, street sounds.

The game play is as follows. The user plays a Columbia College employee who works on the sixth floor of a 20 story office building at 623 S. Wabash Ave. The character does not have a cell phone, and the building has no external fire escapes. While working, the character is caught in four different high rise fire scenarios and must react in a certain way to safely and successfully escape the building. If the player does not follow the correct procedures, the attempt at escaping fails and the scenario must be repeated. In order to win the game, the player must successfully complete the following steps in the sequence appropriate to the scenario (see Appendix A Victory Conditions). A number of conditions will result in the player losing the game. In any Scenarios, players will lose if they fail to follow the correct sequence of events or attempt to exit on any floor other than ground floor. In Scenario 2, players will lose if they approach the panicking office worker and

attempt to assist the fire marshal. In Scenario 4, a player will lose if they open the alternative stairwell door when the hand control shows that the door is warm or hot or if they fail to seal the door.

The effectiveness of the game to train office personnel on safe building evacuation was a primary goal and the game play and feedback had to be constructed to achieve the learning objectives. In order to gauge how well the game met the educational goals a detailed game assessment process was developed.

The efficacy of the game as a learning method was evaluated using a questionnaire that was given to four groups.

Group 1: Control group of individuals who have had no previous training in high rise building evacuation.

Group 2: Individuals who have viewed the C.A.L.M. video

Group 3: Individuals who have played H.E.L.P.

Group 4: Individuals who have viewed the C.A.L.M. video and played H.E.L.P.

Each group, with the exception of the control group, was given a questionnaire after viewing the C.A.L.M video, playing H.E.L.P., or both. The questionnaire was divided into two sections. One section comprised of 6 questions was used to gather the background of the participants in the study, while the other section comprised of 11 questions measured their building evacuation knowledge. A comparison of results was used to ascertain how well each group performed in regard to the learning outcomes of H.E.L.P.

The control group survey was sent via email to all faculty and staff of Columbia College Chicago who were invited to participate in the study. Of approximately 900 invitees a total of 87 responded.

The final three groups were given packets containing the game CD, install instructions, and a link to one of the three surveys. They were randomly given packets that would place them into one of the three control groups. In order to participate in the study the volunteers were asked to sign a roster indicating they had received the CD and would be expected to return the packet after completing the study. They were also asked for any feedback they would like to share with the developers about the experience.Table 1 Summary of Assessment Results

Table 1 Summary of Assessment Results

Conclusions

The results of our study show that there was an increase of correct answers to survey questions as a result of playing the game. Because of time constraints, we were unable to gather larger populations within each assessment group. However, even with our sample size, our results indicate that playing the game clearly met the learning objectives of our project. More time and participants, we believe, would strengthen our findings.

Comments from participants provided us with additional information on how to improve the game and make it more playable by a larger demographic. This project has provided significant insight on how serious games should be constructed and how to create methods of assessing their effectiveness. We hope to apply what we have learned to future projects as well as continue our research of video games as an important and better way of learning.

Acknowledgements

Our thanks to Joseph Cancellaro, David Gerding, and Janell Baxter who were our Columbia College faculty team members. Special thanks to Dr. Warrick Carter, President, Columbia College Chicago, Dean Doreen Bartoni, Chairs Annette Barbier and Bruce Sheridan, Commissioner Raymond Orozco, Chicago Fire Department, The Office of the Commissioner, Chicago Fire Department, and the Army Research Office, Aberdeen, Department of Defense.

Appendix A Victory Conditions

Scenario 1

• Dial 911
• Press the building security button on phone
• Navigate to the primary exit stairwell
• Walk down each flight of stairs
• Use the exit door on the ground floor
• Exit the building through the ground floor main doors

Scenario 2

• Walk down each flight of stairs
• Use exit door on ground floor
• Exit the building through ground floor main doors

Scenario 3

• Dial 911
• Press the building security button on the phone
• Navigate to the primary exit stairwell
• Walk down each flight of stairs until smoke is seen
• Walk back up the stairs and return to original floor location
• Navigate to the alternate exit stairwell
• Use the exit door on the ground floor
• Exit the building through ground floor back door

Scenario 4

• Dial 911
• Press the building security button on phone
• Navigate to the primary exit stairwell
• Walk down each flight of stairs until smoke is seen
• Walk back up the stairs and return to the original floor location
• Navigate to the alternate exit stairwell
• Approach the door until the hand control shows that the door is warm/hot
• Return to the original office location
• Dial 911
• Navigate to the coat rack
• Grab a coat
• Seal the office door using the coat
• Navigate toward a window and signal location to firefighters

Footnotes

1. S. Glazer, “Video Games: Do they have educational value?” CQ Researcher 16 40 (2006): 937-960.
2. D. Rejeski, “Serious Game Initiative.” (2002) http://www.seriousgames.org/about2.html.(Accessed September 14, 2007).
3. R. Naish, “Why games are so serious,” E.learning Age (April, 2005).
4. D. Michael, The Serious Side of Having Fun. Serious Game Summit. Washington, D.C. (2004).
5. D. Heather, C. Martin, et al. Using heuristics to evaluate the playability of games. CHI ‘04 extended abstracts on Human factors in computing systems. Vienna, Austria, ACM. (2004)
6. M. Prensky, “Don’t bother me Mom, I’m learning!” : how computer and video games are preparing your kids for twenty-first century success and how you can help! (St. Paul, Minn., Paragon House. 2006).
7. Maya is a 3D modeling and animation software program used in feature films and games including King Kong (Weta Digital), Transformers (Industrial Light & Magic), Stranglehold (Midway Games), The DaVinci Code (Double Negative) and Surf’s Up (Sony Picture Imageworks).
8. Motion Analysis provides optical motion-capture systems in entertainment, video games, and film. Recent film projects include King Kong, Lord of the Rings: The Return of the King (Weta Digital), Stranglehold” (Midway Games), and I, Robot (Digital Domain).
9. Epic Games Inc was founded in 1991 and is a leading independent developer of 3D games and core technology. Unreal Engine 2 is a complete game development framework. Epic’s Unreal Engine has powered over 50 titles selling a cumulative total of over 16 million copies, including Splinter Cell and Rainbow Six 3 (Ubi Soft); Deus Ex and Thief 3 (Eidos); Lineage II (NCSoft); Harry Potter and Ultima X (Electronic Arts); and America’s Army (US Department of Defense). Unreal Engine powered games have shipped on Xbox, PlayStation2, GameCube, DreamCast, Windows, Linux, and MacOS X.

Works Citied

Carlson, S. “The Net Generation Goes to College.” The Chronicle of Higher Education (Information Technology) (2005): 34.

Games, E. (2007) http://www.unrealtechnology.com/html/homefold/home.shtml. Accessed October 26, 2007.

Glazer, S. “Video Games: Do they have educational value?” CQ Researcher 16 40 (2006): 937-960.

Heather, D., C. Martin, et al. Using heuristics to evaluate the playability of games. CHI ‘04 extended abstracts on Human factors in computing systems. Vienna, Austria, ACM. (2004).

Michael, D.. The Serious Side of Having Fun. Serious Game Summit. Washington, D.C. (2004).

Naish, R.. “Why games are so serious.” E.learning Age (April 2005).

Prensky, M.. “Don’t bother me Mom, I’m learning!”: how computer and video games are preparing your kids for twenty-first century success and how you can help! St. Paul, Minn., Paragon House. (2006).

Rejeski, D.”Serious Game Initiative.” (2002) http://www.seriousgames.org/about2.html (Accessed September 14, 2007).