Gaming and Shared Situation Awareness

Summary

The Defense Advanced Research Projects Agency (DARPA) is interested in exploring key factors that affect how teams, particularly distributed teams, develop what is called shared situational awareness in an operational environment. The DARPA Program Manager for the Wargaming the Asymmetric Environment program asked the Center for Naval Analyses to address these issues, with subcontracting support from ThoughtLink Incorporated. The focus of the project was to demonstrate how wargaming could be used as a testbed for conducting experiments to explore these key factors in team shared situational awareness.

Our survey of the state of the art of research about situational awareness and shared situational awareness indicated some fragmentation of thinking; however, that thinking appeared to be beginning to coalesce into some weak agreement on fundamental concepts. Key among these is the notion that situational awareness is best denned as a dynamic mental model of our operating environment and our place in it. We build this model through a process we call situation assessment, which consists of four interwoven subprocesses: perception, comprehension, projection, and prediction. Similarly, for our purposes, we define shared situational awareness of a team as the overlap in the situational awareness of team members.

But how do we measure situational awareness, not to say shared situational awareness? We found no evidence of a clearly understandable, generally accepted, objective way to measure situational awareness and how it develops, whether for individuals or for groups. As a result, we chose to develop a fresh approach. Our goal was to devise an experimental regime that would allow us to define both an individual's situational awareness and that of a team in clear and understandable terms. In addition, we wanted to minimize the dangers that our attempts to measure situational awareness would affect the behavior of the test population.

Our approach was to develop an experiment in which multiple teams played an online game. We designed the game so that:

• Team members had to share information to do well
• Their decisions could be directly and easily measured and recorded
• The measure of their decisions would describe the degree of shared situational awareness of the team

We focused on developing a sample testbed and using that testbed in a limited-objective experiment to demonstrate the applicability and feasibility of this technique. The game we developed was SCUD Hunt. The experiment we designed used SCUD Hunt to explore the effects of communications and shared visualization tools on the ability of a distributed team of players to develop a shared mental model or operational picture of a simple operating environment. This operational environment consisted of a 5 x 5 grid of featureless squares on which 3 SCUD launchers were concealed.

Each player controlled one or two reconnaissance assets which could provide information about whether a particular square might contain a launcher. Players shared information about the search capabilities of their assets, the results of their searches, and their future search plans and attempted to identify and recommend a number of target squares they believed were their best estimates for containing launchers. We measured the efforts of the players to build their shared picture based on these recommendations and how well the recommendations overlapped among all the players on the team.

The overall experiment was designed in the form of a 6 x 6 Latin Square, in which each team played a well-defined series of 6 games, defined by combinations of 3 different types of communications capabilities crossed with two types of shared visualization tools. There were 6 four-person teams. Each team played the games in a different order so that we could control for possible learning effects.

The results of the experiment indicated that both communications and shared visualization played statistically significant roles in the ability of our teams to develop shared situational awareness. Perhaps surprisingly, however, there appeared to be no significant difference between the effects of voice communications and that of real-time text chat.

The broader results of this effort show that the use of simple games, designed to target specific experimental goals, is a promising technique for conducting research in this field. We collected significant amounts of data that we had neither the time nor the resources to analyze during this effort. Follow-on research to analyze this data may prove useful. In addition, we could apply the SCUDHunt game and experimental technique in support of the research directions taken by several leading practitioners in the field of situational awareness. Finally, a new research approach that combines the game-based environment of SCUDHunt with the burgeoning popularity and feasibility of agent-based computer simulation holds out the promise of exciting new techniques to supplement and help direct human-based testing in this field.

*Originally published in November 2000, this paper was reprinted in December 2024.