ABSTRACT
Operators had difficulty building or maintaining situation awareness, and spent over one half of their time trying to do so. Figure 6 summarizes the percentage of statements the operator made in each category-sender-recipient dyad, function, form, and content. As shown in Figure 6d, 54% of operator statements were related to gaining situation awareness at Levels 1 and 2 (state of robot and robot situatedness, 38%; state of environment, 13%; and information gathered, 3%); considerably less time was spent talking about factors requiring situation awareness (Level 3) to perform (navigation, 21%; search strategy, 16%).Relations between elements in the dimensions of content and function captured indicators of operator situation awareness. The correlation matrix of operator statement categories (Figure 7) revealed operator statements related to search strategy were strongly correlated with statements related to the state of
the environment (r= .94) and state of information gathered (r= .89). These two situation awareness-related content areas were closely tied to each other (r = .91) as well, indicating the importance of linking what is being observed in the environment with what the operator already knows about the environment.Search strategy and planning are an intuitive fit because of the need to plan search activities, and indeed, search strategy statements correlated with statements coded as planning (r= .95) in the function category. However, the significant correlation of planning (a situation awareness Level-3 indicator) with the state of the environment (r= .98, p < .001) emphasizes the necessity of perception and comprehension in performing search operations. This requirement is confirmed by another important relation in this category between the two functions of plan and report (r= .93.) The report element is used strictly when the operator is reporting on the state of the robot (including
situatedness), environment or information gathered, all indicators of perception, and comprehension (Levels 1 & 2 situation awareness.) The statistical association clearly ties situation awareness to operator planning (Level 3) in human-robot interaction. 5.2. Team Process and Communication
Operators demonstrated team-based processes and communication techniques while using the robot in search operations, a finding supported by statement frequencies, percentages, and correlations between statement categories. Results are first presented for the 272 statements made by the operators to team members, because the study’s focus is on the operator’s mental model and situation awareness. Additional results examining operator and team member statements are then presented. Figure 6a provides frequency and percentage of occurrence of each descriptor by coding category. As shown in Figure 4, operators spoke to other participants approximately four times per min while teleoperating the robot (M= 4.4, SD= 1.17 statements per minute).Correlations of operator statement form with content (Figure 7) suggest operators’ instructions were related to search strategy, the state of the environment, and state of information gathered (rs = .99, .95, .92, respectively.) In addition, instruction statements made by operators correlated significantly with statements coded as having a planning function (r= .94.) This result suggests operators were attempting to develop a shared mental model with teammates to increase situation awareness. They also used this information to plan and devise search strategies. The report function used in the coding scheme was defined as “reporting about the state of the robot, environment, or information gathered”—all situation awareness-related topics.Our results show that reporting and planning were closely related (i.e., operators were using what they were seeing through the robot’s eye to form a mental model of the search space, and the robot’s position in that space, to devise search strategies). Planning not only facilitates the building of shared mental models with teammates, it also can result in improved team performance (Stout et al., 1999). It is surprising that navigation statements correlated only with statements function coded as conveying uncertainty (r = .93). This finding may reflect the lack of situation awareness in two of the operators.Although the primary focus of this article is on operator situation awareness and how operators talk to team members to facilitate situation awareness, we did further analyses to explore information exchange between dyads on the team. That is, we examined operator statements both to and from primary rescue team members (operator, tether manager, team member, and researcher-robot specialist). In this analysis, we examine (by dyad) the fre
quency of statements based on form, content, and function combined to give an integrated picture of information exchange between rescue team members (e.g., the operator asked the tether manager a question seeking information about the state of the robot).Naturally, at this level of detail, the number of possible combinations (four forms, seven topics, and seven functions) is formidable. Therefore, Figure 8 presents only the three highest frequency statement types (including ties) for each dyad; the percentages shown do not sum to 100%. Operator statements reflect specific expectations regarding the nature of each team member’s roles (see Figure 9). The data suggest team members did not share the operator’s role expectations. For example, although team members provided information on the robot and the environment, and provided instructions for navigation, they paid little attention to search strategy. In addition, tether managers provided information on the robot and its situatedness; however, they mainly provided instructions regarding navigation. This suggests operators saw tether managers as a resource for obtaining information, whereas tether managers saw their role as providing assistance with navigation. Although the operator saw team members as problem holders, sharing pertinent information about the state of the robot and the environment, and collaborating on search strategy, team members did not address operator needs regarding search strategy. 5.3. Interaction of Situation Awareness and Team Communication
As mentioned previously, the RASAR CCS obtains global assessments of situation awareness for each operator on a 5-point scale ranging from 1 (low) to 5 (high). These ratings were used to identify operators with high versus low situation awareness. Data from two operators receiving a rating of 1 were combined to form a low situation awareness group, and similarly, data from operators receiving a 4 or 5 were combined to form a high situation awareness group (data from one operator receiving a 3 were not used in this analysis). Chi-square analyses are computed to determine differences in high and low situation awareness operator statements relative to who the operator was communicating with (dyad), the statement form, content, and function.Comparisons between operators rated as having high versus low situation awareness on a global rating scale offer support for the influence of team behaviors on situation awareness. Chi-square results (Figure 10) suggest operator communication with the tether manager, %2(1, N = 2) = 16.2, p < .001; and with other team members, %2(1, N = 2) = 18.6, p< .001; was related to high situation awareness. High situation awareness operators also provided instructions more frequently than their low situation awareness counterparts, %2(1, N = 2) = 4.5, />< .05.
