A systematic review of mode awareness measurements for automated driving
Résumé
Self-driving cars are supposed to be commercially available to the public within the next decades. Until then, intermediate levels of automation cohabit. A safe cohabitation between modes implies that the drivers has an efficient awareness of automation mode to assume the role they have to play in the collaboration. Mode awareness, according to Othersen (2016), is the system specific subcategory of situational awareness. When mode awareness is low, it can result in either mode confusion or mode error. Mode confusion refers to a situation in which the user (here, the driver) thinks he/she is in a different mode that it actually is. Mode error refers to the execution of intention that is adapted to one mode that is not the one currently operating [3]. Mode confusion and mode errors are particular cases of automation surprises, which are situations in which the behavior of a system is different from the expectation of the user. Such situations are to be avoided to assure the driver’s safety. Yet, studies proved that today’s vehicles provoke confusions between modes. With an effective design of interfaces informing on automation status, it is possible to reduce mode confusions [5]. Interfaces should then be evaluated to estimate their impact on mode awareness. In order to evaluate interfaces, it is necessary to be able to measure mode awareness. This study aims to present a systematic review of tools used to evaluated mode awareness during human-automation interaction.Method: The method of systematic review used in this study was inspired by Mirnig et al. (2017). Academic publication were gathered through Google Scholar’s database, EBSCOhost and Web of Science. The citations of the collected articles were then crossed to assure that no publications were missed. We employed Boolean algebra’s AND- and OR- building three groups of keyword. Within each group, keyword were connected with the operator - OR. The three groups of keywords were connected with the operator -AND. The first group was related on the type of vehicle and the automation. The second group was related to the interface. The third group was related to the mode identification. The following keyword research is an example of syntax on Google Scholar: ("Automated driving" OR "conditional automation" OR "partial automation") AND ("human-machine interface" OR "driver-vehicle interface" OR "feedback") AND (“mode awareness” OR “mode errors” OR “mode confusion”). The research was conducted without limitation of date. Publication ranged from 1988 to 2019. The paper collection allowed to gather a total of 218 articles : 199 from Google Scholar, 8 from Web of Science, 6 from EBSCOhost, and 5 from citations and personal income. After reading the articles, 23 were kept as they evaluated specifically mode awareness.Results: Within the 23 final articles evaluating mode awareness, 13 studies referred to mode awareness, 8 referred to mode confusion, 2 referred to mode error. Several tools were used to evaluated mode awareness. Among these tools, two categories of tools were revealed: online and offline. Online tools allowed to gather data during the experiment, like the percentage of correct identification with the Freeze-probe technique, duration of eye fixation, reaction times to failures, unnecessary deactivations of automation, and execution of secondary tasks. Offline tools allowed to obtain data after the experiment, and were self- evaluated rating scale, surveys, rating scale filled by the experimenter, and coding of behaviors.Conclusion and perspectives: The systematic review of mode awareness evaluation revealed that several tools can be used. Some of them allow to gather data online, whereas others allow to gather data offline, after the experiment. One advantage of online tools is to identify the situations that caused decreases of mode awareness. It would be beneficial to investigate visual or auditory alerts to inform on mode transitions in particular situations. Moreover, online tools would allow to investigate, in real time, the difference between the believed mode and the real mode, which could be modeled following the Hidden Markov Framework. Such model would allow to evaluate, online, the efficiency of an interface regarding mode awareness. The freeze probe technique would allow to gather data useful to build this model. However, it implies to pause the experiment, reducing immersion. Therefore, measures such as eye fixations or reaction time to failures seem to be more suited as they allow to gather data online without having to pause the experiment. Regarding offline tools, they allow to obtain a global overview of mode awareness. It is useful to assess the global efficiency of the interface as well as its acceptability. Moreover, they are less intrusive than online tools like eye fixation. Therefore, they are recommended as complementary or “quick and dirty” evaluations of interfaces. Yet, no tools allowed to investigate mode awareness, mode confusion and mode errors at the same time. Therefore, combination of measures should be used. For example the freeze probe technique associated with the coding of behaviors would allow to investigate the identification of mode in its globality. To conclude, we highly recommend to investigate online tools that allow to gather data based on the behavior of the participants. They allow to gather data based on ecological situations and to evaluate interfaces in real time to ensure that they are efficient in any situations.