1. Introduction

Potential application of automated driving receives considerable attention from authorities and policy makers (Anderson et al., 2016; Kyriakidis, Happee, & de Winter, 2015; Shladover, 2017). While driving automation could increase traffic efficiency (Broggi, Zelinsky, Özgüner, & Laugier, 2016), reduce road accidents (Hoeger et al., 2008) and reduce the ecological impact of road transport (Wadud, MacKenzie, & Leiby, 2016), automated driving may also have an adverse impact on driving proficiency (De Winter, Happee, Martens, & Stanton, 2014; Gold, Damböck, Lorenz, & Bengler, 2013). While safe implementation of complete automation requires highly reliable machine-based sensing of the environment in combination with faultless understanding of its environment and decision-making, current systems do not meet these requirements (Bengler et al., 2014; Butmee & Lansdown, 2017). This is due to the highly complex nature of traffic circumstances. Implementation of automated driving is therefore targeted from levels of so called ‘semi’ or ‘partial’ automation to ‘high’ automation (SAE International, 2013) – but not (yet) full automation (Gasser & Westhoff, 2012; McGehee et al., 2016; Shladover, 2017). Subsequently, an important task is left at the driver to supervise safe system operation. For the foreseeable future, driving automation will therefore continue to rely on a “responsible” driver to oversee the technology, capable of resuming control and having the foresight to make operational decisions. Because the driver remains fully responsible, he or she is required to be any time available and to retake control in case automation fails or stops (Merat, Jamson, Lai, Daly, & Carsten, 2014). As a consequence, partial automation changes the driver’s task from actively operating the vehicle into supervising the system with occasionally a necessity for intervention (De Winter et al., 2014; Van den Beukel, Van der Voort, & Eger, 2016). The unexpectedness of required counter actions make intervention very demanding. Supervision is neither a task humans are good at, due to e.g. low vigilance and inappropriate levels of workload (Bainbridge, 1983; Martens & Van den Beukel, 2013). To summarize, partial automation creates a gap between drivers’ responsibilities and human capabilities for supervision of system operation when using driving automation. To reduce this gap, drivers of partially automated vehicles have need for interfaces that help them to supervise system operation, support detection of relevant events, as well as provide cues about system status (Johansson & Etemad, 2014; Van den Beukel et al., 2016). However, there is little research on how to provide such information to the driver in an effective and efficient way (Naujoks, Forster, Wiedemann, & Neukum, 2017).

As a first step towards developing interfaces that are adapted to the needs of drivers who are required to monitor system operation, this study investigates the potential support from ‘directional illumination’. Within this study’s scope, directional illumination consists of highlighting frame-sections around the windscreen and side-windows with coloured LED-strips. Size and location of the illumination is configured dynamically in order to indicate real-time when and where road-traffic events occur that require attention (cf. Locken, Muller, Heuten, & Boll, 2015; Pfromm, Cieler, & Bruder, 2013). Similar to a study from Baldwin and Lewis (2014) colour and brigthness convey level of urgency of the requested attention. A previous experiment confirms the potential to guide attention with directional illumination (Kelsch & Dziennus, 2015). Using cues to indicate where potential danger is located is especially important because automation has a larger adverse impact on Take-Over quality (i.e. obtaining safe road position; knowing what and where) than on the speed of taking over (Zeeb, Buchner, & Schrauf, 2016).

Because multi-modal signals influence signal effectiveness (Baldwin & Lewis, 2014), other modalities may be considered. Auditory signals are generally recommended as a base attention retrieving signal especially for urgent situations (Kiefer et al., 1999; Tan & Lerner, 1995). When used in addition to visual signals, auditory signals raise the level of perceived urgency (Wogalter, Conzola, & Smith-Jackson, 2002; Baldwin & Lewis, 2014). However, when used for directional cues multi-modal signals may be prone to misunderstanding, due to signals being misaligned or asynchrone (Lee et al., 2001; Baldwin & Lewis, 2014). This may be especially of danger for our intended directional quality in supporting driver’s attention. An example is when an auditory component produces a perceived hazard at a different location than where the visual cue hints at. Because the goal with our research is to pre-test the directional fashion of offering visual cues, the concepts in this study do not entail any differences in sensoric modalities other than visual.

In contrast to the quality of indicating where attention is needed, interfaces should ideally also indicate why attention is needed in order to improve (a) user’s understanding of automation-mode (like availability, activation or malfunction), (b) prediction of its future state and (c) required (re-)action. This is because the ability to regain control benefits from correct expectation of changes in automation-mode (Merat et al., 2014).

With regard to indicating where and why, directional illumination and traditional graphical interfaces show contrary qualities. Traditional interfaces provide especially information why attention is needed (Damiani, Deregibus, & Andreone, 2009), but the information itself often keeps the driver from observing where attention is needed. Studies confirm the visual distraction of graphical (screen) interfaces (Lansdown, Brook-Carter, & Kersloot, 2004; Rogers, Zhang, Kaber, Liang, & Gangakhedkar, 2011). Directional illumination, on the other hand, is designed to direct attention where needed. Nonetheless, drivers need to conceive a minimum understanding why attention is needed in order to comprise required action. When provided with directional illumination alone this task anticipation needs to be deduced from the directional cues and observed road-traffic events. Therefore, the aim of this study is to identify to what extent directional illumination is able to support drivers to obtain mode awareness and to understand their role.

This study is intended to pre-test task anticipation, before pursuiting driving simulator tests with future experiments. It therefore compares levels of mode awareness obtained when using directional illumination with results when using a base concept. Measurement is conducted with an internet-based survey showing videoclips with signal behaviour from either concept within situations representable for partially automated driving.

Section 2 presents the applied method to conduct this survey and explains how we measured mode awareness. Section 3 presents the results and section 4 discusses what these results mean for answering our research question. Finally, section 5 reflects with conclusions and recommendations on potential merits to apply directional illumination while helping the development of interface solutions that drivers need when driving partially automated.

2. Method

2.1. Concepts

The survey showed video-clips that included illumination behaviour of two concepts, called ‘illumination concept’ and ‘base concept’. Both concepts convey attentional cues on two levels. The first level concerns ‘soft’ warnings that request attention to potentially dangerous road-traffic situation. The second level comprises ‘hard’ warnings requiring drivers to retake control because automation stops. Based on previous research (Van den Beukel et al., 2016), both concepts use the same sound signals to distinguish between soft and hard warning. Both concepts also use a blue-illuminated ring around the steering wheel as an indicator when automation is active. In addition to the base concept, the illumination concept highlights frame-sections at the sides of the windscreen to provide a spatial cue where attention is needed. For example: Figure 1b shows a light-cue at the left side of the windscreen to warn for a cutting-in car. In parallel to sound, the illumination concept conveys urgency with colour: a request for attention (moderate urgency, i.e. soft warning) is combined with yellow illumination. Required take-over (high urgency, i.e. hard warning) comes with red illumination. When using the illumination concept, the assumption is that the directional light-cues improve driver’s observation of the road-traffic situation which in turn improves understanding why attention is needed (Merat et al., 2014). Therefore, the configuration of concepts was chosen to identify if the additional directional light-cue of the illumination concept raises understanding why attention is needed compared to the base concept without such cue.

Figure 1 

Illustration with concepts. (a) Shows the base concept with illumination around the steering wheel, but without ‘directional’ light cues. (b) The illumination concept provides in addition to the base concept ‘directional’ light cues. For example: yellow illumination of the left side of the windscreen to indicate a car cutting-in from the left.

2.2. Scenarios

Table 1 lists the scenarios used in this study with visual examples. Except for scenario 3, the scenarios are selected from a previous study (Van den Beukel & Van der Voort, 2017) and distinguish: (a) ‘hazardous’ scenarios when attention from the driver is requested (i.e. ‘soft’ warning), but without immediate danger for an accident; (b) ‘critical’ scenarios, which hold a danger for collision. During the second category the concepts raise a ‘hard’ warning and require the driver to retake control. The video-clips provide a simulated motorway environment in line with the described scenarios. The video-clips showed other vehicles on close distances, as to simulate traffic congestion. Timing when a soft warning or hard warning was raised depended on the scenario’s specific traffic situation and is explained in Figure 2. Each video-clip lasted between 80 and 100 seconds.

Table 1

Overview with scenarios shown in the video-clips of the survey.

‘hazardous’ scenarios (requesting supervision from the driver)

1 “complex road”
Attention is needed because the vehicle is approaching a combined entrance/exit ramp with the likelihood that vehicles will simultaneously enter and exit the main road.
2 “vehicle passing”
While the vehicle is in the left (fast) lane, extra attention is needed because a vehicle is nevertheless passing on the right.
3 “Approaching on/off ramp”
Attention is needed because the ego-vehicle approaches a highway and the entrance-lane is a combined on- and off-ramp. Attention is needed because vehicles can simultaneously enter and exit.
‘critical’ scenarios (requiring driver’s intervention)

4 “Merge-out target vehicle”
The target vehicle suddenly changes lane. If there is no new target vehicle ahead, the system stops and requests the driver to take back control. Accident avoidance involves preventing the vehicle from leaving its lane.
5 “close cut-in”
A vehicle in another lane brakes heavily in an attempt to cut across to an exit. This violates system boundaries for minimum separation and a collision would be inevitable without intervention.

Note: Except for scenario 3, the scenarios are adopted from (Van den Beukel & Van der Voort, 2017). In the original publication, scenario 1 was named “1a”, 2 was “2a”, 4 was “2b” and 5 was “3b”.

Figure 2 

Timing of soft and hard warnings for each scenarios.

Note: SW = Soft warning, HW = Hard warning, THW = Time headway, 1) Timing close cut-in is based on (Urhahne, 2016).

2.3. Variables and measures

The independent variables comprise Concept-type (illumination concept or base concept) and Scenario (shown with the video-clips). The dependent variables are Situation Awareness (SA) and Concept Acceptance. Measurement of SA covers cognitive capabilities with regard to driver’s perception of automation-mode and understanding of how a system reacts to different situations (Endsley, 1995; Jin & Kaber, 2009). Measurement of SA is therefore widely accepted in areas important to supervisory tasks (Salmon et al., 2009). In parallel to Endsley’s definition of SA (Endsley, 1995), we measured: (1) Operator’s perception of automation-mode (correct knowledge what mode the automation is in); (2) Understanding why the automation is in a particular mode; (3) Interpretation of driver’s role (i.e. request for attention or required intervention). Measurement of SA is based on freeze-probe technique (Salmon et al., 2009). Concept Acceptance was tested by asking respondents’ consent how suitable the signals were for the specific scenario, as well as how hazardous the situation was perceived.

2.4. Survey design

The survey is set-up as a between-subjects comparison of concept. Therefore the survey is distributed in two samples, each representing a Concept-type. The video-clips included the signal-behaviour of a concept. Questions after each video were intended to find out participant’s understanding of the signals. The survey was designed to show participants two times a set of four videos, see Table 2. Each set included scenarios 1, 2, 4 and 5 in random order. After each video within the first set, the survey either asked for respondents’ perception of automation-mode (Q1) or interpretation of their role to control the automation (Q3). Thereafter, the survey always asked for respondents’ understanding of automation-mode (Q2) and their judgement of the signals’ suitability (Q4) for the scenario at hand. The questions on perception of automation-mode and driver’s role were multiple choice. Participants’ understanding of automation-mode was asked with an open question. The combination of multiple choice and open question was done to allow sufficiently easy response for respondents and at the same time acquire sufficiently detailed answers for qualitative assessment of respondents’ understanding. The order of questions was mixed to avoid prejudice about the aspects to be evaluated.

Table 2

Survey design with scenarios (video-clips) and assigned questions.

Video-clips with scenario

1
Approaching
Complex road
2
Vehicle passing
4
Merge-out target-vehicle
5
Close cut-in
3
Approaching
combined
on-off-ramp
1
Approaching
Complex road
2
Vehicle passing
4
Merge-out target-vehicle
5
Close cut-in

------------------- Set 1 (Randomized order) ------------------- ------------------- Set 2 (Randomized order) -------------------
Q3; Q2; Q4 Q1; Q2; Q4 Q1; Q2; Q4 Q3; Q2; Q4 Q3; Q2; Q4 Q1; Q5 Q3; Q5 Q3; Q5 Q1; Q5

Questions:

Q1 (perception): What aspect of the automation changed shortly before the video stopped? (Multiple choice question)

Q2 (understanding): Was there with regard to the traffic situation a reason to give signals? (Open question)

Q3 (projection): What do you think the signals at the end of the video wanted to tell you? (Multiple choice question)

Q4: Consent with statement “The signals are suitable.” (5-points Likert scale)

Q5: How hazardous was the situation? (9-point Likert scale)

1) Concepts: The behaviour of the concepts (illumination-concept or base-concept) was included in the videos. The survey is distributed in two samples, each representing a Concept-type.

The second set showed again scenarios 1, 2, 4 and 5 in randomized order. Now, the survey asked for either respondents’ perception (Q1) or interpretation of the driver’s role (Q3), interchanging the aspects as asked with the first four videos. In addition, the survey asked how hazardously respondents perceived the situations in each video (Q5). Rehearsal of the videos avoided that answering all five questions with each video would rely too much on visual memory. However, to avoid that respondents become accustomed to the situations shown in the videos, the order in each set was randomized. Furthermore, the first and second set was separated with a scenario not shown before (i.e. scenario 3). When a participant finished watching a video, it could not be replayed. The survey could be interrupted and continued, but needed to be finished within 72 hours. On exceedance of this time-span data were excluded from analysis.

The survey was set-up and hosted with web-based software from Qualtrics (www.qualtrics.com) and distributed in the Netherlands, United Kingdom and Germany. Ethical approval of the survey was confirmed by the authors’ department board at University of Twente. To enable participation, respondents needed to confirm possession of a driving license and a minimum age of 20 years. Invitations included a link to the Qualtrics-website where the survey was posted. The invitation explained that participation is anonymous, at a voluntary basis and without remuneration. Invitations were sent by the authors’ employers support offices via e-mail and social media. In addition, invitations were sent within the authors’ professional and private networks. The authors’ direct group of colleagues was instructed not to take the questionnaire because they could be familiar with the concepts. Because participation was anonymous and other than the socio-demographic questions no detailed personal data were taken, exact numbers about participant’s occupation and education are not available.

2.5. Respondent’s instruction and task

Participants were instructed to consider themselves the responsible ‘driver’ of a vehicle that drives automatically. It was explained that automation was only possible within boundary conditions, like travelling on motorways, with low speeds and while a target-vehicle is recognised. Respondents were informed that situations could happen requiring extra attention or to retake control. Participants were explained that in the videos the automation will announce signals (i.e. stimuli provided by the concepts) to either ask for attention or urge for intervention. However, respondents were not familiarized with the signals beforehand. Table 3 lists the instruction which was spoken during the introduction video.

Table 3

Text of spoken instruction during the introduction video.


Thank you for participating. Imagine that you are the driver of a car that drives automatically. That means that the car keeps distance and keeps you on the lane. This automation requires a car in front to follow and is available at low speeds. Nonetheless, you remain responsible for safe driving. Situations could happen which are difficult for driving automatically, or situations in which the car is not able to drive automatically. Then you, as a responsible driver, should pay extra attention, or you should take over the driving task. To help you, the vehicle will give signals to indicate if your attention or intervention is needed. These signals are based on sound, illumination or both. With the next questions, I ask you to watch videos and I am curious what you think the signals mean to you. This will help to research how such vehicles should communicate with their drivers.

2.6. Respondents

Due to the survey’s between-subject set-up, data-sets from two survey-samples were created. Per question, the number of responses varied between 49 and 55 for sample one, and between 39 and 45 for sample two. This variation is due to not all questions being mandatory to answer and posed a mistake in the surveys’ set-up discovered after the questionnaires were already being distributed. In total 100 people answered the questionnaire. Table 4 shows that respondents’ average age was 44.2 and driving experience was in average 23.0 years. Vehicle-usage was dominated by commuting and business trips. Gender was comparably divided over both survey-samples with a minimum of 25% female respondents. Both survey-samples show the same distribution of respondent’s characteristics and therefore allow comparison. With respect to experience with automation, 82% of participants had experience with cruise control (equally divided over both surveys), None of the participants’ vehicles was equipped with adaptive cruise control, but seven participants in survey 1 and one participant in survey 2 had vehicles equipped with automated parallel parking.

Table 4

Respondents socio-demographic characteristics.

General characteristics Survey 1 Survey 2 Car usage Survey 1 Survey 2

N (min.-max.) 49–55 39–45 commuting 65% 58%
Average age (years) 43.8 44.6 business trips 32% 45%
Av. driving experience (years) 21.7 24.5 delivering goods 11% 13%
Gender (%) Male/Female 69/31 75/25 shopping 71% 69%
Car ownership Survey 1 Survey 2 visiting family & friends 92% 89%

Private ownership 84% 71% holidays 87% 75%
Company-owned 5% 13% something else 10% 9%
Undefined (e.g.parent or friend) 3% 7%
No ownership 8% 9%

3. Results

3.1. Perception of automation-mode

The question “What aspect of the automation changed shortly before the video stopped?” measured participant’s perception of automation-mode. The options were: The automation…: (a) stopped, (b) became unavailable, (c) did not change, (d) became available, (e) became active, or (f) I don’t know. During the two hazardous scenarios (1 and 2) the situation only required attention and option (c) was correct. At the end of the two critical scenarios (4 and 5) the automation stopped. Figure 3 shows that in all scenarios the illumination concept provides better (more correct answers) or equally good perception of automation-mode.

Figure 3 

Percentages with participants’ correct perception of automation-mode per scenario and per concept. Results that show a trend (p < 0.10) to identify differences in perception of automation-mode between the concepts are marked with: “(*)”.

Figure 4 visualizes the distribution of participants’ selected options per scenario and per concept. Within scenarios 2 “Vehicle passing” (Figure 4b) correct perception of automation-mode was higher with the illumination concept (60%) compared to the base concept (41%): χ²(1, N = 99) = 3.64, p-value = 0.056 (two-tailed). The results also show that correct perception was difficult in this scenario: With the base concept almost half of the participants (44%) did “not know” the automation-mode compared to 29% with the illumination concept. In scenario 2, the alert was induced by traffic behaviour aside (i.e. illegally overtaking). In scenario 1 “Complex road” the alert was induced by a change in road situation ahead. When comparing results between both situations, analysis of regression revealed that Scenario significantly predicted perception scores, β = 0.232, t(185) = 3.320, p < 0.005 (R² = 0.056, F(1, 185) = 11.02). Concept showed less power to predict perception scores within the hazardous scenarios, β = -0.133, t(185) = -1.861, p = 0.064 (R² = 0.018, F(1, 185) = 3.46).

Figure 4 

Perception of automation-mode per scenario and per concept. Correct perception (i.e. “automation did not change” in scenario 1 and 2; “automation stopped” or “automation became unavailable” in scenario 4 and 5) are outlined green.

Within the two critical scenarios (4: “Merging-out target-vehicle” and 5: “Close cut-in”) most respondents answered with either “automation stopped” or “automation became unavailable”. In scenario 4, a target-vehicle disappears without the availability of a new target-vehicle. Therefore, both answers are considered correct: It makes the system become unavailable and it subsequently stopped. Results show slightly better perception with the illumination-concept (82% correct perception), compared to the base-concept (72% correct perception), see Figure 4c. These numbers do not denote statistical differences.

Although the answers of option (a) “The automation stopped” and option (b) “The automation became unavailable seem identical, there is in scenario 5 (“Close cut-in”) a difference in meaning between both answers: Here automation stopped because the cut-in vehicle violated minimum required follow-distances. Based on assumed system-design, deceleration-rates of the ego-vehicle would not suffice to bring the vehicle to a safe stand-still. This is why automation stopped. The technical preconditions for automation, i.e. line and target-recognition, plus driving on a motorway, did not change. Therefore, the answer “automation became unavailable” does strictly speaking not apply. However, based on general interpretation of the words “unavailable” and “stopped”, it is defendable to consider both options correct. In the first case correct perception is 44% with the base-concept and 41% with the illumination-concept, see Figure 4d. In the second case, correct perception is with each concept identical in this scenario, i.e. 64%.

When comparing results between the two critical scenarios (i.e. scenario 4 “Merge-out” and 5 “Close cut-in”), analysis of regression revealed again that Scenario significantly predicted perception scores β = 0.136, t(185) = 2.044, p = 0.042 (R² = 0.022, F(1, 185) = 4.18) – but Concept did not reveal power to improve perception.

Overall, we see that correct perception of automation-mode is better with the illumination-concept (70% correct perception) compared to the base-concept (60% correct perception), see the bottom item in Figure 3. For this overall result both answers “automation stopped” and “automation became unavailable” are considered correct within the two critical scenarios. These numbers denote a trend that the illumination-concept improves correct perception of automation-mode (χ²(1, N = 374) = 3.82, p-value = 0.051, two-tailed). Moreover, when we consider in scenario 5 (“Close cut-in”) only the option “automation stopped” correct (because strictly speaking the other option does not apply), the concepts show over all scenarios statistically significant differences in favour of the illumination concept: χ²(1, N = 374) = 4.01, p-value = 0.046, two-tailed.

3.2. Understanding automation-mode and changes

Measurement of understanding was based on respondents’ answers to the question: “Was there with regard to the traffic situation a reason to give signals?” This was an open question and participants were asked to give their answer with a (short) explanation. The appendix summarizes the results and shows participants’ understanding why an alert was raised per scenario and per concept.

Results for understanding automation-mode in scenario 1 (“Complex road”) do not differ between concepts: 70% of respondents understood the likelihood that other traffic would enter the main road. Understanding of automation-mode within scenario 2 (“Vehicle passing”) does show differences between concepts. 53% Of respondents had correct understanding with the base-concept, compared to 72% with the illumination-concept (χ²(1, N = 88) = 3.215, p-value = 0.073 two-tailed).

Results during scenario 3 (“Approaching combined on/off-ramp) show that the illumination-concept caused misunderstanding. An alert was raised for entering a main road and only 21% of respondents understood this correctly with the illumination-concept, compared to 46% with the base-concept. A specific explanation why the illumination-concept scored worse is difficult to find. In general the situation was difficult to understand: While it was explained that the system would only work on motorways, the vehicle was in this scenario on an entrance-lane and respondents could have had doubts whether the entrance-lane was part of a motorway or not. The scores also reveal a general difficulty to distinguish whether the signal refers to a road-situation or to potentially dangerous behaviour of other road-users: Hence, a considerable part of respondents (almost 15%) gave in the category ‘something different’ reasons like: “vehicles coming too close”, or “car in blind spot”.

In scenario 4 (“Merge-out target-vehicle”) understanding was generally low and results show that the concepts did not influence understanding of mode-changes. Only 1/3 correctly understood that the target-vehicle left the lane and that there was no new target-vehicle. The road situation was the same as in scenario 3 and again confusion whether the exit-lane belonged to the motorway or not, could be an explanation for misunderstanding. Moreover, after the automation stopped, the uncontrolled ‘ego’-vehicle ran out of its lane. It is likely that it was not the absence of a target-vehicle, but the road-position and behaviour of the ‘ego’-vehicle that made respondents suspicious of the need for attention.

Although results for participants’ perception (see previous paragraph) within the close cut-in scenario (5) showed indifferent concept-scores, understanding of automation-changes show some results in favour of the illumination-concept. In this close cut-in scenario, not the cutting-in as such caused mode-changes, but the close distance caused violation of minimum required follow-distance. With the illumination-concept respondents showed a trend to distinguish better (40% correct understanding) that a minimum follow-distance is being erupted, compared to the base-concept (23% correct understanding): χ²(1, N = 95) = 3.514, p-value = 0.061 (two-tailed). However, if we broaden correct response to more generally considering that cutting in is dangerous and that this behaviour is often referred to in the answers, we don’t observe differences between the concepts (χ²(1, N = 95) = 0.375, p-value = 0.541, two-tailed).

To summarize: If changes were only induced by the behaviour of other road users (i.e., scenario 2 “Vehicle passing” and 5 “Close cut-in”), respondents showed better understanding of automation-mode with the illumination-concept. In both scenarios signals were raised because of hazards that originated from the driver’s side-views. Therefore, we consider these results a confirmation of the directional cue offered by the illumination-concept. If the alert was induced by the road situation ahead (i.e.; scenario 1 “Approaching complex road”), understanding was high but indifferent between concepts. If the situation was characterized by an interplay between road-situation and behaviour of other road users (i.e.; scenario 3 “Approaching an on/off ramp” and 4 “Merge-out target-vehicle”), understanding was low and we assume that confusion whether the alert (or warning) was induced by the road-situation or by the behaviour of other road users, tempered correct understanding.

3.3. Interpretation of driver’s role

Situation Awareness on level 3, i.e. projection of future state, requires correct interpretation of driver’s role. To measure SA-level 3 this survey therefore asked: “What do you think the signals at the end of the video wanted to tell you?” Possible answers were: “I should take over”; “I should watch traffic carefully” and “I don’t know”.

Table 5 shows only within scenario 3 “approaching on/off ramp” differences between concepts to help interpreting driver’s role. Then, the illumination-concept (78% correct interpretation) shows a trend for superior interpretation compared to the base-concept (62% correct interpretation): χ²(1, N = 93) = 2.91, p-value = 0.088 (two-tailed). In all other scenarios, scores between concepts are indifferent and show relatively high values of correct interpretation. Remarkably, the previous section showed that respondents had difficulties with understanding the situations in scenario 3 and 4. Although these difficulties in understanding were based on similar confusion due to the interplay between road-situation and behaviour of other road-users, a vast majority correctly recognised that their responsibility differed between both scenarios and correctly interpreted their role.

Table 5

Percentages with interpreted driver’s role within each scenario and per concept. Correct answers are indicated with italic and bold typed font.

Scenario Concept n Percentages selected answers of driver’s role

“I should take over” “I should watch traffic carefully” “I don’t know”

1: Approaching complex road Base 44 7% 77% 16%
Illumination 53 9% 72% 19%
2: Vehicle passing Base 49 4% 86% 10%
Illumination 39 5% 85% 10%
3: Approaching on/off ramp (*) Base 52 17% 62% 21%
Illumination 41 12% 78% 10%
4: Merge-out target-vehicle Base 45 87% 2% 11%
Illumination 54 85% 2% 13%
5: Critical cut-in Base 49 80% 14% 6%
Illumination 39 79% 15% 5%

Note: Percentages that denote statistically significant differences between concepts are marked “*” if p < 0.05. Differences that show a trend (p < 0.10) are marked with: “(*)”.

3.4. Acceptance: How suitable were signals for the situation?

Figure 5 shows results for acceptance. Respondents were asked after each video to rate their consent with the statement “The signal was suitable for the situation” on a Likert-scale ranging from 1 (“totally disagree”) to 5 (“totally agree”). Differences in concept-scores are in favour of the illumination-concept. Within two scenarios, the illumination-concept receives significant better acceptance. These two scenarios (2 and 3) are both ‘hazardous scenarios’, requiring supervisory control. Based on cumulative scores (taken all scenarios together), illumination (M = 3,57; SD = 0,99) is perceived more suitable than no illumination (i.e. the base-concept) (M = 3,12; SD = 1,14). The difference between both concepts is extremely significant: t(478) = 4,597, p-value < 0.001.

Figure 5 

Participants opinion of signals’ suitability for the situation the signals alerted or warned for, measured on a 5-point Likert scale ranging from 1 “totally disagree” to 5 “totally agree”. Values marked with “*” denote statistically significant difference (p < 0.05). Error bars show Standard Deviation.

3.5. Influence of illumination on perceived hazardousness

Figure 6 shows that the illumination-concept has influenced respondents’ opinion how hazardous they perceived the situations to be. Within scenario 1 “Approaching complex road”, respondents perceived the same scenario with the illumination-concept significantly more hazardous than with the base-concept: t(86) = 2.962, p-value = 0.004. The relatively low scores in perceived ‘hazardousness’ are in line with the designed low level of urgency for this scenario.

Figure 6 

Participants opinion of signals’ perceived hazardousness within scenarios 1, 2, 4 and 5. Values marked with “*” denote statistically significant differences (p < 0.05). Error bars show Standard Deviation.

4. Discussion

To answer to what extent directional illumination is able to support mode awareness and understanding of driver’s role, we measured three levels of situation awareness, i.e. perception of automation mode, understanding of automation mode and interpretation of driver’s role. To achieve more insight we also measured acceptance and perceived hazardousness.

Table 6 provides overall results for all assessment aspects. Situation Awareness was generally high within the changing road situation ahead (scenario 1), illustrated by high scores on all three SA-levels independent of concept. In situations influenced by the behaviour of other road users, i.e. scenarios 2 (“Vehicle passing”) and 5 (“Close cut-in”), the illumination-concept was most supportive: In scenario 2 the illumination-concept shows a strong trend to increase correct perception of automation-mode and both in scenario 2 and 5 a trend for better understanding of automation-mode. In contrast to scenario 1 where attention was needed ahead, scenarios 2 and 5 require attention to the driver’s side-views. We consider the observation that the illumination-based concept demonstrates superior scores in those scenarios requiring attention to the driver’s side-views a confirmation for the benefits of providing directional cues with illumination.

Table 6

Overall results for respondents mean scores for mode-awareness, acceptance and perceived hazardousness.

Percentages1 correct mode-awareness in reference to:

Scenarios Perception Understanding Interpretation Acceptance2 Hazardousness2

Concepts: base illumi. base illumi. base illumi. base illumi. base illumi.

1: Complex road 72 73 76 70 77 72 3.34 3.65 2.31 3.27
2: Vehicle passing 41 60 53 72 86 85 2.85 3.44 3.45 3.56
3: Approaching on/off ramp (not measured) 46 21 62 78 3.02 3.59 (not measured)
4: Merge-out target-vehicle 72 82 32 29 87 85 3.16 3.46 6.44 6.69
5: Close cut-in 64 64 23 40 80 79 3.35 3.71 7.59 7.92
overall 60 70 (not applicable) 78 80 3.12 3.57 (not measured)

Note: 1 Percentages range from 0 to 100%, values represent mean percentages per concept. If a concept scores significant (p < 0.05) better its value is typed bold and underlined. Trend differences (p < 0.10) are underlined. 2 Acceptance-scores range from 1 (not acceptable) to 5 (fully acceptable). 3 Perceived hazardousness range from 1 (not at all hazardousness) to 9 (extremely hazardousness).

4.1. Mode awareness and interpretation of driver’s role

Perception of elements important for situation awareness

Correct perception showed a maximum of about 70% independent of concept. The fact that respondents watched videos without neither the possibility nor the need to actively control the automation themselves, could be an explanation for this moderate perception of automation-mode. When perception was lowest (scenario 2), the illumination-concept demonstrated most advantages, i.e. a close to significant superiority over the base-concept for correct perception of automation-mode. Although these results confirm the advantages of the illumination-concept, with more diversity of signals (also from other systems) correct perception might in practice be lower than our measured maximum of 70% – values not considered very satisfying. Furthermore, concept-type demonstrated to have a moderate influence on driver’s ability to correctly perceive automation-mode and perception is influenced more by Scenario. Therefore, the benefits of directional illumination for correct perception of automation-mode need careful consideration.

Understanding of automation-mode

Interpretation of respondents’ written explanations of mode-understanding teaches us that the signals do (in general) not provide strong explanatory information why a signal is being raised. Furthermore, correct understanding differed strongly between scenarios. When signals are raised because of hazards originating from the driver’s side-views, the illumination-concept provides better understanding. When the need for attention is induced by an interplay between road-situation and behaviour of other road users, understanding is generally low – most likely due to confusion. We consider the similar scores between concepts an indication that directional illumination just not pays off in these combined situations, but neither makes comprehension worse. Further efforts are needed to discover appropriate solutions to provide explanatory information during events that include combinations of changing road-situations and behaviour of other road users.

Interpreting driver’s role

The written remarks for understanding show that respondents had difficulties to distinguish between an alert and an instruction. Nonetheless, percentages of correct interpretation of driver’s role are generally good and concept-type did not influence interpretation of driver’s role. One aim of this study was to identify if directional cues allowed drivers to deduce what reaction is required. With regard to this aspect, the results do not provide strong evidence that directional illumination raises correct interpretation of driver’s role compared to the base-concept. A possible explanation is that the instruction in combination with cues provided by the scenarios themselves, probably provided enough information to correctly interpret the required role, without need for concept-enhancement. Furthermore, the only scenario where the illumination-concept demonstrated significant better support to interpret driver’s role (i.e. scenario 3) was also the scenario with generally lowest scores. A probable explanation for this observation is: When a situation is more difficult there is also more potential for improvements (enabling the illumination-concept to show its benefits).

Relation between perception, understanding and interpretation

Correct perception of automation-mode and correct understanding does not keep pace with one another. In scenario 4 (“Merge-out of target vehicle”) correct perception is high: between 70 and 80%. Correct interpretation of driver’s role is more than 85% with both concepts, but correct understanding is low – approximately 1/3 – with both concepts. This result seems promising: When the relation between behaviour of another road user (merge-out) and system-configuration (required target) is difficult to comprehend, the concepts provided nonetheless support to correctly understand automation-mode and what is required from the driver. However, the illumination-concept does in this context not show additional support.

4.2. Concept acceptance, false alarms and perceived hazardousness

A concern is that illumination influenced ‘perceived hazardousness’, i.e. made situations perceived more urgent for possible intervention. Although the additional level of ‘hazardousness’ is small, it is consistently observed in all scenarios. Besides, in scenario 1 – a scenario with a generally low ‘hazardousness’ – this effect is significant. A small increase would not necessarily be disadvantageous, as it might increase driver’s carefulness. However, a larger influence raises two concerns. Firstly, if in practice soft-warnings are perceived as high urgency this would raise false warnings and cause misunderstanding. Secondly, urgency is in general related to annoyance (Baldwin & Lewis, 2014) and if perceived urgency is considerably higher than intended, this would lower acceptance. The potential influence of directional illumination on annoyance needs therefore investigation during further development of the concept.

Acceptance

When measured as a cumulative score over all scenarios and when measured in situations that involve a hazard within the driver’s side-views (i.e. scenario 2 and 3), the illumination-concept was perceived significant more suitable to raise an alert than the base-concept. Besides, acceptance was measured between-subjects. That is; participants had no ability themselves to compare relevant features (i.e. with or without directional illumination) within a specific scenario. Nonetheless, the acceptance-scores are higher for the illumination-concept and are therefore taken as strong evidence that users recognise and reward the potential benefits of directional feedback provided with the illumination-concept.

4.3. Limitations

When interpreting the results, a concern is that the survey was filled out by participants themselves using their own equipment to watch the videos without the ability to control screen sizes. For participants’ convenience the survey could be interrupted and continued (with time restrictions) at a later stage. Also, this study was based on watching short video-clips without a need for actually retaking control and neither the temptation to monitor the road less, something that would be the case during prolonged driving with partially automation.

Providing visual cues within the windscreen or side windows might be less effective when drivers have taken their eyes completely off the road. Other means to provide directional feedback – for example via the steering wheel (Johns, Mok, Talamonti, Sibi, & Ju, 2017), are not included in this study but need consideration in future research.

Another limitation is that older people and female are underrepresented in our study. The average age was 44.2 years and 28% of respondents were female. An older group of participants and more equal division of gender may have influenced results. Furthermore, the questionnaire has not been controlled to represent a cross profile in societal groups. Hence, a majority of respondents originated from the authors’ social and professional networks. This may have biased the results. Therefore, future research should include trust and acceptance while acknowledging societal differences.

Another comment concerns the absence of competing signals and immersion. In practice, more input from competing functions is likely to cause a larger variety of signals which might worsen correct perception. Therefore, care should be taken with interpreting the results for real-world circumstances. With regard to perception of automation-mode this study revealed that the influence of Scenario on mode awareness was larger than the influence of Concept. Probably, the contextual situation defined situation awareness more than can be compensated for through interface-design. In real world circumstances, the driving situations and events are likely to be more diverse and immersive and might therefore have an even larger influence on mode awareness. At the same time, in real world circumstances the need for support is larger as drivers are expected to monitor the road less when using partially automated driving. Furthermore, the high scores with regard to correct interpretation of driver’s role might be a trade-off of our particular option-based questionnaire, rather than the result of particular interface-features. Finally, a limitation is that mode awareness has not been compared with a traditional interface designed to provide mode-feedback. Consequently, further research is needed within circumstances closer to real-world situations and with additional concept comparison in order to make more reliable predictions on the ability of directional illumination to raise mode awareness and to improve drivers’ understanding of the control task.

5. Conclusions and recommendations

The aim of this study was to answer to what extent directional illumination is able to support driver’s mode awareness when driving partially automated. Respondents demonstrated with the illumination-concept generally higher levels of mode awareness than with the base-concept. Since these differences are consistently, but in limited occasions observed, while demonstrating moderate statistical evidence, we conclude that the results provide at most indications that directional illumination improves situation awareness. This conclusion is in line with previous studies, e.g. from Lee & Chan (2007) and Stanton et al. (2011). The support is especially noticeable in situations when supervision is induced by hazards in the driver’s side-views, like alerting behaviour from neighbouring vehicles (i.e. scenario 2 and 5). Then, understanding of automation-mode shows a trend to be better with directional illumination than without directional illumination. Furthermore, in scenario 2, perception is almost significantly better with directional illumination than without. Nonetheless, correct understanding was usually quite low (varying between 30% and maximally 70%). Especially in situations when the need for attention is induced by an interplay between road-situation and behaviour of other road-users respondents had difficulties to comprehend why a signal was raised.

Considering the relatively low scores for correct understanding of automation-mode, further development of interfaces that combine directional illumination with driver’s instruction is an important recommendation to further improve supervisory support. Also, an important recommendation is to compare in future research performance of the supervising task between different means for providing directional feedback. While visual cues within the windscreen or side windows might be less effective when drivers have taken their eyes completely off the road, support for anticipation of traffic changes via sound or tactile feedback may be more effective; hence alternative sensorial feedback needs consideration in future research (cf. Sarter, 2006; Stanton et al., 2011).

Another observation is that the illumination-concept seems to have influenced ‘perceived hazardousness’, i.e. made situations perceive more hazardous. Although small in our test, this could in practice be a potentially counter-productive effect (like a cause for misunderstanding) for which further research is needed. Nonetheless, the illumination-concept is perceived a significant more suitable means to raise an alert than the base-concept. Taking all findings together – and referring to our goal to pre-test the illumination-concept, we conclude that the illumination-concept is an eligible direction for further development of interface-solutions to support the driver’s supervisory role when operating partially automated vehicles.

Additional File

The additional file for this article can be found as follows:

Data Report Internet Survey

Data-overview Paper Where To Look While Driving. DOI: https://doi.org/10.5334/ijds.9.s1