Emotional driving in AI-powered cars

Master Thesis • UX Designer at Volvo Cars • Sweden 2019
My Role:

Research, Prototyping, Design, Writing

Company:
Year:

2019

Team:

1 Interaction Designer

Time:

6 month

The problem

Every year, individuals lose their lives in traffic accidents triggered by human errors. The automotive industry intends to create autonomous driving to fix this issue by getting rid of the greatest safety issue, the driver himself. But in this area, the industry faces major problems with law regulations and technological difficulties. In collaboration with Volvo cars and Volvo’s vision, that no one should be killed or seriously injured in a new Volvo by 2020. This dissertation investigates how human errors created by emotional driving or bad driving styles can be minimized with the help of artificial intelligence assistance instead of using fully autonomous driving vehicles. Along with Volvo Car’s design strategy including safety, intelligence, modern, and warmth, a concept has been created in which cars can detect human emotions.

The Solution

The outcome of the Thesis is an app connected to a car rental service. While negative emotions are hard to prevent, the service behind the app plays with Prevention, Education, and Motivation to be the test driver in traffic situations. The thesis resulted in an app that is linked to a car rental service. While negative emotions are difficult to avoid, the app's service focuses on prevention, education, and motivation of negative effects of emotions on driving, so that the driver gets the best possible support in driving and safety.

  • The Prevention feature, the application uses AI to reason driving areas that are high in the potential of stress and anger, and with planning, it plans the most convenient route around these areas.
  • The Education feature helps to educate in good driving behavior.
  • The motivation feature is linked to the maintenance and energy costs because good behavior means less yearly costs for the car and as result, the car lasts longer and can produce more money as a rental car. A part of this money gets funded back to the user as a reward.

Thesis Abstract

The outcome of the thesis includes an investigation into safety-critical emotions in traffic and a design concept that minimizes critical errors created by angry driving and bad driving style. The design concept in its final state, called Volvo Intuit, has three features such as "prevention" that takes care of planning a route to a destination in a trouble-free manner with the support of an AI and an emotional traffic heat map, "education" that includes the approach of educating users in good driving style, and "motivation" that rewards the user for good driving style.

As discovered from research, when negative emotions (such as anger or sadness) occur, they are difficult to eliminate and can result in an emotional increase if handled incorrectly. Consequently, I recommend that preventive and reactive strategies should be implemented if driving with emotions occurs in traffic to avoid errors and misbehavior.

Debrief after a drive.

The Design Process

The Design Process explains the author's journey from the problem he has discovered to the challenge of solving it. In the context of this challenge, the author used a the Design Thinking Design Framework and different Design Methodology for the different stages in Design Thinking. This helps to work user-centric and to ensure that the outcome solves the problem as well as possible.

Empathise

The Empathize Stage is about gaining empathy and understanding of the problem that should be solved.

Literature Review


The literature review is a critical essay that summarizes and evaluates the literature on a specific topic. Summarize, in terms of results or claims from prior research efforts on the topic.

The content of the investigated studies mainly focuses on the role of emotions and their negative effects on driving and safety. During the literature review, it became clear that research on positive emotions lacks empirical evidence and needs further research. Moreover, it becomes evidential from the literature review, that the most frequently studied emotions are: anger and sadness, and less considered were happiness and joy, surprise and fear.

Reviewed Topics :
Basic Emotions, different Emotion Models, How to track an emotion?, Body language and Biometrics, Artificial Intelligence, Sensor Fusion, Non - Visual User Interaction, Ubiquitous computing, Gamification, EU law for car safety 2022, Levels of Autonomous driving, Car Sharing, Data Privacy.

Qualitative Research


Qualitative research is primarily concerned with exploratory research and aims to gain an understanding of a topic.

I considered qualitative research because the subject of emotions and driving is too complex to be encapsulated by a simple yes or no hypothesis. Qualitative research techniques are richer and more insightful in the underlying reasons and patterns within phenomena. In order to conduct Qualitative research different Methods were used:

Expert Interviews


Interviews are used to gather qualitative data about a specific topic.

In total, I interviewed three different experts within the field. The first, Interview was a "Specialist in Data Analysis" at Volvo cars. He is focused on the development of Machine learning applications. The second interview was a "Psychologist, with a focus on behavioral therapy" The Third interview was a "Costumer Researcher at Volvo Cars", and focused on user research.

Questionnaire


Method Description: The questionnaire is a research technique to collect answers to a list of questions written about a specific topic.

The questionnaire was conducted with 57 participants from Sweden, Germany, Switzerland, and Australia in an early stage of the project. A questionnaire is a greater method because of it a sure-fire success. When the questionnaire is posted or sent to individuals, I can simultaneously take other significant measures in the study stage.

Data gathering


The findings of the Empathising phase got analyzed and structured into an Affinity Diagram for a better overview.

In the case of anger, it got investigated that anger is a universal facial expression. This means that anger has high generalizability across cultures. Another investigation is that, according to the research, anger and stress have a connection, which is a valuable insight for further steps in this project.

The User


The investigated user belongs to the age group, described as Millennials (also known as Generation Y and born between 1980 and 2000), who grew up with electronics, the Internet and online social communities, spending 18 hours a week on their smartphone and likely to be college graduates.
A user is a person who moved to a bigger city where his job is located. The user lives in areas where space is costly. Rents are high and it’s hard to get parking space. Cars in big cities mean commitment, because of parking place have to be rented out and insurances have to be paid. The user is more likely to avoid such commitments.

The User Needs


The investigated user needs are defined in 9 points, the most important needs are:
• A user needs a seamless and trouble-free journey to his destination.
• A user needs intelligence that minimizes the user’s cognitive workload.
• A user needs minimal interaction.
• A user needs support to minimize driving errors.

In-Depth Interview


In-depth interviews are conducted with a target group (4.2.1 The User) of people within the topic. The in In-depth interviews helps to get a deeper understanding of the target group perspectives, attitude, problems, needs, ideas, or environment.

I expect from the In-depth Interview to investigate which findings from Data gathering and Affinity Diagram created by literature review, Questionnaire, and the Expert Interview are coherent with the user, user’s activities and the user needs.
In order not to waste valuable time and resources of people, a pilot study of the In-Depth Interview took place. Piloting an in-depth interview is an integral aspect and useful in the process of conducting qualitative research as it highlights the major study’s improvisation.

Define

In the Define stage the problem Statement is defined. The problem statement is defined by the findings from the empathy stage that are collected, analyzed and sorted into a well - defined problem statement, which in return helps to create possible solutions in the ideation stage to solve the problem statement.

The Problem Statement


The Problem Statement is a method which is following a User-centered approach to frame a problem to a specific user. Furthermore, it includes ways to formulate (Point of View) and creating a direction to solve the problem (How might we ...?.[59]

Point of View


Point of View (POV) is a method used to define the design challenge to inform a problem statement.

Examples of defined POVs:
• A user needs to avoid stress while driving because stress and driving can easily result in anger.
• A user needs information about risks which can happen while driving angry because users want to realize something is necessary in order to save their life’s.
• A user need reflection on own driving behavior because people who reflecting on there own driving behavior creating awareness of own errors while driving.
• A user need to avoid anger while driving because angry drivers make other drivers angry and so on.

How might we ...?


"How might we" is a follow-up method used as a base to generate questions, which leads to directions of ideas or solutions.

Examples of How might we ...? - Questions:
• How might we design for avoiding Anger and stress?
• How might we design for educating/reflecting on anger and stress behavior in traffic?
• How might we design for motivating to behave in traffic to stay safe?
• How might we design for not making other drivers angry?

Ideation

The ideation stage is based on the problem statement and the understanding of the users and their needs, ideas can be generated to identify new solutions to the problem statement.

Brainstorming


Brainstorming is a design method, that fits perfectly into an ideation session.

The Brainstorming session focused on distinct alternatives, but primarily on a technology that operates seamlessly with vehicles. This implies setting up or preparing functionality in a smartphone app before approaching the car. Some of the ideas can be seen in 4.6.2 Brainstorming in the thesis.

Weighted Matrix


The method weighted matrix compares ideas with criteria that need to be considered when making a decision. A matrix-rating-system can be used to compare ideas and criteria.

In order to do a Weighted Matrix, criteria got formulated from the discovered User Needs and How might we ...? questions. Next, the criteria got ordered from important to less important. Important criteria were rated by 3 points, less important was rated by 1 point and everything in between was rated by 2 points. Finally, the ideas from the Brainstorming have been compared with the criteria and the 4 ideas with the most points will be used for further development. The Weighted Matrix Table can be found in the appendix of the thesis.

Minimum Viable Concept


Minimum Viable Concept also known as Minimum Viable Product or MVP follows the same idea but focuses on a concept rather than a product. The idea is to reduce a concept to the necessary features. This enables prototypes to be created and tests to be carried out as quickly as possible. Quick prototypes and testing generate knowledge for iterative improvements to the concept.

The MVC focuses on shared or subscription cars, which according to the current trends, it seems the future car trend. The concept is reduced to 4 features: Visualize Emotion and AI Route Planner get fused together to the feature Preventing. Proactive ADAS System gets defined into the feature Evasion. Emotional Diary gets split up into the features of Educating/Reflecting and Motivating.

The following table describes the MVC Features of the concept in its functionality:

  1. Preventing means the user can plan a route in advance to get according to live
    traffic data and stress/angry heat map the options to choose the most trouble-free journey. In addition, the driver will have a live route overview about this he will face on the route in order to be prepared for the unexpected.
  2. Educating/Reflecting means the driver is getting textual feedback while driving as suggestions for miss-behavior or suggested solutions to stressed/ angry situation. Furthermore, the driver is getting a debrief after every driving station in order to reflect on what he is doing great and where he could improve.
  3. Evasion means to have a support system in case the driver gets angry. This will be a proactive ADAS system which will support the driver while being angry to not misbehave in traffic.
  4. Motivating means the user stays motivated to behave in traffic because the hours he is behaving is reducing the amount he has to pay for the rental cost. This not only saves the user money but this safe also money for the lessor(/renter) because a good treated car cost less in maintenance over the year.

Prototyping & Testing

Prototyping is an experimental stage, the aim is to identify the best possible solution to the defined problem. A number of low-cost, scaled-down versions of the solution are produced and tested to explore problem solutions. Finally, the constraints of the solution and the problems that occur will be better understood. The Test stage should identify the best solutions or prototype in order to iterate or implement. It is very often in these steps that design fails and new insights lead to a redesign that needs to be re-tested.

Wire-framing


Wire-framing is a low-fidelity approach which is communicating a design but without coding. Wire-frames are limited in content and limited in the use of colors or other visual design elements. The purpose of a wire-frame is to create a user flow and finally a prototype.

Formative Usability Evaluation


Formative Usability Evaluation supports identifying which part of the concept works and does not work and why. It is also frequently used in a design process to support rapid iteration to improve the concept. The method is used to build up tasks to be completed by the users, to ask them to think aloud and monitor the user behavior, to find out when the user is struggling.

The first version of the idea was born with participants who were found in my instant environment. The participants were people from Volvo car UX Design Studio. The testing was organized into a task for each function. The participants had to thing out loud and the problems that occur during the testing were discussed instantly.

Summative Usability Evaluation


Summative Usability Evaluation is an evaluation method that is performed when a design is completed. This method can be performed as a Formative Usability Evaluation that the user has to perform tasks by using the design. Unlike Formative evaluation, this method is unassisted to simulate real-life use.

A scenario was created and tested with 5 participants as recommended by Jakob Nielsen in
"Why You Only Need to Test with 5 Users" for the Summative Usability Evaluation.
The test persons were 3 male and 2 female testers between the ages of 25-31 years old. The testing tasks were divided into Prevention, Education, Evasion, Motivation. Each task consists of sub-tasks. The test results which are necessary for the last iteration are stated in Summative Usability Evaluation Results in the thesis.

Results and Failures


In conclusion, it is quite clear that the feature education has potential but in its current state, it needs a new way of how it educates people. Furthermore, Evasion failed completely. This is quite interesting because from the In-Depth Interview result it got discovered that people will accept taking power away when they see that they go saved for this reason. This could have several reasons, first, my design was not good enough or because of the Limitation of Research.

The Final Concept

This design took the results from Summative Usability Evaluation into account in order to create the final design of my design solution called Volvo Intuit.

Volvo Intuit


This section describes the Volvo intuit design concept in its final state. Volvo intuit is about avoiding stress and anger in traffic. Anger and stress affect our judgment and fine motor skills which can lead to dangerous traffic situations.

The Volvo Intuit Feature: Prevention


To avoid means, prevention of stress and anger with trouble-free driving. For this reason, the Volvo intuit application uses AI to reason driving areas that are high in the potential of stress and anger and with planing it plans the most convenient route around these areas.
This is possible with inside the car collected data about the driver by an IR camera that collects data from Bio-metrics. These cameras will be mandatory by the new EU law regulations of car safety in 2022. This makes this concept feasible by every produced car in 2022.
In order to make a statement about the current driver traffic situation, a process called sensor fusion is necessary where data from outside and inside the car gets fused together, to generate a data set which is comparable to other data-sets. This allows an AI to classify a situation based on situations already learned.

The Volvo Intuit Feature: Education


Furthermore, users often getting stressed or angry in traffic because of bad driving habits, poor understanding of other road users, and poor respect of traffic regulations. The education part of Volvo intuit helps to educate in good driving behavior. This helps to raise the amount of educated and experienced drivers in traffic. The education system of Volvo Intuit recognizes if a driver is behaving poorly and helps him to reflect on this behavior in order to avoid it in future traffic situations. Poorly behavior can be for example:

• Rapid acceleration
• Not Using Turn Signals
• Tailgating [9]
• Speeding [9]
• Using a phone while driving
• ...

If poor behavior gets recognized by Volvo intuit the driver gets warned with a signal tone and a warning pops up in the infotainment system of the car. Later on the behavior and be seen as a log in the application. The poor behavior gets listed with where it happened with a description of why it is poor behavior and how to improve. In this way, both the user and the renter can profit because good behavior in traffic achieves better results in reducing maintenance and energy costs ( ref: chevinfleet.com) for the renter and more safety for the user.

The Volvo Intuit Feature: Motivation


In order to keep the user tight to the system which makes it possible to educate the user over time for good behavior, a motivation system got implemented as one of the last features. The motivation system is linked to the maintenance and energy costs because good behavior means less yearly costs for the car and as a conclusion, the car rests longer and can produce more money as a rental car. A part of this money gets funded back to the user as a reward. This means a driver get a cost reduction of every ride according to is level of good behavior in traffic.
The rewarding algorithm works like a taximeter as long as the drivers are not doing things like stated in the list of Poorly behavior, the driver is getting a cost reduction for every minute he is driving. This reduction can be higher and lower according to the model of the car which is used and the type of energy which is consumed (electricity or fuel). If the driver misbehaves the cost reduction is temporarily suspended for a few minutes until the misbehavior ends. This rewarding system should motivate and gamify good driving behavior for more driver and traffic safety. Gamification has proved to be very successful in engaging people and motivating them to change behaviors, develop skills, or solve problems. That’s because of the human body releases endorphins during a gamified experience that improves memory and learning abilities. This makes the education of a driver combined with gamification to a perfect combination to improve the driving experience and good driving style.