In March, a self-driving Uber car hit and killed Elaine Herzberg in Arizona, marking the first known pedestrian killed by an autonomous vehicle. This, unfortunately wasn’t the first fatality related to self-driving cars; in 2016, Tesla’s Model S car was in autopilot mode when it collided with a large 18-wheel lorry, killing the Tesla’s “driver”.

The last few years has seen automakers and tech companies including Ford, General Motors, Tesla and Google’s Waymo accelerate their investments and efforts into research as they race to develop commercially viable self-driving cars. In November of last year, UK Chancellor Philip Hammond pledged that driverless cars will be on UK’s roads by 2021.

Whilst advocates claim self-driving vehicles promise a host of benefits, such as reduced traffic and increased safety over human-driven cars, potentially minimising road accidents by up to 90%; the latest fatality reignites further debate around the maturity, ethics and legal culpability of autonomous vehicles.

Despite the prospect of fewer accidents autonomous cars could bring about, difficult and complex ethical decisions still need to be made around the value of human life, fairness and morality.

Underpinning the development of self-driving technology is deciding how to programme cars to react in emergency situations through “crash optimisation” algorithms which essentially determine how to optimise an inevitable crash to cause the least harm or damage.

In this way, driverless cars bring to life the trolley problem, a series of philosophical thought experiments that inspects, tests and exposes some of our deepest ethical intuitions on how we value lives and moral responsibility. You can read more about the trolley problem here, but the classic problem is a choice between allowing five people to die by an oncoming trolley or pulling a lever to divert the trolley to another track killing one person. The problem exposes the tension between a utilitarian desire to save the greatest number of people versus actively doing harm and thus taking moral responsibility for killing one person.

Although the specific scenarios discussed in trolley problem debates may be farfetched, and will hopefully be a rarity in reality, such decisions and judgements around valuing some lives over others need to be programmed into the crash optimisation algorithms regardless. For any given potential crash, the risks to all actors (drivers, passengers and pedestrians) need to be evaluated and a course of action needs to be pre-programmed, requiring deliberation that human drivers are not afforded in an emergency.

Taking a utilitarian approach, assigning the same value to all people, we would be inclined to optimise any crash to save as many people as possible. This may sound instinctive to most, until you recognise that such a decision could very well be at the expense of the driver or passengers of the car they own (for example, a car swerving off a bridge rather than crashing into another car or a bus full of people).

According to a 2016 study published in Science, the majority of people who previously supported a utilitarian approach, then wanted to protect passengers at all costs when they were to imagine they occupied the car. This makes sense, since the owner of the car may reasonably expect their car to protect its own passengers above all other unknown vehicles. In addition, people might be reluctant to purchase a car that is programmed to sacrifice its owner. However, this intuition may only be applicable to privately owned vehicles, not holding for publicly owned vehicles such as public buses, for instance. It would also be interesting to consider how moral calculations would change with the shift to transportation-as-a-service options over traditional car ownership for new generations of would-be drivers.

It would be reasonable to hold that pedestrians should always be protected in all crash situations. Investigations into Uber’s fatal crash in March report that the self-driving software detected the pedestrian but did not react and stop in time, partly due to a deliberate delay that makes allowances for false positives, such as plastic bags or other roadside debris. As well as examining such allowances, this, coupled with a desire to protect pedestrians at all costs could prompt discussion for a greater separation in traffic infrastructure between pedestrians and drivers.

Ultimately there are no clear cut right answers to any of these questions, and companies may tackle these fundamental dilemmas differently, within the confines of the limited regulation legislated. These decisions ought to be transparent to consumers and the general public as the technology develops.

Whilst I believe algorithm transparency is vital, there are ways in which this could cause perverse behaviour en masse. If the crash response systems are well-known, this could lead to abuse and misuse, for example, other drivers may attempt to cheat it by cutting in front, knowing that the car will slow down or swerve to avoid an accident. More morbidly, someone could intentionally put themselves in the path of an autonomous vehicle to harm the passengers.

Further, consider the scenario where a car has to choose between hitting a motorcyclist wearing a helmet versus a motorcyclist without a helmet. If we hold that all cars were programmed to optimise car crashes on the basis of harm reduction, then the car may swerve towards the motorcyclist wearing the helmet, as statistically they are more likely to survive with lower fatalities. It would seem unfair to penalize helmet wearing motorcyclists, especially in countries where helmets are legally required. This could encourage motorcyclists to deliberately decide not to wear helmets if they knew that majority of cars are programmed to hit them when wearing their helmets.

Beyond these scenarios, there are other concerns that raise questions about the ethics of autonomous cars, from advertising potential (prioritise driving past certain shops over others) to security and privacy concerns to name but a few.

The latest casualty also raises the question about whether it is ethical to deploy this technology on public streets, unleashing it on to people without their explicit consent or knowledge. Informed participant consent is a hallmark of scientific or medical experiments and yet seems to be lacking when it comes to testing self-driving cars on public roads, putting people at potential risk.

Ultimately autonomous cars will have an array of implications on our societies, cities’ infrastructure and a number of industries, transforming our economic, political and cultural landscapes. This is an inevitability, just as the invention of cars had done so before, making provision for new forms of work, suburban living and drive-through restaurants for instance.

But concerns persist around autonomous vehicles, ranging from security to insurance and legislation, with perhaps the biggest in the wake of the death in Arizona is consumer and societal readiness, which may take longer to resolve than the technology to develop.

In 2016, Microsoft's artificial intelligence (AI) Twitter chatbot "Tay" caused controversy when it started using racist and sexist slurs and calling for genocide, forcing Microsoft to take the bot offline within 16 hours of its launch.

[examples of Tay's tweets]

The pertinent question, what was the root cause of this outburst? Whilst Tay’s Twitter outlash conjured fears of a dystopian future of killer robots, the bot, launched as a pilot to learn and mimic natural language, was merely reproducing the awful speech being tweeted at it. Paradoxically then, we can be somewhat glad that the bot’s unsupervised machine learning vulnerabilities were deliberately exploited by white supremacist users to reproduce their racist propaganda and not the creation of a “racist robot”.

However, this was not the only example of AI exhibiting such discriminatory behaviour. In 2015, Google’s Photos App, an image recognition service, categorised a picture of 2 black users as “gorillas”. Similarly, results of an algorithm-judged beauty contest sparked controversy as its 44 chosen winners (out of 6,000 contestants from 100 different countries) were overwhelmingly white.

We can hope that none of these algorithms were designed with racist and exclusionary intent, so what is going on here?

To break it down (rather simplistically), machine learning models reflect the quality and diversity of the data they are fed. In Google’s case, it could be assumed that the algorithms were not trained and tested on a diverse enough data set before its release, i.e. predominantly white faces were used to train the model to identify a “man” and/or a ”woman”. Whilst diverse data sets free of bias can sometimes be difficult to come by (e.g. homogeneous results if you google image search: “man”), I believe these examples are indicative of a wider diversity problem within the tech industry.

If teams of developers and product owners designing and building these technologies are predominantly of a specific race or gender, they may fail to consider other users and their differences. For example, if testing is conducted on individuals with similar looks, needs and preferences as the designer, the problems won’t become evident until it’s released to the wider public. Encouraging diversity in the creation and testing of these products becomes even more important.

This lack of diversity and consideration may explain why none of the Microsoft team working on Tay thought to include content filters for racially or sexually explicit language, let alone anticipated deliberate abuse of the bot launched on a platform with a well-known trolling community (following incidents such as #gamergate and increasing complaints of death and rape threats on the platform).

Where we’ve tried to side-step into a post-racial colour-blind future, these technologies highlight and amplify the biases still embedded within our societies. Whilst product and service design has always been riddled with biases (sometimes deliberately), the implications of these inaccuracies are becoming increasingly severe with the growing adoption of autonomous technologies, such as self -driving vehicles and within predictive policing.

Worryingly, for example, Propublica reports that the AI software used in the US to predict probability of recidivism was found to be twice as likely to falsely flag black defendants as high-risk of reoffending and white defendants were almost as likely to be falsely flagged as low-risk of reoffending. Not only is this a questionable use of AI, these inaccuracies have grave consequences for defendants of colour, affecting a person’s sentence length, and further criminalises black communities under the guise of objective decision making.

The notion that AI eliminates or is immune to human error and is value-free needs to be thoroughly interrogated, especially as many algorithms rely on codifying historical data, which can be flawed due to a long history of discriminatory policies and reflects historical biases and disparities. Robust systems should therefore be developed to counter these biases, taking historical facts and different viewpoints into account and adjusting accordingly.

  • Instagram - White Circle
  • White Twitter Icon
This site was designed with the
website builder. Create your website today.
Start Now