Google’s head of ethical machine learning on the ways Google is tackling bias in AI

At a Google conference last year, the company’s then head of artificial intelligence (AI) said we don’t need to worry about killer robots, instead, we need to worry about bias.

“The real safety question, if you want to call it that, is that if we give these systems biased data, they will be biased,” said John Giannandrea.

This is something Google has struggled with in the past. There was one particular incident where an algorithm in Google Photos was mistakenly classifying black people as “gorillas”. As a result, everyone from IBM to Airbnb is trying to ensure that the algorithms they are creating are not incorporating similar biases.

For Jen Gennai, Google’s head of ethical machine learning, trust and safety, preventing bias is literally in the job description. In this role, Gennai leads a cross-company team that analyses and interrogates new products and services to ensure that all new Google products are adhering to this ethical standard.

“If we’re going to set up a new technology, we need to make sure that technology is right from the start,” she tells the Standard.

Here are the ways Google is tackling bias in AI.

In June this year, Google’s CEO Sundar Pichai published the company’s AI principles that guide Google’s work and research covering these areas, including ensuring that Googlers don’t create or reinforce unfair bias.

“How AI is developed and used will have a significant impact on society for many years to come. As a leader in AI, we feel a deep responsibility to get this right,” he wrote at the time.

The catalyst for this new stance on AI was affected by the instances where Google had got things very wrong. As well as the gorilla incident, work by the Harvard researcher Latanya Sweeney found instances of racial discrimination on Google Search. When you searched black-sounding names, arrest-related ads were more likely to show up than when you searched white-sounding names.

“Both of those [incidents] were triggers for us that it shouldn’t be our users seeing this, we need to stop them from happening in the first place,” explains Gennai.

Now, whenever a new product is launched at Google it undergoes rigorous testing by Gennai and her team. This brings together experts from areas such as tech, legal, privacy and social sciences to analyse each product against Google’s AI principles.

This testing focuses on the benefits and negatives of a product. “We’re not the scary police force in Google,” says Gennai. “We’re like, I want you to launch the best product you possibly can, so I’m going to identify all the holes that could possibly go wrong so you can fix them.

With the case of Duplex, which allows the Google Assistant to make calls and book appointments on your behalf, there were several ethical considerations taken before the tech was previewed at the I/O developer conference earlier this year.

In this instance, the team had done a lot of work on how the AI would deal with different accents but Gennai’s team raised some other issues.

“We looked at how our users would interact with it, how would [the tech] mitigate against bias. We asked, how would you know someone with a stutter or a speech impediment is received as well as someone who didn’t have one,” she explains.

“The team had done a lot of work, but we try and close all the gaps and raise it.”

One way to improve bias is to improve the datasets. Gennai uses the examples of weddings to illustrate this.

It used to be that if you searched weddings on Google Images, you would be served pictures of white people in wedding dresses because the algorithm was trained on Western-driven data sources. Yet this wouldn’t suit Google’s users in India or China, for instance.

Google asked for people to submit pictures of weddings around the world to improve the data set.

“Now, we’re getting a lot of input from our Indian users in particular where they are sending us so many colourful, great wedding pictures,” she explains.

“We want to understand how bias our existing sources are and create new sources that are a bit less biased to get it to a different position.”

Understanding how Google’s algorithms work in-house helps the team at Google to make changes for the better. There was the case of the Google Search CEO problem – when you typed CEO into the search bar, you would see images of white men in suits and Barbie, though not in the same image.

When Gennai’s team looked into the data, it wasn’t a human bias that caused it but a technological one.

“The algorithm had learned that navy blue pixels equals CEO, and because white men are more likely to wear navy blue suits, it was purely how the model had learned. But, [it shows] you need to scrutinise and interrogate your data,” she says.

It’s also important that Google users understand how the company’s algorithms work. As a Google user, you can now check why an ad was served to you and see the decision the algorithm made. You can also check why you saw a certain link on Google Search, depending on your previous search history. Gennai says this is about helping users understand Google more so they can trust the platform.

Transparency is part of the company’s AI principles too.

“Sharing a full algorithm with a user is not helpful at all – it will probably cause more confusion and feel like we’re deliberately obfuscating. [But] an explanation in readable language will help people understand what Google is trying to do, even if we get it wrong,” says Gennai.

At the end of the day, the problem with bias is that it comes from humans. “We’re all horribly biased,” says Gennai. “We all have our biases, all our existing data sources are built on everyone before us and their historical biases."

But, just because we can blame humans for everything wrong with algorithms, it doesn’t mean we can’t do anything to stop it. Instead, Google tries to investigate and account for the biggest sources of bias as possible, and try to change them.

That’s why the AI principles are so important inside Google and also why the company has created Responsible AI practices for people outside the company who are using its open source software to build AI and machine learning algorithms.

“We’re leading by example, we want to provide the tools and knowledge so people can follow it,” she adds.