Amazon’s Face Recognition Tool Confused 28 Lawmakers With Arrestees

The ACLU compared members of Congress with 25,000 mug shots using the same software Amazon is pitching to law enforcement.

Amazon’s facial recognition tool incorrectly matched the faces of 28 lawmakers with people in mug shots and disproportionately misidentified people of color in a test by the ACLU.

The civil rights organization, which has called on Amazon to exit the facial recognition industry, compared images of members of Congress with a database of 25,000 mug shots. Of the 28 misidentified lawmakers, 39 percent were people of color, including Reps. John Lewis (D-Ga.), Lacy Clay (D-Mo.) and Luis Gutiérrez (D-Ill.).

Customers can upload images and video to the publicly available Rekognition tool for analysis of people’s faces, objects and text. The ACLU said it used the tool’s default settings and asked an independent expert to verify the results and methodology.

The group cautioned law enforcement against using Amazon Rekognition, citing the possibility of bias.

“It’s not hard to imagine a police officer getting a ‘match’ indicating that a person has a previous concealed-weapon arrest, biasing the officer before an encounter even begins,” the report says. “People of color are already disproportionately harmed by police practices, and it’s easy to see how Rekognition could exacerbate that.”

Given the threats that facial recognition can pose to protesters, immigrants and minorities, the ACLU called on Congress to enact a moratorium on law enforcement using the technology.

In response to the report, Lewis and Rep. Jimmy Gomez (D-Calif.) wrote a letter on Thursday requesting to meet with Amazon CEO Jeff Bezos.

“We are alarmed about the deleterious impact this tool — if left unchecked without proper, consistent, and rigorous calibration — will have on communities of color; immigrants; protestors peaceably assembling and others petitioning the Government for a redress of grievances; or any other marginalized group,” they wrote.

Amazon defended Rekognition by saying that the tool has been used to prevent human trafficking, find missing children and prevent package theft. The company also said that the ACLU’s test results could have been improved by using a high confidence threshold ― the probability that a given prediction is correct.

“When using facial recognition for law enforcement activities, we guide customers to set a higher threshold of at least 95% or higher,” Amazon said.

However, the ACLU said Amazon’s default setting was an 80 percent threshold for face-based user verification. It also disputed that setting a higher threshold would change anything.

“Amazon seems to have missed, or refuses to acknowledge, the broader point: facial recognition technology in the hands of government is primed for abuse and raises significant civil rights concerns,” ACLU attorney Jacob Snow said in a statement to HuffPost. “Changing the threshold from 80 to 95 percent doesn’t change that. In fact, it could exacerbate it.”

The revelation of biases in facial recognition technologies is not new. A February study by researchers from the Massachusetts Institute of Technology and Stanford University found that the technology had higher error rates in analyzing darker skinned people and women.

In May, the Congressional Black Caucus wrote a letter to Bezos expressing concern about the company selling Rekognition to law enforcement agencies. The caucus said that communities of color are more aggressively policed and it fears that implementing Rekognition without further research could exacerbate problematic policing practices.

“We are worried deployment of technology like the one you have developed has a high propensity for misuse,” the letter says. “Surveillance of perfectly legitimate and constitutionally protected activity will only further erode the public’s trust in law enforcement.”

Bezos and Amazon did not respond to the letter.

Amazon has already pitched the facial recognition service to local law enforcement, including Oregon and Orlando. Orlando ended a pilot program with Rekognition in June after several civil rights groups spoke out against its use. The police department told that it wanted to uphold privacy laws and protect people’s rights.

Correction: This story previously said Rep. Lacy Clay was a congressman for Mississippi. Clay represents Missouri.

This article has been updated with the ACLU’s response to HuffPost and Gomez and Lewis’ letter to Bezos.

Popular in the Community