/Amazon Is Pushing Facial Technology That a Study Says Could Be Biased

Amazon Is Pushing Facial Technology That a Study Says Could Be Biased


Over the final two years, Amazon has aggressively marketed its facial recognition expertise to police departments and federal companies as a service to assist regulation enforcement determine suspects extra rapidly. It has executed in order one other tech big, Microsoft, has known as on Congress to control the expertise, arguing that it’s too dangerous for corporations to supervise on their very own.

Now a new examine from researchers on the M.I.T. Media Lab has discovered that Amazon’s system, Rekognition, had way more issue in telling the gender of feminine faces and of darker-skinned faces in photographs than related providers from IBM and Microsoft. The outcomes increase questions on potential bias that might hamper Amazon’s drive to popularize the expertise.

In the examine, printed Thursday, Rekognition made no errors in recognizing the gender of lighter-skinned males. But it misclassified girls as males 19 p.c of the time, the researchers mentioned, and mistook darker-skinned girls for males 31 p.c of the time. Microsoft’s expertise mistook darker-skinned girls for males simply 1.5 p.c of the time.

A examine printed a 12 months in the past discovered related issues within the applications constructed by IBM, Microsoft and Megvii, a synthetic intelligence firm in China generally known as Face++. Those outcomes set off an outcry that was amplified when a co-author of the examine, Joy Buolamwini, posted YouTube movies exhibiting the expertise misclassifying well-known African-American girls, like Michelle Obama, as males.

With advancements in artificial intelligence, facial technologies — services that can be used to identify people in crowds, analyze their emotions, or detect their age and facial characteristics — are proliferating. Now, as companies begin to market these services more aggressively for uses like policing and vetting job candidates, they have emerged as a lightning rod in the debate about whether and how Congress should regulate powerful emerging technologies.

The new study, scheduled to be presented Monday at an artificial intelligence and ethics conference in Honolulu, is sure to inflame that argument.

Proponents see facial recognition as an important advance in helping law enforcement agencies catch criminals and find missing children. Some police departments, and the Federal Bureau of Investigation, have tested Amazon’s product.

But civil liberties experts warn that it can also be used to secretly identify people — potentially chilling Americans’ ability to speak freely or simply go about their business anonymously in public.

Over the last year, Amazon has come under intense scrutiny by federal lawmakers, the American Civil Liberties Union, shareholders, employees and academic researchers for marketing Rekognition to law enforcement agencies. That is partly because, unlike Microsoft, IBM and other tech giants, Amazon has been less willing to publicly discuss concerns.

Amazon, citing customer confidentiality, has also declined to answer questions from federal lawmakers about which government agencies are using Rekognition or how they are using it. The company’s responses have further troubled some federal lawmakers.

“Not only do I want to see them address our concerns with the sense of urgency it deserves,” said Representative Jimmy Gomez, a California Democrat who has been investigating Amazon’s facial recognition practices. “But I also want to know if law enforcement is using it in ways that violate civil liberties, and what — if any — protections Amazon has built into the technology to protect the rights of our constituents.”

In a letter last month to Mr. Gomez, Amazon said Rekognition customers must abide by Amazon’s policies, which require them to comply with civil rights and other laws. But the company said that for privacy reasons it did not audit customers, giving it little insight into how its product is being used.

The study published last year reported that Microsoft had a perfect score in identifying the gender of lighter-skinned men in a photo database, but that it misclassified darker-skinned women as men about one in five times. IBM and Face++ had an even higher error rate, each misclassifying the gender of darker-skinned women about one in three times.

Ms. Buolamwini said she had developed her methodology with the idea of harnessing public pressure, and market competition, to push companies to fix biases in their software that could pose serious risks to people.

The new study found that IBM, Microsoft and Face++ all improved their accuracy in identifying gender.

By contrast, the study reported, Amazon misclassified the gender of darker-skinned females 31 percent of the time, while Kairos had an error rate of 22.5 percent.

Melissa Doval, the chief executive of Kairos, said the company, inspired by Ms. Buolamwini’s work, released a more accurate algorithm in October.

Ms. Buolamwini said the results of her studies raised fundamental questions for society about whether facial technology should not be used in certain situations, such as job interviews, or in products, like drones or police body cameras.

Some federal lawmakers are voicing similar issues.

“Technology like Amazon’s Rekognition should be used if and only if it is imbued with American values like the right to privacy and equal protection,” said Senator Edward J. Markey, a Massachusetts Democrat who has been investigating Amazon’s facial recognition practices. “I do not think that standard is currently being met.”



Source link Nytimes.com

TAGS:
Original Source