When Facebook rolled out facial recognition tools in the European Union this year, it promoted the technology as a way to help people safeguard their online identities.
“Face recognition technology allows us to help protect you from a stranger using your photo to impersonate you,” Facebook told its users in Europe.
It was a risky move by the social network. Six years earlier, it had deactivated the technology in Europe after regulators there raised questions about its facial recognition consent system. Now, Facebook was reintroducing the service as part of an update of its user permission process in Europe.
Yet Facebook is taking a huge reputational risk in aggressively pushing the technology at a time when its data-mining practices are under heightened scrutiny in the United States and Europe. Already, more than a dozen privacy and consumer groups, and at least a few officials, argue that the company’s use of facial recognition has violated people’s privacy by not obtaining appropriate user consent.
The complaints add to the barrage of criticism facing the Silicon Valley giant over its handling of users’ personal details. Several American government agencies are currently investigating Facebook’s response to the harvesting of its users’ data by Cambridge Analytica, a political consulting firm.
Facebook’s push to spread facial recognition also puts the company at the center of a broader and intensifying debate about how the powerful technology should be handled. The technology can be used to remotely identify people by name without their knowledge or consent. While proponents view it as a high-tech tool to catch criminals, civil liberties experts warn it could enable a mass surveillance system.
Facial recognition works by scanning faces of unnamed people in photos or videos and then matching codes of their facial patterns to those in a database of named people. Facebook has said that users are in charge of that process, telling them: “You control face recognition.”
But critics said people cannot actually control the technology — because Facebook scans their faces in photos even when their facial recognition setting is turned off.
“Facebook tries to explain their practices in ways that make Facebook look like the good guy, that they are somehow protecting your privacy,” said Jennifer Lynch, a senior staff attorney with the Electronic Frontier Foundation, a digital rights group. “But it doesn’t get at the fact that they are scanning every photo.”
Rochelle Nadhiri, a Facebook spokeswoman, said its system analyzes faces in users’ photos to check whether they match with those who have their facial recognition setting turned on. If the system cannot find a match, she said, it does not identify the unknown face and immediately deletes the facial data.
At the heart of the issue is Facebook’s approach to user consent.
In the European Union, a tough new data protection law called the General Data Protection Regulation now requires companies to obtain explicit and “freely given” consent before collecting sensitive information like facial data. Some critics, including the former government official who originally proposed the new law, contend that Facebook tried to improperly influence user consent by promoting facial recognition as an identity protection tool.
“Facebook is somehow threatening me that, if I do not buy into face recognition, I will be in danger,” said Viviane Reding, the former justice commissioner of the European Commission who is now a member of the European Parliament. “It goes completely against the European law because it tries to manipulate consent.”
European regulators also have concerns about Facebook’s facial recognition practices. In Ireland, where Facebook’s international headquarters are, a spokeswoman for the Data Protection Commission said regulators “have put a number of specific queries to Facebook in respect of this technology.” Regulators were assessing Facebook’s responses, she said.
In the United States, Facebook is fighting a lawsuit brought by Illinois residents claiming the company’s face recognition practices violated a state privacy law. Damages in the case, certified as a class action in April, could amount to billions of dollars. In May, an appeals court granted Facebook’s request to delay the trial and review the class certification order.
Nikki Sokol, associate general counsel at Facebook, said in a statement, “This lawsuit is without merit and we will defend ourselves vigorously.”
Separately, privacy and consumer groups lodged a complaint with the Federal Trade Commission in April saying Facebook added facial recognition services, like the feature to help identify impersonators, without obtaining prior consent from people before turning it on. The groups argued that Facebook violated a 2011 consent decree that prohibits it from deceptive privacy practices.
“Facebook routinely makes misrepresentations to induce consumers to adopt wider and more pervasive uses of facial recognition technology,” the complaint said.
Ms. Nadhiri said Facebook had designed its consent process to comply with the new European law and had previewed its approach with European regulators. As to the privacy groups’ complaint, she said the social network had notified users about expanded facial recognition services.
“We provide clear information to people about how we use face recognition technology,” Ms. Nadhiri wrote in an email. The company’s recently updated privacy section, she added, “shows people how the setting works in simple language.”
Facebook is hardly the only tech giant to embrace facial recognition technology. Over the last few years, Amazon, Apple, Facebook, Google and Microsoft have filed facial recognition patent applications.
In May, civil liberties groups criticized Amazon for marketing facial technology, called Rekognition, to police departments. The company has said the technology has also been used to find lost children at amusement parks and other purposes. (The New York Times has also used Amazon’s technology, including for the recent royal wedding.)
Critics said Facebook took an early lead in consumer facial recognition services partly by turning on the technology as the default option for users. In 2010, it introduced a photo-labeling feature called Tag Suggestions that used face-matching software to suggest the names of people in users’ photos.
People could turn it off. But privacy experts said Facebook had neither obtained users’ opt-in consent for the technology nor explicitly informed them that the company could benefit from scanning their photos.
“When Tag Suggestions asks you ‘Is this Jill?’ you don’t think you are annotating faces to improve Facebook’s face recognition algorithm,” said Brian Brackeen, the chief executive of Kairos, a facial recognition company. “Even the premise is an unfair use of people’s time and labor.”
The huge trove of identified faces, he added, enabled Facebook to quickly develop one of the world’s most powerful commercial facial recognition engines. In 2014, Facebook researchers said they had trained face-matching software “on the largest facial dataset to date, an identity labeled dataset of four million facial images.”
Ms. Nadhiri said Facebook had consulted with privacy experts on its photo-tagging feature. It also recently notified users in the United States who had the site’s face-identification services turned on that they could turn them off, she said. “We have always respected people’s choices,” she said.
But Facebook may only be getting started with its facial recognition services. The social network has applied for various patents, many of them still under consideration, which show how it could use the technology to track its online users in the real world.
One patent application, published last November, described a system that could detect consumers within stores and match those shoppers’ faces with their social networking profiles. Then it could analyze the characteristics of their friends, and other details, using the information to determine a “trust level” for each shopper. Consumers deemed “trustworthy” could be eligible for special treatment, like automatic access to merchandise in locked display cases, the document said.
Another Facebook patent filing described how cameras near checkout counters could capture shoppers’ faces, match them with their social networking profiles and then send purchase confirmation messages to their phones.
In their F.T.C. complaint, privacy groups — led by the Electronic Privacy Information Center, a nonprofit research institution — said the patent filings showed how Facebook could make money from users’ faces. A previous EPIC complaint about Facebook helped precipitate a consent decree requiring the company to give users more control over their personal details.
“Facebook’s patent applications attest to the company’s primary commercial purposes in expanding its biometric data collection and the pervasive uses of facial recognition technology that it envisions for the near future,” the current complaint said.
Ms. Nadhiri said that Facebook often sought patents for technology it never put into effect and that patent filings were not an indication of the company’s plans.
But legal filings in the class-action suit hint at the technology’s importance to Facebook’s business.
The case was brought by Illinois consumers who said that Facebook collected and stored their facial data without their explicit, prior consent — in violation, they claim, of a state biometric privacy law.
If the suit were to move forward, Facebook’s lawyers argued in a recent court document, “the reputational and economic costs to Facebook will be irreparable.”
Follow Natasha Singer on Twitter: @natashanyt.
Doris Burke contributed research.
