It's part of a larger effort to transform retailers' existing security cameras — those ubiquitous black gadgets that produce thousands of hours of mostly useless footage — into a more sophisticated artificial-intelligence surveillance system capable of automatically identifying people, license plates, and other information that can be used to alert store employees of emerging threats and eventually prosecute offenders.
"Right now, there's a big push to use AI," said Read Hayes, director of the Loss Prevention Research Council. "Retailers already have cameras all over the place."
The technology has gained new traction at a time when theft and violence are on the rise, particularly among organized crime groups that steal millions of dollars in merchandise and then sell it online. According to the National Retail Federation, organized retail crime has increased by 60% since 2015, with nearly 70% of retailers expecting an increase in 2021. According to estimates from the Retail Industry Leaders Association and the Buy Safe America Coalition, up to $69 billion in merchandise is stolen from the nation's retailers each year, or 1.5 percent of sales. In 2020, a total of 523 people were killed in robberies and other violent retail incidents in the United States, including 256 customers and 139 employees.
The federal government charged 29 people earlier this month with stealing $10 million in over-the-counter medicine and other items from Walmart, Costco, CVS, GNC, and other stores, then reselling them on sites like Amazon and eBay.
"What retailers are really trying to do is give police more evidence of who is doing this," said Adrian Beck, a University of Leicester professor whose research focuses on ways to combat retail loss.
Facial-recognition technology is contentious because studies have shown that it is frequently inaccurate when identifying people of color and women. According to one study, the worst technology has an error rate of up to 35% when scanning darker-skinned women but less than 1% when scanning lighter-skinned men. What's the reason? Early versions of the algorithms were trained on images of celebrities that were predominantly white and male.
Companies have been working to address the racial bias, and according to the National Institute of Standards and Technology, between 2014 and 2018, facial-recognition software improved 20 times when searching a database for a matching photograph. However, the government discovered in 2019 that some software still misidentified African-American and Asian people 10 to 100 times more frequently than white men.
"People have been wrongfully arrested as a result of this technology," said Jay Stanley, senior policy analyst for the American Civil Liberties Union's speech, privacy, and technology project. "It's not quite ready for prime time."
There has been a push to limit law enforcement's use of facial recognition technology, with cities such as San Francisco, Minneapolis, and Boston prohibiting officers from using it. Companies such as Amazon and Microsoft have stopped selling the technology to law enforcement.
Facial-recognition companies, on the other hand, actively market their technology to retailers, who use it to help gather evidence on repeat offenders before sharing it with local law enforcement.
"There has been a shift to the retailer doing some of this work themselves," said Tony Sheppard, director of loss prevention solutions at ThinkLP. He also mentioned that law enforcement resources can be limited at times.
FaceFirst, which claims to work with one-quarter of North America's largest retailers, compiles time-stamped incidents and a calculation of past losses into a package for law enforcement. "When you send this to the cops, it's a really strong storyboard," said FaceFirst president Dara Riordan at a recent industry conference.
Retailers must create their own watchlists, which is a manual process that involves an employee identifying a person on video footage following an incident and asking the software to alert them the next time the same person walks into one of their locations. It won't help if someone steals for the first time, but it can catch repeat offenders.
"Demand for our product is exploding," said Dan Merkle, CEO of FaceFirst, which now performs more than 12 trillion face comparisons for its customers per day, up from 100 million in 2017. It claims that its technology has a 99.7 percent accuracy rate, with no gender or race differences, and that it can reduce non-employee theft by 34% and in-store violence by 91%.
It assisted one retailer in apprehending "The Philly Fanatic," a thief known for wearing Philadelphia Phillies gear, not to be confused with the baseball team's mascot of the same name. He'd been going to the same store every one to two days with different accomplices, secretly filling a cabinet with $2,000 or so in goods, closing it with double-sided tape, and buying the cabinet normally. The goods were then handed over to a "fence," which was in charge of reselling them. According to FaceFirst, the software assisted in identifying the suspect and sending a real-time alert to the retailer, who apprehended him and turned him over to police.
Facewatch, based in the United Kingdom, reported that business has more than doubled during the pandemic, with convenience stores in particular showing strong demand. Its most extensive deployment is at a retailer with more than 100 locations.
According to Dean Nicholls, chief marketing officer at New York-based Oosto, retail is also a growing customer segment. Its technology is used by both regional and national retail chains, and some of them have deployed it in hundreds of stores. The company, which employs 150 people, has raised $350 million in funding.
When everyone suddenly started wearing masks at the start of the pandemic, the technology suffered a setback. Accuracy rates have been improving as companies train their algorithms on masked faces, but success rates still vary widely, ranging from 60% to 99.9%, according to a report published this month by the National Institute of Standards and Technology. Facewatch claims to have a 90% accuracy rate and has a human employee check each image before sending an alert to a retailer.
"By 2020, it was crippling for everyone," Merkle said. "We can now function at very high accuracies even while wearing a mask."
Retailers are also attempting to determine when to notify employees that an offender has entered the store and how to instruct them on how to act without putting them in danger. "What do you say to the poor minimum-wage checkout clerk?" Beck asked.
Few stores want to admit they use facial recognition because they are afraid of customer backlash. Rite Aid used the technology in hundreds of stores for nearly a decade before discontinuing it after a Reuters investigation revealed that it was mostly used in low-income, non-white neighborhoods. Others, such as Target and Home Depot, have previously tested facial recognition but now say they do not use it.
Other types of video technology are also being used by retailers. During the pandemic, for example, virtual security guards have grown in popularity. Guards are stationed remotely and can respond to a variety of alerts, such as motion detected by a security camera in the parking lot at 1 a.m., by speaking directly to the suspected intruder and alerting law enforcement. According to Sean Foley, an executive at Interface, which provides virtual guards to Zale's, Dollar General, Big Lots, and Gamestop, it's easier than hiring in-person guards during the current labor shortage and certainly cheaper, costing 8 percent to 12 percent of the cost.
Body cameras, similar to those worn by police officers, are also being used by retailers. Two-thirds of the retailers polled by the ECR Retail Loss Group were using the cameras in some capacity and reported a 45 percent decrease in the number of violent and verbal abuse incidents. According to Scott Thomas, national director of sales for signature brands at Genetec, which sells security system software, they can help cover blind spots left by security cameras.
Security cameras are also becoming more capable of recognizing objects, such as license plates, as well as human behavior. Oosto is developing technology that could one day identify weapons, such as a gun, or detect unusual motion, such as multiple people running towards a store, and immediately send an alert to the store to alert employees.
"It's like a radiologist telling you that a tumor might be cancerous," Hayes of the Loss Prevention Research Council explained. "AI does nothing but give you a heads-up."