Published on the 15/02/2024 | Written by Heather Wright

Woolworths, Coles, Palantir push on with digital surveillance…

“There’s lots of technology out there. The adoption and application of that technology is dependent and constrained in some senses by data protection regulations, but also by budgets and internal priorities.”

Nick Ingelbrecht, Gartner Australia-based senior director analyst, is talking retail technology on the back of the launch of a facial recognition trial by Foodstuffs North Island in New Zealand, and the signing of a three year deal between Coles and US data analysis company, Palantir.

“These things are a feature of life and really it is a question of having the data protection regulations in place.”

While very different deals, both have sparked publicity – and in the case of the facial recognition trial in particular, concerns.

In New Zealand, retailers are watching the Foodstuffs trial closely.

Twenty-five Pak’n’Save and New World supermarkets are trialing the technology, scanning and making biometric templates of shoppers as they enter premises. Those templates are then checked for matches against a ‘watchlist’ of people.

The Privacy Commissioner, who asked Foodstuffs North Island to show evidence facial recognition was a justified way to lower retail crime, is also watching closely, using his inquiry powers to keep a close eye on the trial which started last week.

Commissioner Michael Webster says he’s concerned that facial recognition technology (FRT) isn’t a proven tool in efforts to reduce harmful behaviour in supermarkets, especially violent harmful behaviour.

“We wouldn’t accept being fingerprinted and checked at the door before shopping for groceries – that sounds ludicrous – but FRT is a similar biometric process that is faster, machine-run, happens in a nanosecond and creates a template to compare your face to, now and in the future,” he says.

“We want people to be safe as they shop and work. But I have real questions about whether the technology will be effective in stopping violent behaviour or preventing harm.”

But Retail NZ is hoping the trial will be the impetus for wider rollouts with other retailers.

Its figures put the cost of retail crime to its members at ‘well over’ $2.6 billion annually and it has previously called for a sector-wide use of facial recognition technology to help fight those losses.

Carolyn Young, Retail NZ chief executive, says with members facing increasing rates of crime, putting their employees and public at risk, as well as ‘threatening the financial stability of retail business’, the outcomes of the trial will be of ‘enormous interest’.

“We look forward to learning from this trial to see if the technology is suitable to be rolled out across other retailers,” Young says.

Growing incidents of aggression in stores are also cited as a driver for facial recognition. The issue of aggression and violent behaviour in stores is something Ingelbrecht flags as an area where attention – and technology – will increasingly be focused.

He seems almost amused by the variations in sensitivities where facial recognition is concerned. He notes that while Europe is very hot on privacy, in the Middle East and North and South America ‘it is very much part of the furniture’.

In the US FaceFirst retail facial recognition systems are widely deployed, while in the UK Facewatch offers a similar subscription service for businesses, providing an alert when a someone believed to be a ‘known offender’ enters a store.

That service was challenged with a complaint to the UK data protection authority, the Information Commissioner’s Office. After some changes were made including reducing the personal data collected by focusing on repeat offenders or individuals committing significant offences and protecting those classed as vulnerable by ensuring they didn’t become a ‘subject of interest’, the complaint was rejected and the facial recognition system was deemed acceptable.

“Shoppers generally expect a level of surveillance in stores and that’s traditionally for detecting shop lifting, but CCTV in stores is just an accepted thing,” Ingelbrecht says.

“What we are talking about here is capturing biometric data, personally identifiable information and storing and processing it and then acting on it.”

“Because it is in law and enforcement it is a question of what you do with that information, how you treat that information that is going to determine what the response is going to be.

“These things are a feature of life and really it is a question of having the data protection regulations in place behind that to manage what people actually do with that data.”

The same applies, in a different sort of way, to Coles’ Palantir deal – which sees it join the CIA as a customer of Peter Thiel’s company.

Coles says its deploying Palantir’s tools across its 840 supermarket chain in the three year deal to cut costs and ‘redefine how we think about our workforce’ and ‘optimise’ that workforce, boosting productivity.

“If as a big grocery chain you can improve productivity by a few percentage points, I makes a huge difference to the bottom line at the end of the day,” Ingelbrecht says.

“Particularly with large retailers there are big productivity benefits from looking closely at their operations. And it might not just be staff, it could be the whole of the supply chain and also the shopper looking at transactions data and working out shopper preferences and so forth.

“So that big data approach is inevitable as well. What is interesting is when you get companies like Palantir coming in and doing big projects like that.”

Ingelbrecht says its the back end systems – what he dubs the plumbing – where there is greatest opportunity for cost saving and improving processes in retail.

Gartner has forecast that computer vision revenues for retail systems is growing by 13.6 percent over the next 10 years, with the company forecasting a global market worth US$12.8 billion in 2031. (That excludes retail surveillance revenues which are worth two or three times as much again, Ingelbrecht notes.)

It’s one of the fastest growing markets for computer vision solutions, particularly for supporting automation, shopper experience and retail execution applications.

“The technologies are evolving quite quickly to support that. The algorithms are improving, product recognition – face matching if you like – are improving in reliability,” Ingelbrecht says.

“Retail has been a big consumer of these tech for a long time. Managing queues better using computer vision systems to check shelves are properly stocked [reducing revenue losses through out of stock], things like managing car parking better, managing deliveries better, managing the new online and bricks and mortar shopping habits of customers so you can order online and pick up at store and variations on that.

A lot more impetus has come in the wake of Covid and the drive towards greater levels of automation through the use of tech and analytics.

Focus on traditional metrics like shopper footfall, demographics, heat maps, dwell times and optimising store layouts using that information has expanded to a broader customer experience imperative where technology is being used to make it easier for shoppers to purchase, while also protecting store staff and the shoppers themselves with the likes of slip and fall technology in shopping malls.

“The other aspect as well is the way that self-checkout tech is evolving. We can see a future where barcodes and QR codes will be replaced by vision systems that do product identification using smart checkouts,” he says.

Using mobile devices as shopping assistants, navigating stores, comparing prices and even ultimately the long-touted idea of replacing the checkout with your mobile, is also still on the cards, he says.

Scan and go contactless shopping, which enabled customers to scan items as they moved around a store and was used by 15 Woolworths and Countdown supermarkets in New Zealand, was dropped last month, allegedly over concerns that people weren’t paying for everything.

“The reliability of systems and the way that the architectures are evolving to support big data analysis, whether shopper behaviour or supply chain issues in store activities – all these things are up for grabs.

“If you just take area like loss prevention, there are lots of different things that you can track to detect fraud for example at the check out. The technologies have to be able to monitor or detect that automatically,” he says.

“Inevitably the envelope, where there is competitive advantage, where there is revenue opportunity or margin improvement op through the application of data analysis retailers and enterprises generally will try to take advantage of that.”