In 2013 senior Microsoft researcher Kate Crawford issued a warning about big data, saying that a certain ‘data fundamentalism’ – an unwarranted confidence in the objective ‘truths’ that large data sets can provide – was emerging, and that we needed to adopt a more nuanced view of big data and the algorithms we use to extract meaning from it.

“Data and data sets are not objective; they are creations of human design,” said Crawford at the time.

“We give numbers their voice, draw inferences from them, and define their meaning through our interpretations. Hidden biases in both the collection and analysis stages present considerable risks and are as important to the big-data equation as the numbers themselves.”

Simply put, when algorithms are used for decision making, there is always potential for faulty assumptions, flawed thinking and discrimination against individuals.

So, five years later, has Crawford’s warning been heeded?

From the Equifax scandal (145.4 million customers affected) to Uber’s 2016 hush money payoff (affecting 57 million users), the last few years seem to have been testament to the idea that if you give a corporate megalith enough data and they’ll eventually hang themselves with it. And all of it seemed to come to head with the Facebook and Cambridge Analytica scandal, which saw The Zuck chastened by both Congress and EU and offering a predictable vow to do better in the future.

But when it comes to the ethics of data, the tech giants have not changed their ways says James Arvanitakis, professor in cultural and social analysis at the Western Sydney University. Arvanitakis analysed the data ethics principles and initiatives that various global organisations have committed to in the last 18 months to find whether there had been a significant realignment between community expectations and corporate behaviour.

“What we found is concerning,” says Arvanitakis. “Some of the largest organisations have not demonstrably altered practices, instead signing up to ethics initiatives that are neither enforced nor enforceable.”

So far Microsoft, IBM, and Google, have released their own AI ethical principle documents, with Facebook and Amazon joining consortiums, such as Partnership on AI and the Information and Technology Industry Council. While on the surface, that may look like progress, don’t be fooled: Sure, those two consortiums have published statements containing ethical principles, however the principles are voluntary and have no reporting requirements, objective standards or oversight.

“For tech companies, there may be a trade-off between treating data ethically and how much money they can make from that data,” says Arvanitakis.

“There is also a lack of consensus between companies about what the correct ethical approach looks like. So, in order to protect the public, external guidance and oversight is needed.”

New research from the Harvard Business Review seems to confirm that line of thinking.

“It is by now abundantly clear that, left unchecked, AI algorithms embedded in digital and social technologies can encode societal biases, accelerate the spread of rumors and disinformation, amplify echo chambers of public opinion, hijack our attention and even impair our mental wellbeing,” say the authors of the study, entitled Why We Need to Audit Algorithms.

“Companies have long been required to issue audited financial statements for the benefit of financial markets and other stakeholders… Should we not subject societally impactful…algorithms to comparable scrutiny?”

So what are these researchers suggestion? Human oversight of big data crunching?

Yes, exactly that.

“Computer science and machine learning methods will be necessary, but likely not sufficient foundations for an algorithm auditing discipline. Strategic thinking, contextually informed professional judgment, communication and the scientific method are also required.

“The data related events of the last few years have confirmed we need a greater focus on data as a citizen right, not just a consumer right,” says Arvanitakis. “As such, we need greater focus on fairness, transparency and shared benefit – all areas which are currently being neglected by companies and government alike.”

But if this oversight is to be directed by government, there is surely more work to be done. While this year’s Australian Government Data Sharing and Release Legislation seems like a positive step in general terms, its primary focus is privacy and offers little in terms of governance or transparency.

New Zealand might be poised to do better, at least from a public sector point of view. A cross-government review of government use of algorithms is currently underway, aiming to ensure New Zealanders are ‘informed, and have confidence in how the government uses algorithms’.

“As agencies continue to develop new algorithms, it’s important to preserve appropriate human oversight and ensure that the views of key stakeholders, notably the people who will receive or participate in services, are given appropriate consideration,” reads the Algorithm Assessment Brief.

“In addition to connecting expertise across government, agencies could also benefit from a fresh perspective by looking beyond government for privacy, ethics and data expertise. This could be achieved by bringing together a group of independent experts that agencies could consult for advice and guidance.”

As the ramifications of big data extend further beyond the business world – and as data’s significance as an economic asset grows – perhaps it’s time to start considering this a societal issue, rather than a purely a business one.

…