Kiwi researchers have produced an international report designed to help policy makers globally evaluate developing technologies such as AI and maximise the opportunities, while minimising risks.

The report, which provides a framework for exploring the potential of AI and its derivatives through lenses encompassing technological, as well as human and societal wellbeing, economics, politics, the environment and security, was written for the International Science Council (ISC) and launched in Paris last week.

“The conversation needs to go beyond the simplistic narrative of will it create a nirvana or destroy the world.”

It was authored by former prime ministerial chief science advisor Sir Peter Gluckman, now director of think tank Koi Tū: The Centre for Informed Futures and president of the ISC, and Hema Sridhar, Koi Tū strategic advisor for technological futures.

They say it’s critical that rapidly developing technologies are subject to ‘broad and multi-dimensional evaluation’ to maximise benefits and minimise ‘the very real risks’.

“The conversation needs to go beyond the simplistic narrative of will it create a nirvana or destroy the world,” Gluckman says.

“The reality is in the history of humankind, all technologies get used. They always get used both for good purposes and bad purposes. But having this sort of framework allows us to have the discussions about how to take any new technology and make it most likely that the good and beneficial purposes will be supported and the negative will be prevented.”

The framework presented in the report – and available as an accompanying editable Excel sheet – is designed to inform impact assessments and horizon scanning, facilitate stocktakes of existing and evolving measures, and bridge the gap between high-level principles and assessment for regulatory or governance purposes.

It includes an extensive list of ‘dimensions’ that might need to be considered when evaluating a new technology, from user competency and risks to human rights and wellbeing to privacy, distorted realities, business models, technological sovereignty and digital colonisation and geopolitical competition. Data and input is also covered extensively along with the impact of models including transparency and computational limitation.

The report, A Guide for Policy-makers: Evaluating Rapidly Developing Technologies Including AI, Large Language Models and Beyond, says while the use of AI has been heavily discussed there remains an ontological gap between the development of high-level principles and their incorporation into practice in regulatory, policy, governance or stewardship approaches.

“The purpose of the framework is to provide a tool to inform all stakeholders – including governments, trade negotiators, regulators, civil society and industry – of the evolution of these technologies to help them frame how they might consider the implications, positive or negative, of the technology itself, and more specifically its particular application.”

Gluckman says for pervasive technologies such as AI, assessing the risks and benefits requires broader perspectives than just those from industry, and these questions cannot be just looked at generically but need to be looked at in specific context.

He says a systemic framework for analysis is needed but has been missing.

While the report is framed around AI, Gluckman says it can be applied with minor amendments to any rapidly moving technology.

“The guidelines provide a provisional checklist of all the dimensions, both positive and negative, that merit active consideration.”

Issues are grouped into categories including  technological such as system characteristics, design and use; wellbeing of individuals and society; trade and economy, environmental, geo-strategic and geopolitical.

He says the checklist is much like a pilot’s checklist: Even if the pilot thinks there is nothing to ask they are still required to check. In much the same way, he says all of the checklist items should be considered, even if some may not be relevant in a particular context.

“This ensures that any claim or application is tested for both its value and risk and appropriate choices made.”

The preliminary analytical framework was released for feedback late last year, with feedback incorporated into the newly released framework.