Published on the 22/10/2025 | Written by Heather Wright

Gartner outlines ‘golden path’ to AI value…

AI might be the hottest ticket in tech, but Gartner delivered a reality check at its IT Symposium last month, nothing that AI isn’t ready to deliver value – and humans are even less ready to capture it.

Daryl Plummer, Gartner VP, distinguished analyst and fellow, says while some organisations getting value, AI’s technical capabilities, costs and vendors are of ‘questionable readiness’.

“The last thing you want is to be the unintended owner of a negative ROI business case.”

He noted that while technical capabilities like search and summarisation are maturing, AI accuracy and agents still have a long road to travel.

Alicia Mullery, Gartner VP, analyst, notes genAI has an error rate of up to 25 percent.

Worse, research shows 84 percent of CIOs and IT leaders have no formal process to track AI accuracy.

Human review is the most common method, but AI can make mistakes faster than humans can catch them and can distort facts.

She recommended an ‘accuracy survival kit’ including comparison metrics testing the AI output against the established norm; two-factor error checking where one AI model checks another, and the ‘good enough ratio’ – a measure of when AI accuracy is ‘just good enough for your initiative’.

“And by the way, good enough might be higher than you think, because we tend to hold AI to higher standards than we hold ourselves.”

Less chat, more decisions

Plummer criticised the current focus on conversational agents – an area 86 percent of Australian IT leaders are focused on – arguing agents should be making decisions, not just chatting.

Retailers, for example, don’t need agents to just answer questions. They need systems that can monitor inventory, trigger RFPs for replenishment, negotiate contracts and select the best supplier – something Gartner terms multiplex B2B negotiation, and the area where real value lies, Plummer says.

“Don’t settle for conversations. Demand autonomous decision making,” he says, urging companies to also seek AI agents that can handle end-to-end processes.

“Agents as experts can create real-world value as long as you’re clear about what you need from the technology.”

The cost you didn’t budget for

Even when AI works, the financial picture is murky. Gartner says 65 percent of Australian and New Zealand organisations are ‘barely’ breaking even, or losing money, on AI investments.

The technology comes with high costs upfront costs and an ongoing ‘transition mortgage’ for training, change management and hidden ancillary costs like managing agent credentials, acquiring new datasets, maintaining multiple models and dealing with that accuracy survival kit.

The analyst firm says for every 100 days of implementation, AI training and literacy programs require an additional 25 days to train staff, with another 100-200 days – or up to 200 percent more effort – for change management.

“For every AI tool you buy, anticipate 10 hidden costs, conduct an analysis and decide which cost you will fund,” Mullery says. “The last thing you want is to be the unintended owner of a negative ROI business case.”

Married, with triplets, in a foreign country

If choosing an ERP vendor is like getting married, Plummer says choosing an AI vendor is like getting married, having triplets and moving to a different country.

When marrying an AI vendor, their models are like your children – needing to be fed the right data, constantly re-educated and grounded, and which take on your ethics, and the ethics of the vendor you marry.

The vendors themselves are starting to resemble countries, becoming ‘digital nation states’ controlling land, power, water, talent and capital to rival actual nations.

“If you’re planning a massive rollout of AI to your enterprise, bet on the major hyperscalers,” Mullery says. For industry specific use cases, look to startups and those who partner with industry leaders. And, if you want leading edge capabilities, you might consider betting on wild card vendors such as OpenAI, DeepSeek and Meta, but warns Plummer ‘they are fully innovation ready, but not fully enterprise ready’.

Whatever way you go, make sure the vendor is good with AI agents, the pair caution.

Plummer also urged digital tokenisation solutions to anonymise data so the real data doesn’t leave your shores, even inside a model.

That suggestion comes on the back of Gartner research which shows by 2027,35 percent of countries will be locked into region specific AI platforms using proprietary contextual data, leaving organisations in model lock-in – and at risk if they’re cut off.

“Build, buy, I don’t care, steal it if you have to,” he says of digital tokenisation solutions. “It’s a clever way to break model lock-in.”

Job losses and behavioural byproducts

Low human readiness is a major barrier, with 71 percent of CIOs reporting their people aren’t ready for AI.

Mullery describes a ‘toxic mix’ of a steep learning curve and the primal fear AI will replace us.

Despite headlines about an AI-driven jobs bloodbath, Gartner sees minimal impact for now – just one percent of headcount reductions currently are directly AI related. There will be job losses though: Through 2028 Gartner suggests 8,000 jobs will be lost, but AI will create other jobs.

More immediate is the potential for hiring restraint for junior level workers, as more senior staff us AI to do the work juniors used to do, such as building presentation decks.

While some companies are undergoing a ‘talent remix’ removing talent from lower performing areas and adding talent in higher potential areas, that’s not a strategy for all, Mullery says.

Instead, she urged companies to adopt a ‘financial remix’ strategy, looking at how AI can cut backlog, reduce fraud or grow revenue through human empathy. She cited the example of South African finance company SanLam, which is using AI to ‘empathise’ with customers and help them accept the idea of a debt reduction plan, rather than taking on a new loan.

“All of you, especially government, have a human empathy opportunity somewhere in your organisation. But almost none of us are thinking of it as our next big AI opportunity,” Mullery says.

Gartner encouraged organisations to invest in context engineering – providing AI with a frame of reference to improve outputs. It’s the next step beyond prompt engineering, helping avoid unwieldly mega-prompts.

“Context engineering takes all of the constraints, the directions and contextual cues out of the prompt and puts them into the system,” Plummer notes.

He also warned of skills atrophy, saying as AI takes over tasks, organisations must watch out it doesn’t also steal important skills like security and critical thinking.

“Institute periodic reviews or testing to make sure critical skills are not being eroded by AI.”

Another skill to consider: Smart contract managers. As AI begins autonomously negotiating supplier agreements based on real-time market conditions (that’s the multiplex B2B mentioned earlier), someone will be needed to oversee the ‘living contracts’ to avoid costly mistakes.

An extra capacity paradox

AI boosts IT capacity – and while that might be a good thing, it’s not always a good thing to show.

“You never want to look like you have too many people for the job,” Plummer notes. He urges IT leaders to ‘go on the offensive’ and work with business leaders now to identify new value enhancing IT work.

“And we don’t mean more IT projects.” Instead, expand into new markets, create products and services or add features that boost margins.

Gartner also forecast a ‘shockwave’ with revenue for services such as translation, basic legal and video or image editing forced to zero in the next three years.

“To find, capture and sustain value, you have to get both AI and humans ready,” Mullery concluded. “The golden path isn’t easy, but walking it might be one of the most rewarding times in your career.”