Published on the 05/12/2024 | Written by Heather Wright

AI hallucinations, carbon burn and distilling LLMs…

AWS is taking aim at one of generative AI’s more pernicious issues, with a new offering which aims to wipe out AI hallucinations.

The unveiling of AWS Automated Reasoning, which aims to cut down on hallucinations through checking responses against customer data and ‘preventing factual errors’ from hallucinations, came during AWS Re:invent, the annual AWS fest, currently underway in Las Vegas.

“Distilled models are up to 500 percent faster and up to 75 percent less expensive than original models.”

AWS says Automated Reasoning, which will be available as a new safeguard in Amazon Bedrock Guardrails, will use ‘sound mathematical, logic-based algorithmic verification and reasoning processes to verify the information generated by a model, so outputs align with known facts and aren’t based on fabricated or inconsistent data’.

In English, it essentially will cross-check a model’s response against information provided by the customer and if the answer doesn’t match up, it will be sent back to the model for checking.

The company claims it’s ‘the first and only generative AI safeguard that helps prevent factual errors due to model hallucinations’.

One analyst recently told iStart however, that hallucinations might be headline garnering, but it’s AI drift that companies need to be watching out for.

Gartner analyst Luke Ellery warned that drift, where performance degrades, is a less commonly discussed issue, but one which is a bigger concern from a business perspective.

There were plenty of other announcements for Bedrock, AWS’ managed service for building and scaling genAI applications.

Taking aim at another big bugbear with AI – cost – the company also unveiled, in preview, Amazon Bedrock Model Distillation, which aims to reduce the size and cost of AI models.

The offering will enable customers to use larger ‘teacher’ AI models with their greater ‘knowledge’ and whose accuracy they want to achieve for their use case, to train a smaller ‘student’ model that they want to fine-tune.

Prompts are also provided for the use case you’re after.

Model Distillation automates the process of generating responses from the teacher model and using those responses to fine-tune the student model – which can then behave like the teacher models, with similar accuracy at reduced costs.

The company claims distilled models in Amazon Bedrock are up to 500 percent faster and up to 75 percent less expensive than original models. The trade off is accuracy, but AWS claims that’s less than two percent accuracy loss for use cases like retrieval augmented generation (RAG)

Customers will be able to select a teacher model and then a small model within the same family, such as Llama or Claude.

The company is also upping its play in one of the big trends, AI agents, introducing multi-agent orchestration to the Bedrock platform, enabling customers to build collaborative AI agents and streamlined workflows.

AWS CEO Matt Garman told attendees complex tasks such as performing financial analysis across thousands of different variables may require a large number of agents each with their own specialisations.

Multi-agent collaboration capabilities will enable companies to build, deploy and manage multiple AI agents working together on complex multi-step tasks. AWS cites the example of an retail operations multi-agent systems including agents specialised in demand forecasting, inventory allocation, supply chain coordination and pricing optimisation, with the collaboration, communications and task delegation managed behind the scenes.

The Amazon Bedrock Marketplace, meanwhile will provide access to more than 100 specialised foundation models from enterprise providers including IBM and Nividia alongside more niche offerings such as Evolutionary Scale’s ESM3 for protein research and general purpose foundation models from the likes of Anthropic and Meta.

And with AI’s power guzzling demands – and the impact on the environment – still hitting the headlines, AWS was also keen to up its sustainability credentials, launching new data centre components to support AI innovation while also improving energy efficiency.

The new capabilities, which include power, cooling and hardware design ‘innovations’ to create a more energy efficient data centre, will be deployed across AWS’ new data centres. The modularity of the designs will also enable the components to be retrofitted into AWS’s existing infrastructure.