Amazon Web Services (AWS), Amazon's cloud computing division, is announcing new tools to deal with hallucinations, scenarios in which AI models behave unreliably.
The service, Automated Reasoning checks, announced at AWS's re:Invent 2024 conference in Las Vegas, validates model responses by cross-referencing customer-provided information to ensure accuracy. AWS claims in a press release that automated inference checks are the “first and only” safeguard against hallucinations.
But it's, well…just be generous.
Automatic inference checking is similar to the correction feature Microsoft introduced this summer, which also flags AI-generated text that may be factually incorrect. Google also offers tools in its AI development platform, Vertex AI, that allow customers to “foundate” their models using data from third-party providers, their own datasets, or Google Search.
In either case, automated inference checks available through AWS's Bedrock model hosting service (specifically the Guardrails tool) can figure out how the model arrived at the answer and identify whether the answer is correct. I'll try. Customers upload information to establish some kind of truth, and automated inference checks create rules that can be adjusted and applied to models.
Once the model generates responses, automated inference checks validate them and, in cases of possible illusions, tease out the ground truth to get the correct answer. This answer is presented with likely misconceptions so the customer can see how far off the mark the model is.
AWS says PwC is already using automated inference checks to design AI assistants for its clients. And Swami Sivasubramanian, vice president of AI and data at AWS, suggested that these kinds of tools are exactly what attracts customers to Bedrock.
“With the introduction of these new capabilities, we are working on behalf of our customers to solve some of the biggest challenges facing the entire industry when moving generative AI applications into production,” he said in a statement. We are innovating,” he said. Bedrock's customer base has grown 4.7 times in the last year and has tens of thousands of customers, Sivasubramanian added.
But as one expert told me this summer, trying to eliminate hallucinations from generative AI is like trying to eliminate hydrogen from water.
AI models create hallucinations because they don't actually “know” anything. These are statistical systems that identify patterns in a set of data and predict which data will come next based on previously seen examples. This means that the model's response is not an answer, but rather a prediction, within error, of how the question should be answered.
AWS claims that its automated inference checks use “logically accurate” and “verifiable inferences” to reach conclusions. But the company did not voluntarily provide data showing that the tool was reliable.
In other Bedrock news, AWS announced Model Distillation this morning. This is a tool to migrate functionality from larger models (such as the Llama 405B) to smaller models (such as the Llama 8B) that are cheaper and run faster. Model Distillation, Microsoft's answer to Distillation in Azure AI Foundry, provides a way to experiment with different models without breaking the bank, AWS says.
Image credit: Frederic Lardinois/TechCrunch
“When a customer provides a sample prompt, Amazon Bedrock generates a response and performs all the work to fine-tune the model on a smaller scale,” AWS explained in a blog post. distillation process. ”
However, there are some caveats.
At this time, model distillation only works with models hosted on Bedrock in Anthropic and Meta. Customers must choose a large or small model from the same model “family”. You cannot choose models from different providers. And the extracted model loses some accuracy – “less than 2%,” AWS claims.
If you're okay with that, model distillation is now available in preview along with automatic inference checks.
“Multi-agent collaboration” is also available in preview. This is a new Bedrock feature that allows customers to assign AI to subtasks in large projects. As part of Bedrock Agents, AWS's contribution to the AI agent craze, multi-agent collaboration provides tools to create and tune AI for things like reviewing financial records and assessing global trends. Masu.
Customers can also designate “supervisor agents” who split tasks and automatically route them to the AI. The supervisor is[give] “Specific agents have access to the information they need to complete their work,” AWS said.[determine] Which actions can be processed in parallel and which require details of other tasks before them? [an] Agent can proceed. ”
“Once all the technical expertise is provided, [AIs] Complete input, supervisor agent [can pull] information together [and] and synthesize the results,” AWS wrote in the post.
It's wonderful. However, as with all these features, we need to see how well it performs when deployed in the real world.