Anthropic and Accenture Put $2 Billion Behind Independent AI Evaluation

As frontier models become harder to assess from the outside, Anthropic is bringing independent evaluators closer to the systems being built The AI industry is getting better at building increasingly capable models. It is still working out how to properly inspect them. That gap is becoming harder to ignore. As frontier models move from generating text and code to handling longer, more complex tasks, companies need to know not just what a model can do, but how it behaves under pressure, where its safeguards fail, and whether the processes around its development are working as intended. Anthropic and Accenture are now putting significant money behind one possible answer. The two companies said on September 18 that they will work together on independent evaluation of Anthropic's frontier AI models. Each expects to invest at least USD 1 billion over the next five years to build capacity for the work. The partnership will be led by Faculty, Accenture's specialist AI business, and will cover model evaluation, red-teaming, alignment assessments, and testing of model safeguards. As AI developers face growing pressure from regulators, businesses, researchers, and the wider public to demonstrate that increasingly capable systems can be deployed safely. Reuters also reported that recent incidents involving AI agents operating beyond expected boundaries have added to concerns about how easily advanced systems can be monitored and controlled. Bringing evaluators inside the lab The more significant part of the announcement is not the size of the investment. It is the proposed model of evaluation. Anthropic calls it "embedded evaluation". Instead of having an independent organisation assess a model from outside after development, evaluators would work inside the AI company, with access comparable to that of an employee. That access could give evaluators a view that conventional testing cannot. They would be able to follow models during training, observe decisions around how systems are developed and deployed, and speak directly with employees involved in those decisions. Anthropic said: > "From this vantage point, embedded evaluators can assess how a company operates, verify that it is keeping its safety commitments, and identify blind spots." The model safety isn't determined only by the behaviour of a finished system. Decisions made during training, deployment, testing, and the setting of safeguards can all influence what happens once a model is in the hands of users. Faculty will lead the work for Accenture. The company brings experience working with businesses and governments that deploy AI in operational settings, something Anthropic says will inform the evaluation work. Independence is the harder question There is, however, a tension built into the approach. An evaluator working inside the company gets access that an outside reviewer may not. But that proximity also raises questions about independence, particularly when the AI company is funding the work directly. Anthropic itself acknowledges that the model is still being developed. There are currently no established standards defining what information embedded evaluators should receive or how they should report their findings. There is also no settled system for funding independent evaluation. For now, Anthropic will fund Accenture's work directly. The company says it is also in discussions with METR and other non-profit evaluators to test elements of embedded evaluation using their own funding. Anthropic's longer-term position is that frontier AI should have an ecosystem of evaluators working to shared standards, with pooled or government funding eventually supporting independent evaluation. That would move the practice beyond an individual agreement between a model developer and an evaluator. It would also address one of the central questions around the current approach: who decides what evaluators can see, and who ultimately pays for scrutiny of increasingly powerful AI systems? Anthropic and Accenture will each invest USD 1 billion in independent AI evaluation, testing frontier models, safeguards, and the emerging embedded evaluation model. Anthropic is also clear that the Accenture partnership is not exclusive. The company expects to work with additional evaluators, while Accenture plans to work with other AI developers in similar capacities. Anthropic said further evaluators will be announced in the coming weeks. That's crucial because different evaluators can bring different areas of expertise. Red-teaming a model, assessing alignment, investigating an incident, and examining how safeguards perform in practice are related but distinct tasks. The arrangement also comes alongside a wider push for greater transparency around unexpected model behaviour. Reuters reported that OpenAI has said it will publish regular reports on concerning model behaviour, adding another example of AI labs moving towards more formal reporting of model risks and incidents. Anthropic, meanwhile, has stressed that embedded evaluation does not transfer responsibility for safety to the evaluator. > "To be clear, independent embedded evaluators do not reduce our accountability, but help to make it more verifiable. The safety of our models remains our responsibility." That distinction will become increasingly important as AI systems move into more consequential enterprise and public-sector applications. Independent scrutiny can provide another layer of evidence, but it does not remove the responsibility of the company building and deploying the model. The industry still needs rules for the process For now, embedded evaluation remains an emerging model rather than an established industry practice. Anthropic says many operational details are still being worked out, including access, reporting, and funding. That may ultimately prove to be as important as the money being committed. The next stage of AI safety is unlikely to be defined only by better testing tools or more sophisticated red-teaming. It will also depend on whether independent evaluators can get meaningful access to the systems they are assessing, retain enough independence to report uncomfortable findings, and operate under standards that extend beyond a single company's internal processes. Anthropic and Accenture are putting substantial resources behind that experiment. Whether embedded evaluation becomes a broader industry norm will depend on what those evaluators are allowed to see, what they find, and how openly those findings can be reported.