Everyone agrees that frontier AI labs need independent auditors. The popular assumption is that the hard part is getting the labs to say yes. It isn’t. Anthropic and OpenAI have both said yes. According to Don’t Worry About the Vase, the real problem is simpler and uglier: nobody can agree on who the auditors should be, or who pays them.
🧭 What happened
Dario Amodei’s essay “We Must Pace the Frontier” committed Anthropic to embedded evaluators. These are outside people placed inside the company with employee-level access, who give an outside view and report on what’s going on. OpenAI followed with the same commitment, along with what Zvi Mowshowitz calls a “milquetoast but welcome” call for international coordination.
Then Anthropic named its first partner, and it wasn’t a safety nonprofit. It was Accenture.
- The work will be led by Faculty, Accenture’s specialist AI business.
- It covers red-teaming, alignment assessments and testing model safeguards.
- Anthropic and Accenture each expect to invest at least $1 billion in this area over five years.
- Anthropic will fund Accenture’s work directly.
- The deal is non-exclusive. Anthropic says it’s talking with METR and other nonprofits about piloting embedded evaluation with their own funding.
⚖️ The independence wish list
A group led by Geoffrey Hinton, Stuart Russell and Arvind Narayanan published minimum standards for credible evaluators:
- Payment can’t depend on findings, and there’s no ownership, governance or editorial control by the lab
- Conflicts of interest get disclosed
- A range of viewpoints and expertise is included
- Evaluators are transparent and protected from retaliation
- Access matches that of highly privileged employees, with exceptions for data protection
Mowshowitz endorses the list. He also points out that you can’t have all of it at once right now. Someone has to hire these people and pay them. And anyone qualified and trusted enough to do the job almost certainly got that experience at a top lab.
🔍 Why the purists are missing something
This is where the contrarian case gets interesting. Critics say a lab paying Accenture to grade its homework is a conflict of interest. They’re right. But look at the alternatives.
Gabriel Weil argued back in July that developers shouldn’t hire their own referees. Fair enough. But the government doesn’t want to pay, and Mowshowitz notes the White House looks openly hostile to audits on principle. EA-linked funders bring their own neutrality questions. That leaves the labs.
As Mowshowitz puts it, even “well-established, highly trusted auditing schemes” in other industries haven’t solved the revolving door or the funding problem. Financial auditors get paid by the companies they audit. It’s a flawed system, and we use it anyway, because no audit is worse.
Anthropic seems to be running a two-track approach. Accenture brings enterprise scale and a neutrality that’s harder to question. Self-funded nonprofits like METR bring deep safety expertise. Neither one is perfect. Together they’re better than either alone.
💸 The insurance angle
Weil’s alternative is mandatory liability insurance. Mowshowitz likes it in principle, but he’s skeptical. A policy big enough to cover catastrophic risk probably can’t be bought, and the policies that do get sold won’t cover the scenarios people worry about most. He also asks the obvious question: if Anthropic is worth $2.2 trillion and still might be “judgment-proof,” who isn’t?
His sharper idea is to require something like $100 billion or $1 trillion in coverage. That’s not because it fixes the incentives. It’s because labs would then have to disclose what the policy cost, and insurers would have to disclose how they priced the risk. The market price of catastrophe would become public information, and that could be worth more than the payout.
🚨 Why this matters now
The timing isn’t abstract. Mowshowitz flags a new wave of AI hacking incidents, including several OpenAI hadn’t disclosed and a fresh one that forced the company to pause its most advanced model again. He argues these show we badly need embedded evaluators “and plausibly far harsher measures.”
What practitioners should take from this
- Don’t wait for perfect independence. Imperfect oversight that ships beats ideal oversight that never happens.
- Watch the structure, not the logo. What matters is retaliation protection, real access and freedom to publish, not who signs the checks.
- Expect a new market. Two $1 billion commitments point to AI evaluation becoming a real professional field. Consultancies, insurers and safety nonprofits will all compete for it.
- Enterprise buyers, pay attention. Third-party evaluation reports may soon show up in vendor due diligence.
The first round of evaluator announcements is due in the coming weeks, and we’ll learn whether other labs go beyond “the least we can do.” Don’t Worry About the Vase has the full breakdown, including the METR details and OpenAI’s response.