Anthropic’s AI models are now improving about twice as fast as they were a year ago, and AI itself is doing most of the speeding up. That’s the main finding of a new report from the AI safety research group METR, published Thursday after an 11-week investigation inside the company, according to The Verge AI. It’s some of the clearest evidence so far of AI speeding up its own development. It also shows how little the public still gets to see.
📊 What METR found
METR’s benchmark analysis suggests “that Anthropic has made roughly two years of historical progress over the last year.” What stands out is the cause. Bigger training runs have historically driven most AI progress, but the researchers said they “are confident” that more computing power doesn’t explain this jump. They concluded that the majority of the acceleration as of October “is due to AI,” and that AI’s share keeps growing.
Here are the headline numbers:
- Rate of model improvement: About 2x faster than a year ago
- Progress in the past year: Roughly 2 years’ worth, by historical standards
- Self-reported productivity (latest internal model): About 12x without AI help
- Self-reported productivity a few months earlier (Mythos 5): About half that
- Unreleased models trained in the last 6 months: 3
🔬 How the investigation worked
METR combined several kinds of evidence:
- Benchmark trends tracking how quickly Anthropic’s models improved over time
- Confidential research and performance data that Anthropic shared with the team
- Employee interviews about how staff use AI day to day
- Informal Slack polls asking employees how much AI speeds up their work
The 12x productivity figure comes from self-reports, so treat it as a signal and not a precise measurement. Still, it lines up with the benchmark data, and that’s why METR takes it seriously.
🤖 Humans are still in the loop, for now
Right now, most of the gains come from AI helping human researchers, not from AI working on its own. METR flagged that “there are some signs that fully autonomous contributions will play a larger role soon.”
The researchers also found no clear signs of diminishing returns and no obvious bottlenecks that would stop “even faster acceleration.” That’s the part that worries safety researchers. If AI keeps taking on more of its own development, humans could end up with a smaller role in deciding how fast the technology moves.
🔒 Three models nobody outside has seen
Anthropic’s researchers work with models the public can’t use. METR says that in the past six months, the company finished training three unreleased models:
- Model 2: finished training in July
- Model 3: finished August to September
- Model 4: finished September to October
METR also found an unreleased model from 2022, called “Unreleased 2022.” The researchers noted these are codenames shared with them and that the models have different names inside Anthropic. Much of the reported speedup comes from these internal systems.
⚠️ The redaction problem
Anthropic gave METR more access than it had to. METR said the company “shared more information than was initially scoped for this project, surpassing our expectations.” But that access had clear limits. It fell short of the “employee-like” access Anthropic recently promised safety researchers in a separate program.
Anthropic was also allowed to redact the final report, and it used that power a lot. “We submitted a ~100-page full report on our assessment to Anthropic. For the public version, Anthropic redacted a majority of the evidence we found most informative,” METR wrote on X.
METR listed several limitations:
- Its “pipeline of information was relatively narrow”
- A small number of its data requests were denied
- Much of the redacted evidence had “multiple plausible interpretations”
- Anthropic cited competitive concerns, but METR said “most of the redacted information was very high-level”
The report puts the result bluntly: “the public has to rely on METR’s judgment much more than would be ideal, given the weight of the questions answered here.”
💡 Why this matters
For practitioners, the takeaway is practical. Frontier labs are already running models a generation or more ahead of what you can buy, and those models are speeding up the next generation. If you’re planning products or workflows, expect capabilities to jump more often, not less.
The timing matters too. Anthropic is preparing to go public, and CEO Dario Amodei has called for labs to slow down. Outside audits like this one build credibility, but only if the public can actually check the evidence. Expect pressure to grow for independent reviews that come without a lab-controlled redaction pen. You can find the full story at The Verge AI.