Anthropic is developing its artificial intelligence systems around twice as quickly as it was a year ago amid the CEO's calls to slow down the technology's pace of improvement, according to an analysis released Thursday by an independent monitoring group.
Anthropic invited researchers from the Berkeley, California-based nonprofit, Model Evaluation and Threat Research, to review its internal data and interview staff as it prepares for its Wall Street debut with debate swirling over the potential for AI models to elude human control with catastrophic consequences.
The findings are significant because details about the technology — and its risks — have been provided primarily by AI companies, which have been encouraged by President Donald Trump's administration to police themselves on matters of safety.
It was during the research group's assessment period that Anthropic CEO Dario Amodei called for the AI industry to slow its work so safety measures can catch up. In a Sept. 12 essay responding to mounting concerns about runaway AI, he urged the AI world to "slow the pace at which we improve the capabilities of AI models."
Sam McCandlish, one of Anthropic's co-founders, responded to the report Thursday in a lengthy post on X, writing that many of METR's conclusions match his own. "AI is speeding up our research, but it's hard to say exactly how much," he wrote. He also said the company plans to release a report with its own analysis of the speed of AI progress within Anthropic in the coming weeks.
McCandlish said this work with METR began "well before and is separate from the embedded third-party evaluators that are part of our commitment to pacing the frontier," but added that "it's a good illustration of why we think evaluators are a good idea."
RELATED STORY | Websites shut down globally after AI finds flaw affecting 70% of the internet
The San Francisco-based company is planning to go public later this month in what could be the biggest initial public offering of stock ever.
AI is playing a bigger role in its own improvement
METR estimated that Anthropic is improving the capabilities of its models to do "well specified tasks" around twice as fast as it was a year earlier. The nonprofit's researchers said they believed much of the reason for the faster development is the help of AI itself.
Anthropic has said its Claude chatbot increasingly is helping the company develop newer, more advanced versions of itself.
METR has conducted evaluations on model capabilities for Anthropic, OpenAI and other labs for years. The nonprofit is funded through donations and does not accept compensation for its work, but AI companies it has partnered with for evaluations have provided access and tokens, which METR uses for evaluations, research and engineering.
One of its two reports was based on analysis of Anthropic's internal data provided by the company over a period of about 11 weeks, from mid-August through late October, as well as employee interviews.
METR said its assessment focused on a few key questions, including whether the growth rate of Anthropic's internal capabilities has accelerated and how much of that was attributable to AI. The group said it was given "unprecedented access to internal data" on Anthropic's AI research and development process.
Anthropic's redactions to the report limited how METR could communicate more detailed estimates of acceleration, and there were some internal disagreements and uncertainty among METR's team, but the group said it believed the company's rate of improvement had doubled.
"Anthropic appears to have made as much progress on benchmarks in the past year as they did between October 2023 and October 2025," according to the METR report.
"How quickly AI progress is accelerating has many important safety implications, since it determines how quickly the risks from advanced models increase," McCandlish, who is also Anthropic's chief architect, wrote. "But, because most research happens within the frontier labs, it's difficult for external observers and policy makers to get a strong sense of what that rate is."
RELATED STORY | Voters elected governors to curb data centers. Lowering bills for gas and groceries is harder
METR also reviewed both a public risk report Anthropic published in August and a private version, as well as other non-public data, and the organization said the public report "overall understates the level of acceleration that was occurring."
McCandlish acknowledged in his statement that the redactions make it harder for others to "check METR's work," but added that the company decided that sharing "exactly what is driving the speedup isn't the right thing to do," because it's both "commercially sensitive" and "risks handing out a roadmap for others to go faster."
"That would make the risks METR is worried about worse. That's a tradeoff," he continued, noting that the company continues to weigh the pros and cons.
Assessors see little holding back AI's progress
The report was peppered with caveats. METR said it could be overstating its findings, because the acceleration might only be limited to "well-specified tasks" that can be graded automatically or with a language model, instead of "messy" tasks that are harder to define. Or it could be that Anthropic understated its previous progress, making the latest number look bigger.
METR said it was able to "confidently" rule out the growth of computing power used in training AI models as the reason for the acceleration. Traditionally, the amount of computing power, known as compute, is directly related to how well an AI model performs. The researchers said their guess was that the improvement was down to using AI itself to improve AI.
A key concern, METR said, is that using AI to improve itself could start to compound the acceleration, which could be especially worrying given that researchers found there were no "bottlenecks" such as restrictions on computing power or data holding back Anthropic's AI progress.
"We understand that AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of the model training process," the report said. However, Anthropic researchers described significant "signs of life" implying that could happen soon, METR said.
RELATED STORY | AI hardware can be hacked so chatbots provide dangerous answers, research shows
Amodei touched on this issue in his September essay, saying AI advances have been driven primarily by the concept of recursive self-improvement, or what he defined as "AI's growing ability to build the next generation of AI."
"Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all," Amodei wrote.
In that same essay, Amodei championed the idea of having external evaluators embedded within the leading AI labs and specifically highlighted METR as an appropriate third-party evaluator. OpenAI CEO Sam Altman said at the time that he also supported the idea of independent evaluators with employee-like access.
