What did METR find inside Anthropic?
METR, a nonprofit that evaluates frontier AI models, published an independent assessment of Anthropic on 5 November 2026. The evaluation ran from 12 August to 29 October, about eleven weeks. It used 2.5 full-time METR staff and roughly fifteen interviews with Anthropic researchers.
The assessment had three questions. Has Anthropic's rate of capability improvement accelerated? How much of any acceleration comes from AI? Do bottlenecks look likely to slow it down?
The headline finding is a rate, not a single capability. Anthropic's internal benchmark scores grew about 1.9 times faster after October 2025 than before it. METR gives a 95% confidence interval of 1.5 to 2.8.
Put another way, Anthropic made roughly two years of historical progress in the past year. METR compares that span to the distance from Claude 2.1 to Sonnet 4.5.
The report rules out one simple explanation. METR says it is confident the acceleration was not caused by faster growth in training compute. That matters because more compute has been the industry's default path to better models.
Attribution is less certain. METR guesses that a majority of the current acceleration, perhaps about 80%, comes from Anthropic's use of AI in research and development. It describes that estimate as uncertain. Non-AI factors likely drove much of the earlier acceleration in late 2025.
Internal surveys point in the same direction, though METR calls them loose evidence. Slack polls where staff rate their own speedup reached about 12 times in October 2026. The Verge reported that this was roughly double what staff had reported a few months earlier while using Mythos 5.
METR also saw three unreleased models. Model 2 finished training in July 2026, Model 3 in August or September, and Model 4 in September or October. An older unreleased model from 2022 was included for comparison.
None of this shows a lab running itself. METR says AI still assists human researchers more than it replaces them. It also reports some signs that fully autonomous contributions could grow soon.
How did METR measure AI accelerating AI research?
METR did not invent a new leaderboard. It used AECI, Anthropic's version of the Epoch Capabilities Index, which combines benchmark results into a single capability score.
The method fits two straight lines to the score history and estimates where the slope changes. METR places the breakpoint at 7 October 2025.
The estimate depends on modelling choices. Across different link functions, the measured acceleration ranges from 1.1 to 2.9. Varying snapshots or benchmarks moves it between 1.9 and 2.2.
METR calls the result approximate. Its staff would not be surprised if the true rate were closer to 1.3 or 3 times the historical pace.
The counterfactual is the hardest part. METR can observe historical and current growth, but not what would have happened without AI. It pieces that view together from interviews, surveys, code counts, experiment usage and compute records.
The evidence has known gaps. Benchmarks reward well-specified tasks that can be graded automatically. Messy research work is harder to score and may improve at a different rate.
Saturation is a concern too. Model 4 scored above 50% on every AECI benchmark it was tested on. METR says the acceleration looks reasonably robust to that problem, but it cannot fully rule it out.
The data also arrive late. A model's effect on research appears only after training finishes, so the newest acceleration may not be visible yet.
METR checked other explanations. It found no clear bottlenecks in algorithms, training compute, experiment compute or data for the next twelve months. It also said its view of those bottlenecks changed several times.
One comparison helps. Anthropic's Haiku 5.5 system card reported a 1.7 times acceleration. METR's higher figure comes mostly from including newer, unreleased models.
A baseline problem works the other way. Before late 2025, Anthropic's models grew more slowly on AECI than the wider industry frontier. METR says that unusually low starting rate could make the acceleration look larger than it is.
Why did Anthropic redact the report?
The public report is shorter than the work behind it. METR submitted a roughly 100-page full report to Anthropic on 22 October. The public version appeared on 5 November.
Under the agreement, Anthropic had unilateral rights to redact non-public information. METR could disclose that redactions happened and whether its conclusions depended on them.
METR says Anthropic removed a majority of the evidence it found most informative. Its public table shows what survived. Benchmark scores were shared and described. Intermediate capability measures, algorithmic progress estimates and most R&D inputs were shared but not described.
Bottleneck analysis and forecasts were partly shared but not published. Interviews were published only in part. METR says some redactions were hard to describe without revealing the withheld material itself.
Anthropic gave a reason. It said the redactions were “aimed at removing confidential information that could accelerate other AI developers.” METR told The Verge that most of the withheld material was very high-level, though it did not judge whether the commercial argument was justified.
The effect is a trust problem. METR writes that readers cannot reconcile the evidence with other sources or reach their own conclusions. “The public has to rely on METR's judgment much more than would be ideal,” the report says.
Anthropic cofounder Sam McCandlish responded on X. “We redacted a lot, and I accept that this makes it harder for others to check METR's work,” he wrote. He argued that sharing what drives the speedup would be commercially sensitive and would hand rivals a roadmap.
McCandlish said Anthropic made the right call and would publish its own report on AI progress in the coming weeks. Anthropic declined to answer The Verge's questions and pointed to that post.
One planned part of the audit was dropped. Anthropic cancelled a verification of how much compute it allocates to capabilities versus safety. METR says the company wanted to keep the engagement focused on risk assessment and would revisit the question later.
Did Anthropic already cross its own warning threshold?
A second METR document examines Anthropic's own safety case. It reviews the “Risks from automated R&D” section of the company's August 2026 Risk Report. METR saw both the public report and a private version during the wider assessment.
Anthropic's Responsible Scaling Policy sets two thresholds for automated R&D. The Full Substitution Threshold asks whether a model could replace the entire research staff at a cost within a factor of five. The 2x Acceleration Threshold asks whether progress has doubled for reasons linked to automation.
Anthropic's public report said the second threshold had not been crossed. It described leading indicators showing acceleration “by less than a factor of 2.” It also attributed the 2025 acceleration mostly to non-AI factors.
METR agrees that models deployed internally by 15 July 2026 posed low risk of directly automating key R&D or causing acute catastrophic harm. It is less comfortable with the threshold call. METR says it is plausible that the early-warning threshold had already been crossed.
Both METR's researchers and Anthropic's staff lean toward “not crossed,” according to the review. METR still argues that the public report relies on contestable judgment calls and understates the acceleration.
The distinction matters. Anthropic frames the threshold as an early warning, not proof that harm has occurred. A company can therefore cross it and still describe overall risk as low.
METR's main audit puts current acceleration near 2x. Its separate review says the warning line may have been reached months before the public knew. Neither document claims that a dangerous system exists today.
What are the limits of an 11-week audit?
The audit was unusually deep for the industry, but it was not unrestricted. METR worked on Anthropic devices for the duration and received free tokens for analysis. It did not take payment.
Access exceeded expectations in some areas. METR says Anthropic shared more than the project initially scoped. It provided internal benchmarks, compute metrics, code statistics, survey data and documents on autonomous research.
Access was still narrow. METR could request information, but Anthropic could approve or reject each request. A small number were denied, and METR did not request or receive access to sample from non-public models.
The team also lacked staff-level access. METR says reading documents on demand and observing internal debate would have produced a richer picture. It changed its interpretation of key data several times as context arrived.
Redactions compound the problem. Outside researchers cannot test METR's reasoning against the withheld evidence. That makes the report an expert judgment, not a reproducible measurement.
The estimate could be too high. Non-AI factors might explain more of the gain, the benchmarks may overstate real-world progress, and Anthropic's earlier growth rate may have been unusually low.
It could be too low. Growth might be accelerating continuously, automation might arrive sooner than METR assumes, and benchmark scores may lag real-world impact.
Scope is another limit. This is one company over one period. It says little about OpenAI, Google DeepMind or Chinese labs, except that similar methods could be applied to them.
METR draws a line between this project and Anthropic's embedded evaluator programme. The 15 October programme gives six METR employees employee-like access. The November report came from a separate, smaller engagement.
That programme followed a public commitment. Anthropic says it signed the White House Accord on Super Intelligence, a voluntary pledge to work with outside auditors, before announcing the deeper access.
Who is affected by the findings?
Anthropic is preparing to go public while arguing that frontier development should slow down. A report showing faster internal progress cuts against that message.
Investors now have an outside estimate of research velocity. They also have a reminder that key evidence sits behind a redaction agreement. Both facts are likely to surface in IPO scrutiny.
Safety researchers are split between alarm and calibration. Marius Hobbhahn, chief executive of Apollo Research, told The Verge the pace looks faster than the AI 2027 scenario forecast. His advice was to “brace for craziness.”
Alan Chan of GovAI said the findings should make sceptics reconsider how quickly AI can accelerate AI work. METR researcher Ryan Greenblatt wrote that he “was seriously wrong about RSI speed up.”
Critics focused on consistency. Daniel Kokotajlo of the AI Futures Project accused Anthropic of “recklessly gambling with our lives.” He argued it was inconsistent to cite rivals' progress as a reason to hide evidence.
Brianna Rosen of the Institute for AI Policy and Strategy tied the issue to democratic oversight. She said evaluators need freedom to explain their conclusions and that the report was not transparent enough about its evidence.
Politicians are starting to respond. New York Assemblyman Alex Bores called on Congress to ban recursive self-improvement, according to The Verge. Many Democrats elected this month campaigned against AI companies.
Other labs face a precedent question. If Anthropic can commission an outside audit and redact the core evidence, rivals can copy the model. If METR's criticism sticks, they may need to offer stronger publication rights.
Buyers who rely on system cards and voluntary evaluations now have a clearer view of their limits. The report shows how much supporting evidence can remain private.
The dispute also lands inside a wider argument about disclosure. Anthropic's own measurement post says labs should publish how much R&D AI performs, how agents are overseen and how compute is allocated. The METR episode shows how hard that is when the evidence is commercially sensitive.
What to watch next
The next data point is Anthropic's own report. McCandlish said it will describe the speed of AI progress and new safety steps. The timing matters because it will either confirm or dispute METR's estimate.
Watch whether Anthropic publishes verifiable metrics. Its measurement post says that, as of August 2026, Claude was not fully autonomous for any measured R&D task. It says Claude “leads” 26% of the work and that more than 90% sits at or above “collaborates.”
Those numbers give outsiders a baseline. If the share of autonomous work rises while the redaction policy stays, the gap between public claims and public evidence will widen.
Watch the IPO filings. A prospectus that cites outside audits will invite questions about what those auditors could not publish.
Watch how the RSP thresholds are interpreted. METR notes that Anthropic does not publish a precise risk level that would indicate high risk. Its review describes the 2x threshold as an early warning rather than proof of danger.
Watch the embedded evaluator programme. Anthropic has said it cannot veto publication but can require redactions in that arrangement. Whether the programme produces checkable findings will test whether access alone improves oversight.
Watch other labs. OpenAI said in September it is making strong progress toward an AI researcher that can advance deep learning and alignment by March 2028. If that work accelerates, independent measurement will matter more than another benchmark chart.
The near-term question is simpler. METR says the acceleration is probably real, uncertain in size and only partly inspectable. The next year will show whether that combination becomes the industry norm or a warning that was ignored.