It's a common question when you're choosing AI tools: a vendor, or someone writing about a vendor, says its AI can now do something. How much of that should you believe? On 5 November 2026 the research group METR published an independent assessment that makes a good worked example, because the report is careful, and a short summary can easily lose what makes it careful.
This isn't a piece about AI safety. It's about how to read a capability claim, with checks you can apply to any of them.
What the report says
METR assessed how much AI is speeding up AI research and development at Anthropic. Its central measured finding:
"Since October 2025, benchmark scores (measured via AECI, Anthropic's version of the Epoch Capabilities Index) for Anthropic's models have been growing at an estimated 1.9x the pre-October 2025 rate [95% confidence interval: 1.5–2.8x]"
Its overall conclusion is worded just as carefully: "As of October 2026, Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025." And on how much of that is down to AI itself: "We guess that a majority (perhaps ≈80%) of the current (October 2026) acceleration is due to AI."
Check 1: keep the interval
"1.9x" on its own sounds precise. The report gives a 95% confidence interval of 1.5x to 2.8x, and says separately that "We would not be surprised if the current rate of growth was closer to 1.3x or 3x the historical rate." That's a wide range. A summary that keeps the 1.9x and drops the range has changed the claim.
Check 2: keep the hedging words, and the scope
"Likely". "We guess". "Perhaps ≈80%". These aren't filler. They're the authors telling you how confident they are. Strip them and an estimate turns into a finding.
Scope matters as much. The 1.9x is about benchmark scores, and the conclusion is about "well-specified tasks". It isn't a statement that the products are twice as good, and the report says so itself: if recent models have been improving faster on clean, easy-to-grade tasks than on messy ones, "then we may be overstating the acceleration in real-world impact."
Check 3: if a threshold is cited, find it in the document
Anthropic lists AI R&D acceleration as a key risk indicator in its Responsible Scaling Policy (RSP), which sets thresholds. So it would be easy to read this report as a verdict on those thresholds. The report rules that out itself, in its process section: "In this assessment, we do not pay special attention to the thresholds set in the RSP."
If you see this report described as showing a threshold reached or approached, that conclusion isn't in the report. Whenever a claim cites a threshold, a regulation or a standard, search the source for it. If it isn't there, someone added it.
Check 4: read the limits the authors give you
The most useful parts of the report are the easiest to lose in a summary.
- Redactions. "Anthropic redacted a majority of the evidence we found most informative for these conclusions." METR adds that "Much of the redacted evidence we viewed had multiple plausible interpretations", and that as a result "the public has to rely on METR's judgment much more than would be ideal."
- Access. METR could ask for any information it thought relevant, but "Anthropic could unilaterally approve or reject our requests." Anthropic also changed the scope midway, cancelling a planned check of how its compute is split between capabilities and safety.
- Both directions. The report sets out why the real figure could be higher, for example if growth is speeding up continuously, and why it could be lower, for example if non-AI factors explain most of the change.
- Independence. The assessment was a high-access evaluation carried out at Anthropic over eleven weeks. METR says it "did not request or receive compensation for this assessment", and that Anthropic provided laptops and token credits. It publishes a conflict-of-interest checklist alongside the report, so readers can weigh that for themselves.
None of this makes the report weak. A wide-interval estimate from a partial view, with its limits stated, is more honest than a clean number would be. The problem only starts when the limits get left behind.
The checklist, for any vendor claim
- Open the primary source. Not the post about it, and not the headline.
- Keep the range. If there's a confidence interval or a "between X and Y", it belongs with the number.
- Keep the verbs and the scope. "Likely", "we guess" and "on well-specified tasks" are part of the claim.
- Check that cited thresholds, standards or regulations actually appear in the document.
- Check the dates. Stories get bundled together. Make sure anything presented alongside a report was actually written about it, and not before it.
Source: METR, "Independent Assessment of AI R&D Acceleration at Anthropic in October 2026" (published 5 November 2026; read 6 November 2026).