An outside assessment can improve our understanding of AI while leaving important evidence unavailable to its readers. That distinction matters when a business uses an independent review to decide how much confidence to place in a technology supplier.
In a report published on November 5, METR describes its assessment of AI research and development at Anthropic from August 12 through October 29, 2026. It says Anthropic redacted most of the evidence it considered most informative for its risk conclusions. Readers therefore have less opportunity to inspect the reasoning themselves. METR also describes substantial uncertainty in interpreting the evidence.
Check the engagement behind the label
The report explicitly distinguishes this project from the embedded-evaluator arrangement Anthropic announced on October 15. METR says this assessment did not involve full employee-like access. Anthropic's separate announcement describes that newer arrangement and its publication and redaction rules. Treating every reference to METR as the same engagement would blur an important boundary.
For a buyer, the useful question is what a particular review actually covers. An evaluation of an AI developer's research process addresses a different decision from a test of whether its chatbot can correctly handle your company's order exceptions. Both may be valuable. Neither should quietly become evidence for the other.
Start by writing the decision on a page: approving a pilot, expanding access to customer information, or allowing a tool to perform a particular action. Then identify which part of the assessment bears on that decision. If the connection requires several assumptions, record them beside the recommendation rather than hiding them inside a general confidence score.
Separate access from what a reader can verify
An evaluator may examine material that a customer cannot see. That creates a practical reliance decision. The customer can assess the evaluator's explanation, methods and declared limits, while recognising that some underlying evidence remains beyond direct inspection.
This is not a reason to pretend the unseen material is either reassuring or alarming. Its contents cannot be reconstructed from a blacked-out section. It is a reason to make the remaining uncertainty visible when deciding how much responsibility to delegate.
Consider a hypothetical purchasing review. One colleague proposes allowing an assistant to draft internal summaries. Another wants the same approval to cover sending final advice to customers. A broad research assessment may inform their discussion, but it does not supply the missing customer-specific test. The second proposal still needs its own evidence about accuracy, boundaries and correction.
Use the assessment without stretching it
The following approach is our business analysis, not a conclusion that METR tested in client operations. Keep a short evidence register alongside the purchasing decision. Identify the claim, the material a reviewer could inspect, the limitation and the next observation that would change the decision.
For an internal summary pilot, that next observation might be a comparison between the assistant's output and the underlying records. For a customer-facing task, it might include how a human reviewer catches an incorrect interpretation before it is sent. Those are proposed tests, not performance results reported here.
Also distinguish an approval that can be reversed easily from one that is difficult to unwind. A limited trial can create useful local evidence without committing the organisation to every capability a supplier advertises. Make the review date and the person responsible for interpreting failures explicit.
When a new assessment appears, return to the original decision. Ask whether it answers the unresolved question or simply adds context. Useful independent scrutiny deserves careful reading; careful reading includes preserving its boundaries.
Source scope: this article discusses the public assessment and announcement, not the confidential underlying evidence. Business examples and proposed review steps are original analysis.
AI-generated conceptual editorial image; fictional review scene, not a photograph of METR or Anthropic.