LIVE TICKER
[Live wire]Connecting to the newsroom…
← Back to the news wire
Flesch Reading Ease: 28•350 words from the publisher
AIPublished by MIT Technology Review • 11/20/2026, 6:15:15 PMOpen Original Report (MIT Technology Review) ↗

The most misunderstood graph in AI just got a major update

METR’s latest report finds that AI models are still improving at a rapid clip, but they show some weaknesses in tasks that require judgment.

The most misunderstood graph in AI just got a major update
Image supplied by MIT Technology Review.

Reported by MIT Technology Review

METR’s latest report finds that AI models are still improving at a rapid clip, but they show some weaknesses in tasks that require judgment.

A new report from the AI evaluation nonprofit METR finds that some AI capabilities are continuing to improve along an exponential trajectory—but it also indicates that models aren’t quite as good at tasks that require judgment as they are at coding.

The report is an update to the organization’s time horizon plot, which was widely shared— and widely misinterpreted , due to the complexity of the measures it used. METR evaluated models by having them attempt a large suite of tasks, each of which had also been completed by humans. Each task’s difficulty was scored by how long it took the human testers to finish it—and then models were rated according to the length of the tasks that they could complete. That plot showed that AI systems were rapidly improving—every four months, the length of the tasks that frontier models could successfully complete doubled.

More from the original report

The updated plot adds more tasks—some that are estimated to take humans far longer than the tasks in the original suite, and others that are more open-ended. One of the biggest questions in AI right now is the extent to which progress on “verifiable” tasks—where it’s possible to check automatically whether an AI system has succeeded or failed—carry over into tasks on which success or failure is harder to measure. We know how to teach neural networks how to do verifiable tasks: Researchers have the systems attempt the task many times, strengthen the neural pathways that contribute to success, and weaken the pathways that contribute to failures. That’s why AI systems have become so good at coding and math, two domains where it’s possible to determine automatically whether a solution does or doesn’t work.

It’s much harder to apply this technique, known as reinforcement learning, to tasks where success is less well defined or takes longer to ascertain—such as choosing a new research direction. For the same reason, it’s tough to evaluate AI performance on these tasks, so METR’s original time horizon…

Original Source Attribution
Published via MIT Technology Review • Source domain: technologyreview.com
Visit Original Source (MIT Technology Review) ↗

Client & Community Discussion (0 Comments)

Share your benchmark feedback, workflow questions, or implementation notes below.

● Live Comments Open
Instant publication • Moderated for technical relevance