On October 26, 2026, Epoch AI published an evaluation that tests models on real work: 11 tasks drawn from its own daily operations, across five categories, given to six frontier models to see how far they are from genuinely taking over a research job. The result is sobering: the leaders, Claude Fable 5.1 and GPT-6 Astra, averaged only 65% of the rubric items, a long way from the 100% that would mean work ready to hand over.
How the test was actually run
Every task comes from real work inside Epoch: making a graphic in the house style, writing a short data-driven analysis, building a data browser, researching an AI data center, and proposing a research idea plus a pilot experiment. Each model got the highest reasoning setting and the same level of material a new hire would receive, with no human intervention in between, and was then graded item by item against employee standards. Epoch stresses that this fills a gap in conventional benchmarks: tests that only ask whether an answer computes correctly can hide how clumsy models are at real work.
Three places where models get stuck
First, they fail to learn one organization's house rules. Given months of design drafts, palettes, and templates, Claude Fable 5.1 still produced an information-overloaded graphic, where an Epoch designer delivered one clean chart; GPT-6 Astra, writing a data insight, also misjudged which topics readers actually care about. Second, judgment lags behind ideas. In a pilot experiment, GPT-6 Astra set the token budget for the models under test so low that 22% of runs returned no answer at all — yet it wrote that failure up as a finding about models being sensitive to budgets. Third, open-weight models fall further behind: Kimi K3, analyzing physics papers, got its filter wrong and counted papers from the whole site as physics papers, badly exaggerating the trend — even though its 158 on the Epoch Capabilities Index is roughly on par with Grok 4.6.
65% does not mean saving 65% of the time
Epoch puts it bluntly: by employee standards, every one of these outputs would be sent back for revision. The 65% is a share of rubric items, not hours saved. There is one exception, removed from the suite: Fable 5.1 now passes a basic chart task on the first try most of the time, and Epoch already partly automates that step. For ordinary users, the conclusion splits by task: well-defined work like coding and data analysis can safely go to a model first, but anything involving taste, judgment, or what a result actually means still needs a human at the end — the more capable the model, the less skippable that review becomes.
