ToolNavs Find Useful AI Tools
Submit Sign in
Back to AI information
OpenRouter Benchmark: Same Task, MCP Burns 42x the Tokens of a CLI

OpenRouter Benchmark: Same Task, MCP Burns 42x the Tokens of a CLI

AI information • Admin • • 1 views

In a benchmark published on November 11, 2026, OpenRouter ran the same GitHub task through three different ways of giving tools to an agent — and the gap was stark. On Claude Sonnet 5.5, the command-line approach used just 1,421 input tokens, while the MCP server with its default 46 tools burned 59,417, a full 42 times more. And the biggest cost is not even the tool definitions — the tool results themselves are more expensive.

Same task, five bills

ApproachSonnet 5.5 inputCost per task
CLI1,421$0.0075
Skill3,315$0.0118
MCP (only 2 tools)23,228$0.0552
MCP (default 46 tools)59,417$0.1274
MCP (all 91 tools)101,351$0.2113

At that rate, a thousand tasks cost $127.40 with the default MCP setup, versus $7.50 with the CLI. The task was to look up the latest release of the cli/cli repository and the three most recently opened issues labeled bug, with each of four models (Haiku 5.5, Sonnet 5.5, GPT-6 Luna, Gemini 3.8 Flash) run three times and the median taken.

The bill is driven by results, not definitions

Counting definitions alone, the 46 default tools of the GitHub MCP server take 19,436 tokens on every request. But the real cost is a single tool result: the full release object returned by get_latest_release is 42,182 characters long, and 87% of it describes 22 download files, while the model only needs the version and the date. That one result cost about 20,400 tokens on Sonnet 5.5 — more than all 46 tool definitions combined. GPT-6 Luna went further: it asked for the full text of 100 issues in one go and got 281,963 characters back.

Three ways to bring the MCP bill down

First, expose only the tools the task needs: with just 2 tools loaded, the whole task drops from 59,417 to 23,228 tokens. Second, shrink results before the model sees them: capping a single result at 4,000 characters brings the 2-tool setup down to 5,357. Third, fetch fields instead of whole objects: asking list_issues for only the number, title, and creation date shrinks its result from 8,336 to 714 characters. OpenRouter's own Tool Search (in beta) takes another route: it defers all 46 tool definitions, cutting the up-front cost to under 578 tokens, and the whole Sonnet task falls from 59,417 to 30,334 — though it costs more on Haiku 5.5, and OpenRouter recommends keeping the three to five most-used tools loaded. OpenRouter was acquired by Stripe for $7.5 billion on November 10; see Stripe completes its $7.5 billion acquisition of OpenRouter, buying the single gateway to 400+ models.

Recommended Tools

More