Skills vs MCP: what the data says

Two standards that started at Anthropic now shape how most AI agents reach tools and learn procedures. The Model Context Protocol, released in November 2024, connects an agent to tools and data in other systems. Agent Skills, released in October 2025, are folders with a SKILL.md file of instructions and scripts that an agent loads only when a task calls for them. From the day the second one shipped, developers argued about whether it made the first one obsolete. Simon Willison predicted a "Cambrian explosion in Skills" that would make the MCP rush look pedestrian, and by February an essay called "MCP is dead. Long live the CLI." was on the front page of Hacker News.

Most of that argument ran on anecdotes. We wanted the numbers: what each one costs in context before the user types anything, what happens to accuracy as tools pile up, what the head-to-head evaluations measured, how fast each spread when you line them up by age, and what the security scans found. So we collected every published eval and measurement we could find, pulled download counts, star histories and client lists ourselves, and checked the claims against their sources. Vendors that sell one side or the other are labelled, and every number links to where it came from. The short version: the data doesn't crown a winner, and in September the two standards merged.

Two layers, not two rivals#

The two are different kinds of thing. A skill is text and, often, scripts: a name and a one-line description the agent always sees, and a body it reads when the description matches the task. MCP is a protocol: a server, local or remote, that exposes tools an agent can call, with its own transport, versioning and authorization. Anthropic's own one-line summary, from its guide to how the pieces fit, is that "MCP connects Claude to data; Skills teach Claude what to do with that data."

OpenAI's plugin docs draw the same line as a rule. "Use skills when instructions and the tools already available to the model are enough to complete the task", and build an MCP server "when your plugin must connect to a service, expose a controlled set of tools, authenticate users, or run behavior on infrastructure you operate." A plugin can carry both.

What each costs before you type#

Every MCP tool a client loads comes with a name, a description and a JSON schema, and by default all of it goes into the model's context at the start of the session. Anthropic measured a common five-server setup (GitHub, Slack, Sentry, Grafana and Splunk) at "58 tools consuming approximately 55K tokens before the conversation even starts", and says it has seen tool definitions reach 134K tokens internally. Mario Zechner measured Playwright's MCP server at 13.7K tokens and Chrome DevTools' at 18K, and replaced both with a README for his own browser scripts that came to 225 tokens.

A skill costs much less up front because only its name and description are loaded. Anthropic's docs put that at "~100 tokens per Skill", with the body, under 5K tokens, loaded only when the skill is triggered, and bundled files costing nothing until they're read. Armin Ronacher, who moved all his MCP servers to skills in December, put the difference in one line: "skills do not actually load a tool definition into the context."

Put those per-unit figures into a simple model and the gap is two orders of magnitude.

The model has two soft spots. The 100 tokens is a budget in the docs, not a measurement. And skills have a ceiling of their own: Claude Code's docs say its skill listing is capped at 1% of the context window, dropping descriptions when it overflows, so a large skill collection degrades by losing descriptions rather than by growing.

The MCP side answered by borrowing the idea. Anthropic's Tool Search defers tool definitions until the model searches for them, "an 85% reduction in token usage", and Claude Code turned it on automatically in January for setups whose tool descriptions would take more than 10% of the context. Cursor's version, loading MCP tool descriptions only when needed, cut total agent tokens by 46.9% in runs that called an MCP tool. Anthropic's code execution with MCP pattern, which presents tools as files the agent reads on demand, went from 150,000 tokens to 2,000 in its example, and the same post notes that "adding a SKILL.md file to these saved functions creates a structured skill." By August the protocol's maintainers conceded the point on their roadmap: with a hundred tools, "the model pays for that entire surface before the user has asked a single question."

More tools, worse choices#

Tokens are the visible cost. The less visible one is accuracy: the more tools a model can see, the more often it picks the wrong one. Anthropic says the most common failures "are wrong tool selection and incorrect parameters, especially when tools have similar names", and its docs say Claude's tool choice degrades past 30–50 tools. Three studies measured the gap between showing every tool and showing only the relevant ones.

The MCPVerse result has a twist worth keeping. Its authors note that agentic models like Claude-4-Sonnet did slightly better with a handful of relevant servers (62.4%) than with only the exact tools a task needed (62.3%), so a few extra tools don't hurt a strong model. Mounting all 65 servers, an action space of more than 140K tokens, did, and only Claude-4-Sonnet and Gemini-2.5-Pro could run that setting at all. IBM's LongFuncEval saw the same direction across five models: "a performance drop of 7% to 85% as the number of tools increases", from 49 tools to 741.

This is where skills have a structural edge. A skill doesn't add a tool to the menu; it teaches the agent to use tools it already has, usually a shell. The menu stays short however many skills are installed.

Head to head#

Only a handful of public evaluations put the two approaches on the same tasks, and none of them compares a skill alone with MCP alone on a real service with sign-in. Most compare MCP with a command-line tool, with or without a skill that explains it.

The cleanest cost comparison is Scalekit's, with the caveat that Scalekit sells agent authentication. It ran five read-only GitHub tasks through GitHub's MCP server, the gh command-line tool, and gh plus a skill that explains it. The MCP arm's median token use ranged from 4 to 32 times the CLI's, depending on the task, and all seven of its failures were TCP timeouts reaching GitHub's hosted server.

Mario Zechner's earlier test is the counterweight. When he wrapped one terminal tool both ways and ran it 120 times through Claude Code, both versions succeeded every time and MCP came out slightly cheaper and faster, which led him to conclude that "MCP vs CLI truly is a wash" and that the problem with many servers is design: "badly designed wrappers that dump unnecessary JSON everywhere." GitHub, for its part, moved the data fetching in its own agentic workflows from its MCP server to gh steps that run before the agent starts.

Arize's eval is the largest. Over 500 trials, correctness was a tie, and Claude with no guidance at all scored 0.845 because it already knows the GitHub CLI from training. The difference was on the hardest tier, where MCP made about 12 tool calls per task to the skills' 5, cost more than six times as much, and the agent gave up on the MCP tools often enough that their "tool fidelity" fell to 0.33. Arize's conclusion wasn't a winner: "MCP plus the command line."

Skills have their own failure mode, and it's quieter: the agent doesn't use them. Vercel found that "in 56% of eval cases, the skill was never invoked", so the skill added nothing by default; telling the agent to use it raised the pass rate from 53% to 79%, and an 8KB docs index the agent always sees scored 100%. Academic benchmarks agree that the skill itself matters: curated skills raised pass rates by about 16 points on SkillsBench, while on SWE-Skills-Bench "39 of 49 skills yield zero pass-rate improvement."

Adoption, aligned by age#

MCP is nearly 11 months older, so comparing the two on the same day flatters it. On 28 September 2026 MCP was 672 days old and Agent Skills 347. We lined both up by days since launch where the data allowed it.

The first signal is attention. Anthropic's skills repository, the reference collection most people installed first, picked up stars far faster than MCP's reference servers repository did at the same age.

The second signal is client support, and here the order flips once you look past the biggest names. Skills reached all five of OpenAI, Google, Microsoft, GitHub and Cursor faster, partly because MCP had already done the persuading and a client only needs a file loader to read a SKILL.md.

Past the big five, MCP's client list grew much longer. Its official clients page, which took community pull requests, listed 93 clients at day 343. The agentskills.io showcase, which Anthropic curates, listed 46 at day 347 and hadn't changed since August.

On every number shaped like usage, MCP is bigger, by a margin that depends on what's counted. Downloads are the clearest, with the caveat that they compare different things: the MCP SDK is a library that every server and many clients depend on, while the skills package is Vercel's installer, which counts every npx skills run.

The registry row needs its caveats. Vercel, which runs skills.sh, said on 25 September that the registry "grew to one million agent skills and recorded nearly 280 million installs" in seven months, and that 48% of skills were installed exactly once. For MCP we rebuilt the official registry's history from its public API: at least 37,060 server names still listed today had first been published by 28 September, a lower bound because deleted entries don't show. Agent Skills has no official registry at all.

Security and operations#

The two standards put the risk in different places. A skill has no permissions of its own: it runs inside the agent, with whatever the agent can reach. In Claude Code, Anthropic's docs say skills "have the same network access as any other program on the user's computer", warn that malicious skills "could lead to data exfiltration, unauthorized system access, or other security risks", and advise using skills "only from trusted sources". The risk is the supply chain and prompt injection, and the defence is vetting what you install.

MCP is a network service, so it has a network service's problems and tools. Remote servers can use OAuth, which Arize called MCP's real enterprise advantage: "OAuth lets administrators control who and what gets access to which resources. CLI auth usually does not." But authorization is optional in the spec, local servers read credentials from the environment, and the specification itself says that "MCP itself cannot enforce these security principles at the protocol level." Exposed servers, auth bugs and tool poisoning are the result.

Operations follow the same split. MCP servers are versioned packages or hosted endpoints, and a remote server can change its tools without the user doing anything, which Ronacher counted against it: servers "keep changing their tool descriptions at will". Skills have the opposite problem. The spec has no version field, updating means copying the folder again, and Anthropic's own surfaces don't sync them: a skill uploaded to claude.ai isn't available in the API or in Claude Code.

From debate to merge#

The argument ran for about a year, and it moved from "which one" to "both, together".

Google's framing, when it launched its own skills repository in April, captures where most vendors ended up: heavy MCP use causes "context bloat", and skills are the condensed expertise that fixes it. Not a replacement; a layer on top.

How skills travel over MCP#

SEP-2640 names the problem it solves plainly: "A server and the skill that teaches an agent to use it are versioned, discovered, and installed separately." Its fix adds no new primitive. Each file in a skill folder becomes an MCP resource, conventionally under a skill:// URI, and the skill format itself is left entirely to the Agent Skills spec. The proposal took 143 days from pull request to merge, on 13 September 2026.

Two details matter for anyone shipping a server. The digests are unsigned and come from the same server as the content, so the spec says plainly that "a match proves the two are consistent, not that either is trustworthy." And client support is still thin: Arcade, which sells MCP tooling, reports that OpenAI imports skills over MCP as snapshots taken when a plugin is submitted, so later updates on the server don't reach users. We found no OpenAI statement confirming it.

What this means if you build with AI#

The data doesn't pick a winner. It does point to a sensible default for each job.

1. Write a skill when the agent already has the tools. If the job is a procedure, house style, or how to use a CLI the agent can already run, a skill costs about 100 tokens until it's needed and adds nothing to the tool menu. That's OpenAI's rule too.
2. Build an MCP server when someone else's agent needs your service. Live data, actions on an account and per-user sign-in are what MCP is for, and OAuth is the advantage a CLI lacks. If you control both ends, a CLI plus a skill was cheaper in every cost comparison we found but one.
3. Ship both for anything non-trivial. The best published scores came from MCP plus a skill, and SEP-2640 now lets a server carry its own instructions.
4. Keep the tool menu short. Accuracy falls as tools pile up, past 30–50 tools by Anthropic's account. Expose a small default toolset, and use clients with tool search or deferred loading.
5. Write descriptions that trigger. A skill that never loads does nothing, which is what happened in 56% of Vercel's cases. Say in the description what the skill does and when to use it, and test that it fires. For knowledge the agent needs on every task, put it where it's always loaded.
6. Vet skills like software, and lock down servers like services. A skill runs with the agent's permissions, so install only from sources you trust and read the scripts. An MCP server is an endpoint: put authentication in front of it and pin the versions you depend on.
7. Measure your own setup. Every number above comes from someone else's tasks. Count the tokens your servers load and check whether your skills fire before you decide.

host0 uses both: agents deploy apps through the host0 skill and our REST API, and an MCP server manages the apps once they're live. It's the same split the vendors describe, at the scale of a small product.

Methodology#

This post draws on three research passes: context cost and evaluations, positions and security, and adoption. Each was limited to primary sources: specifications, vendor documentation and engineering posts, academic papers, and the evaluations' own write-ups. The post is dated 28 September 2026, and nothing cited postdates it. Before publishing we re-opened every single-source page behind a headline number and confirmed it still said what we quote. Two things changed on that pass: MCPVerse's abstract says Claude-4-Sonnet improved with more tools, which is true from its smallest to its middle setting, so we report all three; and Arize's eval had a fourth, no-guidance arm, which we added.

Our own numbers:

• Worked example: 11,000 tokens per MCP server is Anthropic's five-server total divided by five; 100 tokens per skill is the docs' budget; tool search is Anthropic's ~500-token search tool plus about 3,000 tokens for the tools it finds.
• Scalekit sums: we added Scalekit's five per-task medians for each approach.
• Stars: the star count on each repository page in Internet Archive snapshots, because the GitHub stargazers API returned errors for these repositories.
• Client lists: entries counted in each version of the official pages, from their git history.
• Downloads: npm's downloads API and the public ClickHouse PyPI dataset, cut at 28 September 2026.
• Registry: a full pull of every version in the official MCP Registry, taking each name's earliest publish date; deleted names are missing, so it's a lower bound.

Limitations:

• The evals are small and few. Five head-to-heads, two from companies with a stake, none with a skill-only arm on a real service with sign-in.
• The context model is a model. It uses averages and a docs budget, not a measurement of any one setup, and clients now defer MCP tools by default.
• Downloads are not users. The skills CLI and the MCP SDK are different kinds of package.
• Security numbers don't compare. Each study had its own sample, scanner and definition.
• Live docs are undated. Where we cite Anthropic's or OpenAI's current documentation, we say "the docs say".

Open questions#

1. Skill alone vs MCP alone. Nobody has published an eval of each on its own against a real API that needs sign-in.
2. One scanner, both ecosystems. No study has run the same security method over skills and MCP servers.
3. What deferred MCP loading really costs. Clients don't publish how many tokens a deferred tool's name and server instructions take.
4. How big a skill's metadata is in practice. The 100 tokens is a budget; nobody has published a measured average.
5. How often skills fire outside one eval. Vercel's 56% hasn't been replicated, and no vendor publishes trigger rates.
6. Who implements SEP-2640. Which clients list and load skills from MCP servers today, and does OpenAI's import track updates?
7. Comparable usage. OpenAI says 26.6% of weekly Codex users invoked a skill in June; no vendor publishes a matching share for MCP tool calls.