PullMD Part 4: The Skill, and What It Actually Delivers

The service was running, and I could call it with curl. That wasn’t the point, though. The point was for Claude Code to call it, without me standing next to it every time and telling it to.

That’s what skills are for. And because I’m never quite sure with this sort of thing whether it really changes anything or whether I’m only imagining it, we measured it.

What a Skill Actually Does

A skill isn’t a function. It doesn’t provide a tool, and it can’t do anything the agent couldn’t already do anyway.

It describes behaviour. Roughly: if someone asks you to read a page, take PullMD and not WebFetch, the fetching tool built into Claude Code. Not for GitHub addresses, though, there you take gh, GitHub’s command-line tool. And if PullMD doesn’t answer, fall back to WebFetch instead of giving up.

That’s the difference between “the tool exists” and “the tool gets used in the right place”. A tool schema can’t express that at all, because it only describes what a call does, not when it’s the one to make.

Back then the skill was still called web-reader. The name the project runs under today didn’t exist yet.

Three Tasks, Run Twice

So the question was: can you tell the difference at all? Claude wrote down three tasks and ran each of them twice, once with the skill and once without. The answers still sit side by side in the project today, together with a short note on which route was taken each time.

The three tasks:

1. Read a Wikipedia article and summarise it in three sentences
2. Fetch the Hacker News front page and name the top five titles
3. Fetch an address that doesn’t exist at all

The third one was the actual test. The first two were meant to show whether anything happens in the normal case.

The Result Was More Awkward Than I’d Have Liked

With the summary there was practically no difference. Both versions are correct, both roughly the same length, both name John Gruber and the year 2004. Put the two paragraphs side by side and you see two usable answers.

Hacker News got more interesting. Without the skill, five titles came back; with the skill, the same five titles plus scores and comment counts:

1. **Motorola GrapheneOS devices will be bootloader unlockable/relockable**
   (773 points, 249 comments)
2. **RFC 9849. TLS Encrypted Client Hello** (93 points, 42 comments)

Not because the skill is smarter, but because through PullMD the whole page arrives as markdown and the model can take what it needs. With WebFetch, a model has already decided beforehand what’s important.

And with the address that doesn’t exist, the difference became obvious. Without the skill:

Error message: unable to verify the first certificate

That’s simply misleading. The domain doesn’t exist, and certificates have nothing to do with it.

With the skill, the answer said that the .invalid suffix is deliberately never resolvable under RFC 2606, along with the chain of attempts: PullMD first, then WebFetch, both failed, and why.

What We Took Away From It

The gain isn’t big. It’s structural. The token saving is a different calculation, namely PullMD against raw HTML; this was only about the skill against WebFetch. PullMD returns word for word what’s on the page, WebFetch only what’s left of it.

With a summary you don’t notice, because summarising is what happens anyway. With numbers you notice. And with errors you really notice, because the intermediate stage invents a plausible explanation where an implausible one would have been correct.

I’d have liked a clearer result. “Three times as good with the skill” would have been a nice number. That’s not how it was, and looking back I think it’s rather good that I wrote it down instead of just telling the story.

At the end of April, by the way, someone on Reddit explained the same thing to me from the other side, without knowing the skill at all. That’s in Part 9.

---

Part 4 of 15 in the PullMD series.

← PullMD Part 3: The API, and How Reddit Became Any URL | PullMD Part 5: Measuring the Wrong Engine →