The half-life of an agent skill

Two of the most-starred agent-skill repositories on GitHub contain the material I'd bet against surviving the next model release. Both are excellent at what they do. The trouble is what much of that work does: it patches weaknesses in today's models, and model weaknesses keep disappearing.

Since Anthropic shipped Agent Skills, two camps have formed. One camp quotes Rich Sutton's Bitter Lesson and argues that hand-crafted scaffolding always loses to raw model capability. Nicolas Bustamante put it bluntly: LLMs eat scaffolding for breakfast. The RAG pipelines of 2023 and the prompt rituals of 2024 both got absorbed by later models, and this camp expects skills to go the same way. (I count at least four "prompt engineering is dead" posts this year alone.)

The other camp points at their own workflows and says the opposite. Their agents got better month over month on the same model, because the instructions improved. Martin Garramon published numbers from production work: 90% of his workflows run on skills, 10% on agents. Skills compound, he argues, so why would you delete a compounding asset?

There is already a useful distinction nearby. Daniel Miessler's Bitter Lesson Engineering separates the what from the how, and his follow-up on good and bad harness engineering contrasts prescriptive instructions with durable context about the user and the desired outcome. Nikola Balic's Disposable Scaffolding makes a similar cut between model workarounds and durable business knowledge. The underlying idea, then, isn't new. What I want to do is apply it narrowly, line by line, to the thing now filling agent repositories: skill files.

I think both camps are right, and they're arguing about different contents inside those skills.

Two kinds of skills

Open any skill file and you'll find one of two things.

Capability patches compensate for what the model can't do yet. Reasoning crutches like "investigate the root cause before proposing a fix." Checklists of known failure modes: "don't assume the API exists, check the docs first." These skills exist because the model, left alone, gets these things wrong.

Context carriers encode what the model can't know. Your team's deploy process. Your codebase's naming conventions. Which Slack channel gets the incident report. No frontier-model release can know a private, changing release checklist unless you provide it in context or put it into the training data.

The Bitter Lesson applies with full force to the first kind and barely at all to the second. A capability patch competes with the model's intelligence, and the model wins that race every time; each patched weakness is a free bug report for the next training run. A context carrier competes with nothing. It carries information, and a smarter model still can't guess information it has never seen.

That does not make context carriers immortal. Every skill has a maintenance clock. Capability patches age on the model labs' schedule. Organizational context ages when the organization changes. Tool mechanics and specification knowledge age when the platform owner changes the format or protocol. The useful question is not simply will this skill change? but what external event makes it stale?

Auditing popular skills

I ran a deliberately mixed sample of popular skills and skill categories through this lens. The results made me wince.

GitHub star counts checked August 13, 2026. They are snapshots, not stable properties.

The Karpathy behavioral skill is the clearest case. It went viral in January 2026 after Karpathy published his observations about LLM coding pitfalls, like models that assume an API into existence or declare victory before running the tests. The skill distills those observations into instructions, and it works. Thousands of developers swear by it.

But its core rules describe bugs in current models. A public list of embarrassing model habits is also a ready-made evaluation and training target for the labs. That makes the skill's popularity feel less like evidence of permanence and more like a countdown clock.

Superpowers gave me more trouble. (I went back and forth on where to place it; it resists the taxonomy more than anything else on the list.) Its debugging methodology, four phases with root-cause analysis before any fix, teaches the model to reason in a way smarter models may eventually do unprompted. That half rots. But Superpowers also specifies a shared process: tests fail before implementation, and review happens before completion. A prose skill can shape that behavior; it cannot guarantee it. Actual enforcement comes from tests, hooks, permissions, and CI gates outside the model. The durable part is the process contract, especially when the surrounding system makes the contract enforceable. You want ten engineers and their agents to follow the same release discipline, whatever the model's IQ.

Meanwhile the boring skills at the bottom of the table keep working on a different clock. Nobody writes think-pieces about the docx skill. It encodes the mechanics of OOXML manipulation, and those mechanics change when Microsoft changes the format or when the surrounding product absorbs the operation natively—not merely because the model became better at reasoning.

Look at the popularity column, though. The stars concentrate in the patches. That made sense once I thought about what stars measure: a skill goes viral when it fixes a pain thousands of people feel right now, and that pain is often a current model weakness. This is a hypothesis, not a result: popularity may be a negative signal for durability because today's sharpest pain attracts the most attention. Proving it would require tracking skill contents across model releases. But I would not treat a star chart as a longevity chart.

The gray zone

The taxonomy has messy edges, and I don't want to pretend otherwise.

Some skills look like context but are patches in disguise. A detailed code-review checklist feels like team knowledge until you read it closely: "check for N+1 queries" is something a stronger model does unprompted, while "we allow raw SQL in the analytics service, don't flag it" is real context. Most long skills interleave the two, and the patch lines quietly become dead weight.

Patches can also outlive their necessity for economic reasons. Even when a model could derive your preferred approach from scratch, it's cheaper to state the preference in a skill than to make the model re-derive it every session. Some patches will survive as compression long after they stop being correction.

Writing skills that survive

My test for any line in a skill file: does this tell the model how to think, what I want, or what an external system requires?

"Investigate before fixing" tells the model how to think. Expect to re-test it on the next release. "Deploys go out Tuesday and Thursday, never Friday" states what I want; it changes when the team changes the policy. "This API requires an idempotency key" describes an external contract; it changes when the API does.

Two habits follow. Re-audit your skill library every major model release, and count deletions as wins, since every line you delete may be a capability the model gained. And when you write a new skill, label the maintenance clock in your head: model capability, organizational context, or external specification. The first category should be treated as deliberately disposable. The other two are infrastructure, but infrastructure still needs an owner.

The moat

The Bitter Lesson never said scaffolding is worthless. It said scaffolding that competes with intelligence loses. Scaffolding that complements intelligence—your context, constraints, accumulated decisions, tool access, and external contracts—is a different substance. A model release cannot replicate private facts it was never given or authority it was never granted.

The labs will keep shipping smarter models, and your conventions will stay yours either way. I'd write skills with that in mind.

Sources: Rich Sutton, The Bitter Lesson · Daniel Miessler, Bitter Lesson Engineering and Good and Bad Harness Engineering · Nikola Balic, Disposable Scaffolding Over Durable Features · Nicolas Bustamante, LLMs Eat Scaffolding for Breakfast · Martin Garramon, Skills, Not Agents · Gao et al., From Registry to Repository · anthropics/skills · Karpathy-derived guidelines · obra/superpowers · ComposioHQ/awesome-claude-skills