Your AI skills are unsigned code

Your AI Skills Are Unsigned Code

2026-06-28

Natural language is ambiguous by structure. A skill written in plain English can never be formally verified the way code can.

The load-bearing claim in Goju’s walkthrough

A new report dropped this month and it should have stopped every “install this skill” conversation cold. Researchers built malicious agent skills, marketed them through Instagram ads like any other growth play, and hijacked roughly 26,000 AI agents. The skills went through the marketplace gauntlet. Three separate scanners (Cisco, NVIDIA, and skills.sh) looked at the package and waved it through.

Goju (Justin Gottschlich) walked through the mechanics, and the mechanics are the whole story. This wasn’t a clever zero-day. It was a structural gap that sits underneath every skill marketplace that exists today. The scary part isn’t that the scanners missed it. It’s that they couldn’t have caught it.

The trick: review the files, swap the payload

Here’s how it works. A skill gets submitted for review. The files in the package are clean: there’s nothing in the reviewed bytes that does anything malicious. The scanners read those files, find nothing, and approve.

But the skill calls an external URL. And the behavior lives behind that URL, not in the reviewed files. After approval, the operator quietly swaps what that URL returns. The reviewed files never change. The scan result is still “safe.” But the payload the agent actually executes is now whatever the attacker wants it to be.

The thing you reviewed and the thing that runs are two different objects. The review checks one. The agent executes the other.

This is an old idea wearing new clothes. Security people call it a time-of-check / time-of-use gap. You verify something at one moment, you use it at a later moment, and nothing guarantees they’re the same. We solved a version of this for software decades ago with code signing, dependency pinning, and hash-locked artifacts. The agent ecosystem shipped marketplaces before it shipped any of that.

And even the static files can’t be verified

Now here’s Goju’s deeper point, and it’s the one that matters for where this is all going. Set aside the external-URL swap for a second. Pretend the skill is fully self-contained, no remote calls, everything in the reviewed files. It still can’t be formally verified the way code can.

Because a natural-language skill is just instructions in English. And English is ambiguous by structure. That’s not a flaw you can lint away. It’s what natural language is. The same expressiveness that makes prose easy to write makes it impossible to check.

Code has a grammar, types, and a defined execution semantics. You can prove things about it. You can run a static analyzer and get a real answer. A paragraph of English instructions has none of that. Two skills can use the same word and mean different things. One skill can use a word one way in line two and another way in line nine, and no tool will flag it, because there’s no canonical definition to check against.

So when a marketplace “reviews” a natural-language skill, a human is reading it. The model that runs it is resolving meaning from it. Those are different operations. They can produce different outcomes. The review is an opinion, not a proof.

I hit the same wall from the other side

I’ve been living in a quieter version of this exact problem. Not security. Quality. I was building domain-driven design doctrine for an agent system: a set of architecture rules that AI coding agents are supposed to follow. The docs were good. Well-written, thorough. And completely inert.

When I spawned agents to write code, they didn’t load the doctrine, didn’t follow it, and had no way to know when they’d broken it. So I built a hook system: a single resolver script that injects the right doctrine fragment into each agent’s context window before its first prompt. I proved it works by injecting a sentinel string and watching the agent echo it back verbatim. The delivery gap closed.

But the meaning gap didn’t. The thing I’m injecting is still English. The agent still reads it, interprets it, and acts on its interpretation. If two doctrine files use a term differently, the agent can’t surface the conflict; it just silently picks one. The exact same disease Goju is describing, minus the malice. Natural language drifts, contradicts itself, and fails silently, and there’s no compiler to catch any of it.

A malicious skill and a sloppy skill fail for the same reason. There’s no contract to check them against.

The fix is to stop treating skills as documents

If the problem is that natural language can’t be verified, the answer isn’t better natural language. It’s to compile it into something that can.

That’s the shape of a design I’ve been working through: a formal skill compiler. It’s in the planning stage, not built, so I’ll be honest about what’s real and what’s intended. But the architecture is the right answer to the problem above, so it’s worth laying out.

The idea is to treat a skill like source code. The English you write is the PlainSkill: authoring input, easy to write, ambiguous. The compiler converts it into a formalSkill: a machine-checkable artifact. After compilation, the formalSkill is the source of truth. The original prose becomes historical evidence, not the executable contract.

// the compiler treats a skill like a program
PlainSkill text
  → clauses                                  // atomic instructions, each with a source span
  → controlled vocabulary                     // one canonical meaning per term
  → typed predicates                          // boolean or scored statements
  → obligations / prohibitions / transitions  // the actual rules
  → formalSkill                               // the verifiable artifact
  → compiled runtime prompt                   // what the model actually runs

What compiling a skill actually looks like

The interesting move is what happens to the words. Every important phrase gets pulled out, given a single canonical definition, and bound to a typed rule that traces back to the exact span of text that created it. Take a real clause from a content-extraction skill: “Always include ‘Quotes That Hit Different’ if the source has quotable moments.”

What does “hits different” mean? What counts as a “quotable moment”? Is “Quotes That Hit Different” a section, a category, or a quality bar? The author knows. The next reader guesses. The model improvises. The compiler forces the question and writes down one answer:

// surface phrase → canonical symbols
"Quotes That Hit Different" →
  QuotesThatHitDifferentSection   // a section entity
  Quote                            // a content entity
  HitsDifferent                    // a judgment predicate

// each symbol gets exactly one definition
Quote        = "an exact contiguous span from the source, not a paraphrase"
HitsDifferent = judgment predicate, rubric: hits-different.json, passing_score: 4

// and the clause becomes a rule that traces back to its source
HasQuotableMoments(source) ⇒ SectionExists(output, QuotesThatHitDifferentSection)
item ∈ QuotesThatHitDifferentSection ⇒ IsQuote(item, source) and HitsDifferent(item) ≥ 4

Now “hits different” has a definition, a rubric, and a passing score. The vague word “include” became two hard rules: the section must exist when the condition holds, and only quotes above the bar can go in it. And every rule carries a pointer back to the sentence that produced it. If a rule is wrong, you fix the English and recompile. You never hand-edit the compiled artifact.

Why this kills both attacks

Two things fall out of compiling skills that you can’t get any other way.

• The ambiguity problem goes away by construction. A skill that can’t be compiled to typed rules (because a core term is undefined, or a clause means three contradictory things) doesn’t compile clean. It throws an unresolved-definition issue. The ambiguity becomes a build error instead of a runtime surprise. Writing malicious intent that survives compilation to typed, traceable rules, passes its rubrics, and round-trips cleanly is a dramatically harder problem than writing a benign-sounding paragraph.
• The swap problem gets a place to live. A compiled artifact has a manifest with a source hash and is immutable by version. External references become something you can require to be pinned and hashed, not fetched live and trusted. None of that is possible when the unit of distribution is a folder of prose. You can’t hash a meaning. You can hash an IR.

I want to be careful here: compilation isn’t a magic shield. A determined adversary will find ways to smuggle intent through any formal system, and the external-URL problem ultimately needs runtime enforcement too. But it raises the floor enormously. Right now the floor is “a human skimmed it.” That’s not a floor.

The mental model: code signing for skills

Step back and the whole thing snaps into focus. We have unsigned code running in production at scale. The security review is a person reading the source. We’d never accept that for software: we have signatures, reproducible builds, hash-locked dependencies, static analysis. The agent world skipped all of it and went straight to marketplaces.

Goju’s demonstration is the proof that the gap is already being exploited. 26,000 agents is not a research footnote. The formal skill compiler (or something like it) is the missing infrastructure. Not a governance process. A compiler pass.

Skills are source code. The marketplaces are shipping them as if they’re signed and audited, because the tooling that would make signing and auditing possible hasn’t been built yet. The sooner that catches up, the better the odds that “AI takeover” stays a demonstration and doesn’t become the default.

Mechanics and the 26,000-agent figure are from Goju’s (Justin Gottschlich) walkthrough of the fake-skill agent-hijacking report: skills marketed via Instagram ads and passed clean by Cisco, NVIDIA, and skills.sh scanners. The formal skill compiler (PlainSkill → formalSkill, symbol transformation, observability scoring, canonical-IR round-trip testing) is a design specification in the planning stage, not a shipped product. The doctrine-injection hook system is built and tested. I’ve flagged which is which throughout.