The ecosystem standardised half my idea (the other half is the product)

Months ago I built a CLI that moved my AI agent's skill documents into a Go binary. The reasoning felt obvious at the time: an agent that preloads every instruction wastes its context window, so store the knowledge outside the conversation and let the agent query for what the task needs. I even gave it a retrieval command, discover --task "write a commit", and it would return the two or three documents worth reading.

Then I abandoned the project. Not because the idea was wrong, but because I had not yet learned how to properly lock the agent down. The LLM agent kept ignoring my guardrails and sometimes even switching them off (the repository still contains the folder of logged bypasses, a tidy little record of non-compliance). And while the project sat idle, something interesting happened: the ecosystem standardised half of it.

What the standard did

In December 2025 Anthropic published the Agent Skills specification, a markdown format called SKILL.md with a three-tier loading model: only name and description sit in context at startup, the full document loads when triggered, supporting files load on demand. Progressive disclosure, as a format. It spread fast. The standard's own client list now runs to roughly forty products, including Claude Code, OpenAI's Codex, Gemini CLI, VS Code, Cursor, and JetBrains' Junie. Marketplaces followed, and the public catalogues have since grown into the hundreds of thousands of skills.

That is the half they standardised: where skill text lives, and when it enters the context window. I had assumed that keeping instructions out of the conversation was mine to build, because when I started, no harness did it. Since Anthropic shipped Agent Skills in October 2025 and published the format as an open standard that December, every harness I use does it natively: name and description at startup, the body only when a skill triggers, bundled files only when something reads them. So I deleted the storage half of my design and kept the retrieval half, which is the half nobody standardised.

What the standard left out

The other half of my old design was retrieval: an explicit way to ask "which knowledge fits this task?" The standard does not ignore that question, exactly. Its three-tier model puts every skill's name and description in context and leaves the choosing to the model's judgement. What it does not define is a retrieval policy, and the numbers on how that judgement holds up at scale are already published.

• Selection is unreliable at scale. Anthropic documents the ceiling itself: the skill listing gets a character budget of about 1% of the model's context window, and when it overflows Claude Code drops descriptions starting with the skills you invoke least. Earlier it dropped whole skills, and users reported libraries of 63 skills surfacing only 42. Being visible is not the same as being chosen: in Vercel's own Next.js eval suite the agent never invoked the installed skill in 56% of eval cases when left to decide on its own, scoring no better than shipping no documentation at all. Skill retrieval is still an open research problem, underexplored enough that the benchmarks for it are themselves new, not a shipped feature. The vendors are quietly solving it one harness at a time: OpenAI's Codex changelog for 21 July 2026 lists fielded BM25, character n-gram, and multi-query lexical skill selection. That is a retrieval engine being bolted on privately, because the standard does not define one.
• The supply chain is ungoverned. A Snyk audit of the marketplaces scanned nearly 4,000 published skills and found 36% carrying security flaws, with active malicious payloads confirmed in the wild. No signing, no review, no sandboxing. A skill inherits every permission your agent has.
• Nothing observes any of it. No tool tells you which skills actually triggered, which were ignored, or whether the advice changed the outcome.

The specification defines the file format and deliberately stops there. That is a deliberate scope (small, adoptable, unopinionated). But it means governance and observability are nobody's job.

Trellis

So that is the direction of trellis, the system I abandoned and have now revived: follow every standard the ecosystem converged on (SKILL.md, MCP, lifecycle hooks), and be the layer the standards deliberately left out to stay small.

Concretely, in the current build:

• Enforcement that binds mechanically. Lifecycle hooks inject session context at start, gate dangerous shell commands, and capture unfinished work at end. No prompting, no "please use the tool".
• Skills backed by guards. A trellis skill can declare its enforcement, and the matching guard catches the agent that ignored the advice. The developer being governed by his own product is the point, and it keeps happening: this month trellis rejected my commits for working outside a governed session, for missing decision records, and for trying to ship after my own configured curfew.
• One spine across hosts. The standard defines the format but not the installation path (Claude Code, Codex, and Gemini all read different directories). Trellis keeps one canonical source and projects it into whichever host you work in, with hashes to catch drift.
• A value ledger that answers in numbers. Every checkpoint, every blocked command, every resumed session is recorded, so the system can answer the question that matters: what did it remember, what did it protect, what did it teach? Those began as promises. They are turning into actual, empirical measurements.
• Observability in the standard's own dialect. Trellis exports that ledger as OpenTelemetry spans in the GenAI semantic convention, so what it remembers, blocks, and teaches lands in Datadog or Honeycomb beside the rest of what you already watch. The half of my original complaint that said nothing observes any of this now has a reply, and it speaks the ecosystem's language.

What this buys your business

Everything above sounds like plumbing, so let me translate it into the terms the work actually gets paid in.

If you run a business where AI agents now write real code (and if your engineers use Claude Code, Codex, or Cursor, they already do), you have quietly acquired a new class of employee: fast, tireless, and unable to remember yesterday. You pay for that amnesia twice. Once in tokens, as the agent rediscovers context it already had, and once in your calendar, as half-finished work gets re-explained instead of resumed. A session that picks up exactly where the last one stopped is not only a developer comfort, it is also rework taken off your invoice.

The protection side is insurance you can price. A blocked force-push or a gated shell command is normally an event that would cost you a billed production incident, so trellis just made it an event that never happened. And as a result, no restore from backup, no weekend recovery, no client call that starts with an apology. And because every block, every bypass, and every resumed session lands on one record (exported in a format your ops tooling already reads), "what did the AI actually do to our systems?" stops being a shrug and becomes a report. In sectors where operational clarity and compliance directly affect revenue, which is precisely where FireStack builds, that report differentiates between adopting your agents confidently or hoping nothing breaks.

There is a management lesson hiding in the origin story too. I abandoned the first version of this tool because its guardrails cried wolf and I kept switching them off. That is the fate of every control that alarms falsely, in software and everywhere else. It is why the false-positive rate below is the number I watch most: a guard nobody mutes is the only guard that was ever really there.

What self-governance produced

Numbers from the tool governing its own repository:

• The ledger audited its own gate and kept indicting it. The first audit found a 50% false-positive rate: half the "dangerous commands blocked" were innocent, strings like a force-push mentioned inside a quoted test payload, never executed. Rebuilding the matcher pulled the rate to 27%. Weekly classification of every denial has since walked it down to 12%, against a target of 10% that it still misses. That 12% is computed over reviewed denials only, and thirteen of thirty-seven blocked commands are still unclassified, so the number can move in either direction as the backlog clears. A figure I quote about my own product should carry its own asterisk. Both original miscarriages are permanent regression tests now. A trust instrument that can indict itself, publish the number, and show the trend is worth more than one that only ever reports a pass.
• The gate blocked its own repair. The patch that fixed the substring bug contained the banned pattern in a code comment, so the old matcher rejected the commit that would retire it (you could not script better dogfood).
• The docs guard's maiden run surfaced 22 dormant proposed decision records. Eighteen were demonstrably shipped, sitting in implemented-but-never-accepted limbo, one had been quietly superseded, and three were genuine gaps. Roughly half an architecture record had drifted from reality. It is worth being exact about what did the work there: the guard is what made anyone look, and the search that confirmed each case was ordinary text matching, the same reflex any coding agent reaches for unprompted. That is fast, and it is not exhaustive. Telling the agent which mechanism a class of problem actually warrants is a gap in the design, not a feature of it.
• Session residue went from 13 stray files to one. Telemetry streams moved to a dedicated branch (hash-chained, with heads anchored in main), and boundary commits at each session edge sweep the rest. The working tree stays clean without a single byte of evidence going unversioned.
• Onboarding collapsed to one command, and offboarding exists. Leaving prints exactly what memory you would lose (checkpoints, unfinished work, the guard history) before deleting anything. Easy exit is a feature: a governance layer that is hard to leave reads as lock-in.

The instrument answers the hard question, and the answer is still no

A ledger that counts is only half of it. The half I cared about was whether the count could drive a decision, so I wired a readiness gate: thresholds over the recorded outcomes, and one command that reads them and returns yes or no. I pointed it at the question most founders duck. Am I ready to sell this?

Today it runs seven checks and four pass. Ninety-nine sessions wrapped cleanly, the must-have capabilities all resolve, skills trigger often enough to matter. Three checks hold the line: the false-positive rate sits at 12% against a 10% bar, three field bugs rated High are still open against a bar of zero that cannot be waived, and only 72% of interrupted work gets picked back up, thirteen of eighteen saved notes, against a bar of 80%. A product that can tell its own founder "not yet", with the receipts underneath, is the instrument I was actually building. I am taking it at its word: no client sees this until those three turn green.

Release week is where the gate earned its keep

The repository went eight days without a release at one point, and the reason turned out to be the thesis of this whole piece. The security gate (no green scan, no tag) was failing, and when I finally sat down with it, the failures were not code regressions. They were my infrastructure misreporting its own state.

First: the previous version had only ever released because a leftover build directory happened to survive on my machine. From a clean checkout, on any other computer, the gate could never pass. Nobody but me, on this laptop, could ever have shipped the product. Any founder reading this knows the business name for it: a bus factor of one, discovered by accident, fixed for the price of a one-line change. Cheap now, existential later.

Then, the night before the next cut, the security scanner caught a real vulnerability in a dependency (an infinite-loop CVE in a text-processing library) and blocked the release until the upgrade landed. The gate doing precisely its job.

And the morning after, the same gate failed on my machine and passed on the build environment, for the dumbest of reasons: a security tool lived in a directory that one shell had on its PATH and the other did not. My successful run was environmental luck; the failing run was the reproducible result, the one every future tester's machine would see. The fix makes the gate fetch the tool itself, and it was verified against a control: the old build under a stripped environment reproduces the failure, the new build under the same environment scans and passes.

A gate that fails the same way on every machine is worth more than one that passes on mine. Most of what I have learned building trellis compresses into that sentence.

Records that cannot be quietly edited

The newest layer came from watching my own escape hatches. Trellis keeps ledgers that feed decisions: which denials were false alarms, which field bugs are still open, which of its own warnings I dismissed. This month, every one of those trails stopped accepting a bare "resolved" mark. Closing an entry now demands evidence: the commit that fixed it, the decision record that supersedes it, or a written reason for dismissal. A registry guards the pattern itself, so the next record cannot ship without the same evidence requirement, and a gate will refuse to read from an unregistered one.

The same week, my configured curfew blocked a late-night commit, and every override that followed went onto the audit trail with a written reason (the escape hatch exists deliberately, but it signs its name). Within hours the command gate had been tightened to parse exactly the environment-variable prefix those overrides used, so even the workaround now leaves a full fingerprint.

If you have ever bought software, you know why this section exists. Every vendor claims their numbers; almost none hand you the instrument that produced them. Trellis's answer to due diligence is one command a prospect can run themselves, against ledgers the tool cannot quietly edit, and the command is currently willing to say "not yet" about its own maker's launch plans. I can think of no cheaper way to prove the reports mean something. Watching the system close a seam behind me, hours after I walked through it, is the strangest and most convincing feedback loop I have shipped yet.

The first field failure made the argument

Then trellis let me down in the wild, and the miss argues for the direction better than the wins do.

I onboarded a second repository, got a cheerful "Configuration ready!", and started prompting. Later I asked the agent there, point blank, how it had used trellis. It answered: not at all, no proof exists, and the local decision records say the system is parked. Every word was true. The init command had hit a legacy code path, seen a config file left over from the abandoned era, declared success, and quietly skipped the entire binding stage. No hooks, no skills, no memory spine. The human ran the command; the machinery never engaged; a stale document then backed the agent up.

Two things about that episode convinced me more than any feature demo. First, an agent reporting its own state plus an audit trail made the defect provable in minutes: one terminal transcript, one code path, case closed. Second, a false "ready" from a setup tool is the same disease as an assistant claiming compliance it does not have, and a governance product must hold itself to the standard it enforces. The fix shipped with a regression test that reproduces my exact terminal session.

Where this stands

Twenty-four tagged releases now carry the project, and the latest one crossed a small line: the version number is finally decided by a written policy instead of a library default, because a feature-heavy release that bumps only the patch digit is a version quietly understating what changed. Even the numbering is under governance now.

The first two people I demoed this to both named the same pain, unprompted, before I pitched anything: the agent drifts off context and forgets the goal mid-task. Neither of them led with security, and neither cared much about my memory pitch until they saw a session resume where it left off. Two conversations prove nothing, and the gate will not let me pretend otherwise. But they are the first outside voices on a record that has only ever heard mine, and I was very happy trellis opened them up to name their daily pain points - memory and context drift.

Where this goes

One rule shapes everything below: the roadmap itself is governed. Feature work queues behind discovery conversations, and the validation gate for re-investment is built so that my own usage can never turn it green; only outside people can. So what follows is not a promise list. It is the set of bets currently winning the argument, each one held to the same evidence bar as everything above.

The governed loop. The next engineering milestone is an agent that carries a real task across many iterations with nobody watching, where the whole run is auditable afterwards. The design is deliberately boring: a reconciliation controller, something that people have trusted for years. Declare the desired state, observe the actual, diff, act, converge. That method buys the three properties an unattended run must have: it always halts, it survives interruption (the state lives in the world, not in the loop), and every iteration lands on the trail as evidence. Autonomy is arriving across the industry either way. The bet is that unattended agent work only becomes sellable when the receipts come built in, and a run you can replay beats a run you can't reproduce.

Governed or drifted, said out loud. This week I caught a session in my own dashboard running under an arbitrary label beside all the deterministic ones. An agent had slipped outside the governed lifecycle, and while the commit guard caught it at the boundary, nothing said so ambiently; the drift was only visible because I spotted it. So the dashboard learns to state it plainly: this session is governed, that one has drifted, and here is the sentence to paste that brings the agent back. The same detection doubles as something I want even more: transparent adoption telemetry. Which agents run governed, which quietly ignore the layer, which humans let them. A dashboard willing to display "this session is NOT governed" is the trust pitch demonstrating itself, and the usage patterns it reveals will steer the roadmap better than my assumptions ever have.

One spine across a fragmenting harness market. Claude Code, Codex, Gemini, and the rest read different directories, bind different hooks, and ship different memory conventions, and the churn is not slowing. The direction is for the control plane to be the portable constant: your memory, your guards, your ledgers travel with you, and the host becomes interchangeable. What makes that real is already a decision record: configured is not the same as live, so every binding gets observed.

The classes, mechanised. The most valuable thing this project has produced is not a feature. It is a small taxonomy of ways autonomous agents can exhibit various classes of hallucination in increasingly nuanced ways: checks that did not run but look like clean results, records that drift from the state they feed decisions with, gates that pass by environmental luck. Each class, once named, becomes a guard: the vacuous-pass tripwire and the evidence-bound ledgers exist today, and the remaining classes are queued behind them. Follow that thread to its end and the category becomes clear. Trellis is not really in the business of blocking risky commands. It is in the business of making every green mean something, for work a human did not watch happen. Evidence integrity, as a product.

And the pull gets the last word. The unprompted asks from the first two conversations are already filed: mark paths permanently off-limits to the agent, force-stop a run mid-task, run headless in CI. Any of them could be the next bet, and none of them jumps the queue on the strength of one voice. More than a few conversations with early adopters are likely to provide a lot of unexpected and surprising requests, which will guide and shape the roadmap.

The standards gave every tool the same file format. What nobody ships yet is agent work that arrives with receipts. That is the lane I currently see, and it is wide open.

So the accurate version is less flattering than being early. I built the half that was about to become free, and I came within one revision of deleting the half that turned out to be the product. Storage and loading are handled now, by everyone, for nothing. Retrieval, governance and evidence are still nobody's job. That gap is the entire bet.

So here is where I leave it, depending on who you are.

If you are an engineer running coding agents: trellis goes into closed alpha when its own gate turns green, and I am in the middle of the first eight discovery conversations right now. Thirty minutes, a live demo against your actual workflow, and your unfiltered reaction, which is worth more to me than praise. Write me: hello@firestack.eu.

If you run a business where AI is quietly writing production code, and it already is if your team uses these tools at all: the question that matters is no longer whether agents do real work. It is what you can verify about the work they actually did, what gaps remain, and what the evidence trail looks like after the fact, to a client, an auditor, or yourself. That proof is the product here, and the same evidence-first approach is how FireStack builds client systems. If your operations are missing that kind of certainty, we'd like to hear from you.

And whoever you are, the next time a tool, a vendor, or an agent tells you everything went fine, try the one question this whole project taught me to ask: can you show me the receipts?

---

Figures read 23 July 2026, 16:49 to 17:05 CEST. A dated snapshot, not a live dashboard.

Jojoo Imbeah is the founder of FireStack, where he builds production backend, data and AI systems and takes on fractional engineering work. Trellis is FireStack's control plane for AI coding agents.