Shipped, but Never Switched On

TL;DR

The most common disease in my own repo is not the bug. It is the feature that is green and never fires. A feature without one event per activation is a claim, not a capability. In the more uncomfortable variant it is not even decidable whether it runs: the code exists, an event for it does not. Two cases from the past weeks show the pattern. A fast path built precisely for the most frequent session type, whose activation appears nowhere in 258 of roughly 265 runs. And a second machine that has been standing by for weeks and is practically never used. Plus four grep commands that will find your own case in a couple of minutes.

The Case That Would Not Answer the Question

I wanted to know something harmless: how many of my sessions run waves at all. The answer is one line:

tail -30 .orchestrator/metrics/sessions.jsonl \
  | jq -s '{n:length, zero_wave_sessions: ([.[]|select(((.waves//[])|length)==0)]|length)}'
# {"n":30,"zero_wave_sessions":22}

22 of 30 sessions run zero waves. All 22 are housekeeping runs, and all seven deep sessions in the same window did run waves. So the picture is not broken, it is just different from the blueprint in my head: the default case is the short session, not the big one.

One detail of that command matters more than the result. waves is an array, not a counter. A filter on .waves == 0 silently measures nothing at all, it returns zero hits and looks like a clean answer while doing so. My first attempt was exactly that.

For this default case a fast path has existed for months: a shortcut that routes short housekeeping runs past the heavy planning machinery. It is built, it is tested, it is described in the skill text. Whether it took effect in those 22 sessions, I could not determine. Not hard, but impossible: in hindsight there is no trace from which it could be reconstructed. That is the systemic disease of this repo in its observability variant, and the name for it even carries a date: shipped but never switched on, six cases in a single day on 2026-08-08.

Four Questions Every Feature Has to Answer

A recipe came out of that case. Four questions, four commands, each with the result I measured:

# 1. Does the configuration know the key at all?
grep -n "express-path" CLAUDE.md
# no matches

# 2. Does the parser know it?
node scripts/parse-config.mjs | jq '."express-path"'
# null

# 3. Is there exactly one event per activation?
grep -c "express" .orchestrator/metrics/events.jsonl
# 0

# 4. In how many runs is the activation logged, in how many field shapes?
grep -o '"express_path":[^,}]*' .orchestrator/metrics/sessions.jsonl | sort | uniq -c
# 14 false, 6 true, 1 deviating shape, 258 runs without the field

Four answers, one finding. The skill text claims the fast path defaults to on. The parser does not know the key. There is no event. And where a field exists at all, it exists in two different shapes. Two field shapes are not a history, they are two halves of one.

The fourth question is the most important one, because it is the only one you cannot argue away. A feature without an event is a claim, not a capability. Verification beats trust, and that holds for your own infrastructure just as much as for generated code, as I described in The Scarce Skill Is Not Typing, It Is Verification.

When the Parser Has No Allowlist, Every Silence Is Permitted

The cause sits one layer down and is thoroughly unspectacular. The config parser is 149 lines with no key allowlist. A missing key is therefore never an error, always a silent off. Mistype one and you switch off a subsystem without a single word about it. Never write one down at all, because the documentation lists it as a default, and the same thing happens.

When I counted this through once: twelve subsystems sat at enabled: false, among them the docs orchestration, several staleness checks, the wave reviewers and the reconcile run. Another 42 documented top-level keys were missing from my project file entirely and fell back to their defaults in silence. That is not sabotage, that is erosion: every single key was consciously not set at some point, and in aggregate a different tool was running than the one I believed I was using.

The after number is the sobering part. Once a real event was wired in, 36 evaluations were available across 7 repos. The fast path activated in 2 of them, which is 5.6 percent. The verdict of my own evaluation is accordingly: unfalsifiable. Making a feature measurable does not mean it looks good afterwards.

The same pattern sits in neighbouring instruments of the same fleet, measured across 18 repos, 143,967 parsable events and 1,692 distinct session IDs. A stop event whose duration is literally 0 in 8,127 of 8,127 cases. An end event whose reason reads plainly "other" in 96.3 percent of records. A lock with 1,050 acquisitions against 95 releases, of which only 11.3 percent carry any resolution event at all. All three findings are the same thing: layers that cost context and maintenance and whose effect nobody can measure.

The Second Half: a Machine on Standby That Went Unused

The second case is more uncomfortable, because here no parser is to blame, I am. Since 2026-08-27 I can offload heavy waves to a second Mac, over an SSH alias, with a dedicated skill for it. Set up, documented, working. My own count of remote commands actually executed, as of 2026-09-02:

• 08-27: 129 commands with 1 skill load, that was the setup day
• 08-28: 31 commands with 7 loads
• 08-29: 16 commands with 1 load
• 08-30 to 09-01: 0 commands with 2 loads
• 09-02 before the rework: 3 commands with 5 loads

So since 08-30 the skill was loaded five times and applied zero times. It was read, the host was recognised as reachable, and then the work ran locally anyway. The load picture did not help: on the work machine 24 Claude processes at 47 percent free memory pressure, on the target machine 9 to 11 processes, 95 percent free and a load around 2. Without a bracket for process priorities the one-minute load on the work machine hit 98 across 18 cores during a single quality gate.

The sessions read "reachable" and planned locally anyway.

Four Causes, None of Them Laziness

When I looked into the non-use, four reasons came out, and none of them reads "could not be bothered".

First, the manual labour. One offload was ten steps: check reachability, prepare the worktree, build the prompt, start the run, fetch the patch, review the patch, apply it, clean up, record the result, handle the failure case. Ten steps lose every race against starting an agent locally.

Second, broken authentication for headless operation. The credentials lived in the macOS keychain, which is locked for SSH sessions. The file-based access that would have covered exactly this case had been deleted on the same day during a cleanup.

Third, no mechanical trigger. The gate that shrinks a wave under resource pressure knew of no second host at all. So it could never propose what was standing by anyway. That rule is now inverted: the gate never probes the network itself, but every decision writes an event, on every attempt and on every refusal. Without a witness the decision stays local. The payload carries the configured alias, never a hostname and never an address.

Fourth, the leftovers. An earlier offload attempt ran inside the main checkout instead of a dedicated worktree and left 67 modified files and a sync branch behind. Once you have cleaned that up, you reach for it more hesitantly next time.

And now the part I cannot talk my way around: I have no benchmark. There is no before and no after, no measured time saved and no measured RAM saved by offloading. The payoff of this exercise is not a gain, it is the audit itself. I now know that a fully set-up tool can sit unused for weeks while I mention it in conversation as an available capability. That is the finding. Anything beyond it would be invented.

The consequence is now part of the skill too. Exit codes from 0 to 8, and an empty patch counts explicitly as a failure, not as a success. Five roles never travel to another machine at all: core implementation, security review, migrations, releases and anything touching secrets. And whoever does not read the gate log or the patch has no result: completed and correct are not the same claim.

What I Take Away From This

All four grep commands come down to the same question: is exactly one event written per activation? If it is not, the question of the feature's effect is not hard, it is unanswerable, and no amount of reading the code closes that gap.

Two detail findings explain how something like this stays unnoticed for months. 149 lines of parser with no key check switched off twelve subsystems without a single word about it, because a typo there quietly turns into an off. And two field shapes are not a history: 14 times false, 6 times true, one deviating shape and 258 runs without the field add up to no evaluation at all, just two halves.

The second case, the unused machine, has a different cause and the same shape. Where no mechanical trigger sits, time pressure decides, and time pressure always picks the local route, no matter how well the other tool is documented. That is why this non-use audit is cheaper than the next feature: four grep commands against several days of build time for something nobody switches on afterwards either.

Conclusion

These days I count the activations first and the lines second. 73 percent of my sessions are housekeeping, 21 repos on this machine carry the orchestrator folder, and in almost all of them there is more capability in the prose than in the data. A check that finds unwired configuration keys already existed before this. It just does not find unobservable phases, and those are exactly the problem.

If you are building a larger agent setup yourself: take the four questions above and ask them of your favourite feature. What the surrounding procedure looks like at my end is in How session-orchestrator Matured in 9 Days and in Building an Agent Fleet: Sven, BuilderBob and the Loop. How I work is under way of working, and if you have a similar construction site, get in touch.