Part IV — Leveling Up — Home for Deranged Scientists

§13 Parallel agents & worktrees

Parallelism pays off exactly once: when the tasks are genuinely independent and the overhead of coordination is less than the time the fan-out saves. Everything else — racing on shared state, fan-out as a way to avoid thinking about dependencies, worktrees that quietly accumulate forgotten work — costs more than it earns. This chapter is about knowing the difference before you commit to the shape.

Rule: Fan out only when tasks are independent.

Why: Parallel agents on dependent tasks produce merge conflicts, races, and subtle interleavings you'll spend longer debugging than the parallelism ever saved you. Two agents that both need to edit services/timer_service.py will produce a conflict. Two agents that need each other's types but don't wait for them will make different assumptions and ship two incompatible halves of a feature. How to apply: If task B needs task A's output, they are sequential. If they touch different files with no shared state, they are parallel. The test is honest: can you describe the handoff between B and A in one sentence? If yes, they are sequential even if the files don't overlap. "B uses the interface A defines" is a handoff. You can make that interface explicit — agree on it in the plan, stub it, and run in parallel — but you have to do that work first, and it counts as a dependency.

The heuristic that holds across every release is grouping by file overlap, not by issue priority. Two high-priority issues that both touch the same test file are not a parallel pair — they are a serialization point. Parallel Work — told in full in Part V — is the textbook clean case: two agents, two languages, different runners, zero shared state, both green on first run because the file map guaranteed the conflict couldn't happen. A later release ran 14 subagents across 29 files with zero conflicts, for the same reason: the plan mapped file ownership before any agent started work. The file map is the contract. Write it first. Pin: group by file overlap, not by issue priority.

The Interface That Never Needed a Sixth Method (Part V, case study 2) is the strongest evidence for interface-first parallelism the playbook has: a five-method adapter contract absorbed 70+ adapters across nine releases without ever changing, because the interface itself refused to permit conflicts. Four agents working in parallel against a tight contract can't collide — not by discipline, by construction. Pin: the interface you design before the parallel work begins is the coordination protocol the parallel work runs on.

The Wave Pattern (Part V, case study 2) is the rhythm that made seven-agents-per-release safe: schedule trivial items first as a load test for the build/config/test harness, then moderates, then complex. The cheap wave shakes out every infrastructure surprise before the expensive agents commit to 150-line state machines — software's version of checking the parachute before jumping.

Rule: Worktrees are workspaces, not stashes — and they are not a coordination strategy.

Why: A worktree is an isolated place to do work. It is not a place to park something "for later." Worktrees rot. Uncommitted experiments accumulate in long-lived worktrees the same way food accumulates on a desk — slowly, invisibly, until what's there is unrecoverable. The cost is not the disk space; it is the work itself: an afternoon of exploration with no commit, no branch, no artifact — just a worktree that got stale and got deleted. And a second failure mode the original rule missed: worktrees look like they isolate parallel agents, but the git state underneath them is shared. Worktrees isolate the filesystem; they do not isolate branches, refs, HEAD, or merge state. Five agents in five worktrees on five feature branches are all operating against the same underlying git database, and without an explicit coordination layer above that, they will step on each other's branches, cross-contaminate commits, and produce orphaned refs that have to be recovered by hand. How to apply: Every worktree should have a definite end — merged, deleted, or explicitly reopened — within days. If you can't immediately name what the worktree is for, it is time to close it. Clean exit means committed (even as a draft branch), merged, or explicitly discarded — not suspended. "I'll come back to this" is how worktrees die. For parallel-agent work specifically: do not reach for worktree-per-agent as the isolation mechanism. Use file-boundary parallelism inside a single worktree, with a team or dispatch layer that assigns non-overlapping files to each agent. The third case study learned this the hard way (see The v6 Worktree Experiment below) and pivoted in the next release to a team-in-one-worktree approach with file-level ownership — same agents, same discipline, dramatically different outcome. Pin: isolate context with worktrees; isolate work with file boundaries. Never rely on worktrees to isolate git state across agents.

The v6 Worktree Experiment and The v7 Team Pivot (both Part V, third case study) are the cleanest paired evidence for this rule the playbook has. Same project, back-to-back releases, same kind of parallel work — worktree-per-agent in v6 produced branch confusion and orphaned refs that had to be recovered by hand; team-in-one-worktree with file-boundary ownership in v7 shipped cleanly. No agent was undisciplined in either release. The architecture changed and the outcome changed. This is what §11's partnership-architecture failure mode looks like when you fix the architecture instead of the agents.

The right mental model is that a worktree is a sprint, not a shelf. You open it with a task in mind, you work the task, you close it. A worktree that outlives its task is drift in physical form — and unlike a branch, there's no PR to force a reconciliation. If the work is real, commit it. If it's experimental, commit it to a draft branch. If it's done, delete it. The worktree lifecycle should be boring. Pin: if you can't remember what it's for, close it.

Rule: Designate merge points explicitly; update them last.

Why: The hardest part of parallel agent work is not the parallel code — it's the few shared files every agent has to touch: the config registry, the main entrypoint, the integration test, the dependency manifest. If every agent edits those in parallel, you get conflicts by construction. If one agent edits them and the others wait, you've serialized the parallelism. The clean move is to declare those files as merge points — shared touchpoints that nobody edits during the parallel phase — and update them after all the parallel work has landed. The parallel phase becomes conflict-free by construction; the merge phase becomes a single mechanical pass against a known list of files. How to apply: Before fan-out, identify every file any agent would plausibly need to touch to integrate its work. That list is your merge-point set. Split the plan into two phases: phase one is the parallel work that touches only each agent's own files; phase two is a single-threaded pass that updates each merge-point file with every agent's integration. The file map becomes a contract with two columns: owned by an agent and merge point. No file can be in both. The second case study ran this pattern cleanly across every release with parallel agents: "the only shared touchpoints — the config registry, the main entrypoint, the integration test — are updated after all adapters land, eliminating contention." Six agents, seven agents, twelve agents — zero merge conflicts on core logic across every release that used this pattern. Pin: parallel work is a two-phase plan. Phase one is the fan-out; phase two is the merge-point pass. Mixing them is where conflicts come from.

Field note — case study 2: Six agents, zero conflicts, by construction. One release had four adapter agents plus two CI-pipeline agents working in parallel — six agents, one codebase, zero merge conflicts on core logic. The file map assigned every file to exactly one writer before any agent started; merge points (config registry, main wiring, integration test) were updated in a single-threaded pass after all four adapter agents reported back. The conflict that didn't happen was impossible for the structure to produce.

Second-edition clause: merge-point discipline extends to the commit log. When subagent waves land through a single controller, the controller owns the truthfulness of the history, not just the cleanliness of the merge. The fifth case study caught a wave commit whose message described only half of what the wave had actually changed — and amended it before push, for a reason stated precisely: "it only matters if you use git bisect, which Seth does." A commit message that undersells its diff is a lie the bisect will surface at the worst time. The merge pass ends with a read of the diff against the message, not just a conflict-free merge.

Rule: Subagents protect your context window; they don't hide work from you.

Why: The point of a subagent is to do a large-process task — search 200 files, read 25 retros, trace a call graph across 14 modules — and return a small output: a decision, a summary, a short report. A subagent that writes 5,000 lines you don't read is a liability, not a feature. You've outsourced the work and the accountability at the same time, and when something breaks in those 5,000 lines the debugging session starts from scratch. How to apply: Dispatch when the output you need is small but the process is large. Research, analysis, searching, reading — good subagent work. Code generation is not exempt from review just because an agent wrote it: if the subagent's job is to write code, you still have to read the code. Context-window savings from dispatching do not transfer to accountability for the code produced.

Qualifier — reading can be mechanized. At high parallelism, literal line-by-line review stops being tractable and in mechanical domains becomes unnecessary. Case study 2's seven-agents-per-release rhythm reached every line with automated gates — build+test+vet+race, integration tests spinning up every adapter, coverage matrix pinning feature coverage — and the delegation was safe because the pipeline was boring enough to trust (§3). Accountability stays yours either way; "reading" is what's delegable when gates are trustworthy. When they aren't, parallel agent work outruns supervision and the pattern collapses.

Pin — at very high velocity, the qualifier becomes the mode. At case study 3's tempo (nine major versions in three days), no human reviewed every line. Gates-as-reading became the default; human line-by-line review was reserved for craft-sensitive code (interaction design, data migrations, auth boundaries). The velocity is the signal that all three layers are doing their own jobs.

The Vitest Cache Incident (Part V) is this story viewed from the other side: four parallel agents, clean code, green tests — and thirty-eight unrelated tests hanging because the agents' concurrent writes corrupted vitest's module-graph cache in a shared node_modules. The fix was mechanical. Finding it required diagnosing an effect that emerged from the parallel structure, not from any individual agent's output. What you dispatch, you own — including the parts that emerge from interactions between dispatched tasks. Pin: dispatch to compress process, not to avoid reading the result.

Field note — case study 1: The Subagent Orchestra. Fourteen subagents across two parallel tracks, 29 files touched, zero merge conflicts — the file map assigned every file to exactly one dispatch before any agent started. Full entry in Part V.

Rule: Review delegated work in two stages, with reviewers who don't trust the implementer.

Why: A single review pass on delegated work has to hold two different questions at once — "does this do what the spec asked?" and "is this good code that will still be good next quarter?" — and in practice the first question eats the second. Worse, a reviewer who starts from the implementer's summary inherits the implementer's blind spots. Splitting the review into two independent stages, each blind to the implementer's report, catches two distinct failure populations that a single combined pass reliably misses. How to apply: After a subagent (or wave of subagents) reports done, dispatch two reviewers in sequence: a spec-compliance reviewer that reads the spec and the diff and answers only "is every requirement delivered, and does the verification witness it?" — this is where the revert question (§8) gets asked — and a code-quality reviewer that reads the diff cold and answers only "what here is wrong now, or will be wrong someday?" Neither gets the implementer's self-assessment as input. The human integrates the three reports.

The first case study ran this as standing practice through its v6.x waves — "neither trusting the implementer's report" is the retro's own phrasing — and the division of labor showed up in what each stage caught. The spec reviewer caught the verification theater (the E2E tests that bypassed the layer under test). The quality reviewers, pass after pass, caught the not-wrong-now-but-wrong-someday class that spec compliance can't see: value-equality assertions guarding a reference-identity invariant, an event-handler decoration that swallowed propagation, a regex ported faithfully from an archived config that would have matched human users as crawlers, a try/finally that leaked on one path. One wave's retro summarized: three passes, three different failure modes caught. None of the three would have surfaced the other two's findings. Pin: the implementer reports, the spec reviewer checks the claim, the quality reviewer checks the future. Nobody grades their own homework.

Rule: Isolate even the "read-only" agents.

Why: An agent dispatched to report — audit, survey, research — still runs tools, and tools touch state: package managers mutate lockfiles, test runners write caches, builds regenerate artifacts. "Report-only" describes the agent's assignment, not its side effects. A parallel fleet of readers sharing your working tree can leave it subtly dirty, and the dirt shows up later as a diff nobody can explain. How to apply: Give fan-out agents — including auditors and surveyors — a disposable checkout, a worktree, or at minimum a git status check before and after the fleet runs. If a "read-only" pass leaves the tree modified, treat the modification as unexplained state (§9): investigate or revert it before it rides along in someone else's commit.

The fifth case study's audit fleet — four parallel report-only agents, ten minutes, eighteen issues filed — was a clear win with one asterisk: afterwards, the working tree contained a phantom package.json version downgrade nobody had made, most plausibly a side effect of one auditor running a package-manager command mid-analysis. Harmless, caught, reverted — and a perfect miniature of the failure shape, because it surfaced days later as "why does the tree say 1.0.3?" The lesson isn't "don't run audit fleets" (run them; see §15). It's that parallel plus shared mutable tree is the §11 topology failure in its mildest form, and the mild form is free to prevent. Pin: "report-only" is a job description, not a sandbox. Give the fleet its own copy of the world.

§14 Plan quality

A plan is a contract you write with future-you. Future-you will be context-deprived, possibly stressed, definitely without the reasoning you had during planning. The plan is either detailed enough to carry that reasoning forward, or it isn't — and if it isn't, future-you will fill the gaps with whatever seems plausible in the moment. That is where features drift and bugs are born.

Rule: A plan with placeholders is a wish list.

Why: "TODO: handle errors later" is the engineer outsourcing the hard part to future-them. Future-them will be confused, stressed, and without context. The hard part has to be done at plan time, not at implementation time. How to apply: Every step contains the actual content. No "TBD," no "similar to above," no "implement later," no "add appropriate error handling." If you can't say what the error handling is, you're not ready to plan it yet — go back to brainstorming.

The shape of a placeholder looks reasonable. "Handle edge cases" sounds responsible. "Wire up the error path" sounds thorough. Neither tells you what the edge cases are, what the error path does, or what the downstream behavior should be when it fires. When the agent hits that step, it fills the gap with something plausible. Not wrong, exactly — but not the thing you had in mind, either, and by the time you notice you've built two layers on top of it. The tell is a plan that you can read and nod along to but couldn't execute yourself. If you couldn't hand it to a new team member and walk away, it isn't specific enough. Go back. Name the thing.

Second-edition clause: a known-failing test deferred to "future work" is a TODO, not TDD. The placeholder has a sneakier costume: a risk you did name, parked in the plan's "known risks" section, with a test you know would fail if you wrote it. The fifth case study caught one in self-review — a grammatical guard listed as a risk while the negative fixture that would exercise it was quietly going to fail on day one. The retro's verdict: "That's not TDD; that's a TODO." If you can already write the failing test, the work belongs in this plan, not the next one. Naming a gap is not the same as closing it; the plan gets credit only for the second one.

The self-updating preflight check from the blue-green deploy release is the inverse of this problem stated positively: instead of hardcoding a list of required environment variables in the script — effectively "TODO: remember to update this list when we add a new var" embedded in deploy infrastructure — the script learned to read required vars from the compose file itself. A single grep against the compose YAML. The list is always current because the plan was complete enough to ask "what is the actual source of truth for required vars?" instead of proxying it with a placeholder that would go stale. A plan that asks the right question before deferring never has to ask it again.

Field note — case study 1: The Preflight That Would Have Saved Us a Week. An earlier deploy shipped with a missing environment variable because the script's required-vars checklist was hardcoded and three vars behind the compose file. The fix was self-updating validation: the preflight grep'd the compose file for variable references and checked each one was set. The list was always current because nobody had to remember to update it. Self-updating validation beats a manually-maintained checklist, every time. Pin: if you can't fill in the step right now, you can't implement it right now.

Rule: Each step is one action, two to five minutes long.

Why: Bigger steps hide complexity and break the TDD rhythm. "Implement the feature" is a chapter, not a step. Small steps also mean small blame radius — when a step fails you know exactly where. How to apply: "Write the failing test" is a step. "Run the test and verify it fails" is a step. "Write the minimal implementation" is a step. If a step would take longer than five minutes, split it. Qualifier: the five-minute rule is a proxy for "small blame radius." When architecture provides that for free — one-adapter-per-directory, a tight interface contract, a self-contained package with its own test file — a single step can be larger and still satisfy the rule. Case study 2's unit of parallel work was "implement the next adapter" (sometimes thirty minutes or more), fine because a failure was trivially localized to one directory, one package, one agent. If the architecture isn't doing that work for you, hold to five minutes.

The failure mode is a step that sounds atomic but isn't. "Add date range filtering to the admin list" — sounds like one thing. Actually it's: write the query parameter model, add the backend filter clause, write the service method, add the frontend filter state, wire the UI controls, add the E2E test, and handle the edge case where the end date is before the start date. Seven steps collapsed into one. When the agent "does" that step, it makes all seven decisions at once and you get to review seven decisions at once — which means you'll miss the one that's wrong. The Month That Wasn't Thirty Days hotfix is a clean example: the planning step was "write tests for the 30-day date filter." The step that would have caught it was "write a test that generates dates 30 days apart" and then a separate step "verify that 30-day offset doesn't cross a month boundary on CI." The first step sounds sufficient; the second is where the flakiness lived. Step granularity is how you surface the complexity before the agent buries it.

Field note — case study 1: Hotfix 1: The Month That Wasn't Thirty Days. A test that used a 30-day date offset worked fine most months and failed on calendar days when "30 days ago" crossed a month boundary. The plan step "write tests for the 30-day date filter" was too coarse to name the constraint; a separate step — "verify that the 30-day offset doesn't cross a month boundary on CI" — would have caught it at plan time instead of from a user report. Pin: if a step takes longer than five minutes, split it.

Rule: The plan must cover the spec — and match the codebase.

Why: A plan that drifts from the spec produces a feature that drifts from the requirement. Every section of the spec without a corresponding task is not a gap in the plan — it is a gap in the feature. How to apply: After writing the plan, walk every spec section and point at the task that implements it. If a section has no task, you have a hole — not a feature, a hole.

The Four Scoping Gaps release is the canonical case. The spec was clear: the release-automation tool should open Release PRs on the develop branch and trigger deploy when a Release is published. The plan was one step — "add target-branch to the config" — that sounded like it covered the spec. It did not. Walking the spec against the plan would have surfaced four separate gaps (cataloged in Part V), each plausible in isolation, none visible unless you walked the spec step by step and asked "which task covers this?" The test for spec coverage is pointed and mechanical: take each requirement sentence and name the task that delivers it. If you can't name one, write one.

The Trust Your Local Tests positive case is worth holding alongside it. That spec broke the work into five phases and was explicit about which items were already solved by earlier phases — one issue was marked "no code changes needed, resolved by an earlier phase"; another had been fixed in a prior release. One whole phase required zero commits. The spec knew this before the engineers did, because they'd written it down. Spec coverage in the forward direction lets you close issues before the agent ever touches the keyboard.

Field note — case study 1: Four Scoping Gaps, spec-walk edition. Four scoping gaps in one release-automation plan, each of which looked reasonable in isolation and none of which a spec walk would have allowed. The plan had one step; the spec had four requirements; the math was right there on the page if anyone had written both columns next to each other. Pin: walk every spec section and name the task that implements it.

Qualifier: spec coverage includes temporal ordering across systems. The "plan covers the spec" rule, as written, catches missing tasks. It does not by itself catch tasks executed in the wrong order across systems whose dependencies cross the boundary between them. When a plan touches more than one system — code + infrastructure, code + database migrations, code + secrets vault, code + DNS, code + third-party config — the order of operations is part of the spec, even when the spec doesn't say so out loud. A plan that says "add startup-time validation that requires IP_SALT to exist" must include the step "add IP_SALT to the secrets vault" and must execute that step before the validation deploys. Otherwise the validation deploys, the process refuses to boot, and the deploy bricks itself the instant it lands. The fix is one line of plan; the cost of skipping it is an outage. Treat cross-system ordering as a first-class spec coverage check: for every task, ask "what must already be true in some other system before this task can run?" — and add the prerequisite as its own earlier task.

Field note — case study 1: The Guard That Came Before the Secret. A release added a "fail fast" startup check: if IP_SALT wasn't set in the environment, the API would refuse to boot rather than fall back to a hardcoded default that would silently weaken the IP-hashing scheme. The PR was clean. Tests passed. Code review approved it. The deploy script ran. The new container started. The startup check fired. The container exited. The deploy script retried. The container exited. Two retries later the deploy script gave up and rolled back to the previous version. Root cause: the plan added the guard but never added the step "set IP_SALT in the production secrets vault" — which a teammate had assumed was already there because the variable name had appeared in earlier code. It hadn't been. The fix was thirty seconds (add the secret, redeploy). The lesson is the rule above: a plan that introduces a precondition must also introduce the step that establishes the precondition's prerequisite, in the right order. Walking the spec is not enough; you also have to walk the dependencies between systems and verify the order of operations across them. Pin: when a task assumes another system is in a particular state, the assumption is part of the spec — and the step that gets the system into that state is part of the plan.

Second-edition clause: a plan can be wrong about the codebase, not just the spec. Spec coverage catches missing tasks; nothing in it catches tasks that describe a codebase that doesn't exist. The first case study's v6.0 plan said "create new test file" for a module that already had one — with a different mocking convention the new tests then had to be rewritten into — and used field names from the planning conversation (name, arrives_at) where the real schema said event_name, target_datetime. Both collisions were found at implementation time, which is the expensive time. This is §5's explore rule cashing out as a plan-quality check: before execution, walk the plan's nouns — every file it says to create, every field it names, every function it claims exists — and verify each against the tree. A plan whose nouns don't match the codebase is fiction with good structure. Pin: walk the spec for coverage, walk the codebase for truth.

Rule: Predict the failing output before you run it.

Why: A TDD step that says "write the failing test, run it, see it fail" has a silent weakness: any failure looks like the expected failure if you haven't said what the expected failure is. A test failing for the wrong reason — an import error, a fixture problem, a different code path — reads as "step complete" and quietly voids the whole red-green contract. Writing the predicted output into the plan turns each test run into a self-checking instrument: prediction matches observation, proceed; prediction misses, something unaccounted-for is happening and you just found out at the cheapest possible moment. How to apply: For each TDD step in the plan, write the expected failure concretely — the exception type, the assertion message, the count (1 failed, 1 passed). At execution, compare actual to predicted before moving on. A mismatch is not an inconvenience; it is the drift detector firing. Stop and reconcile.

The first case study's consumer-tier plan specified the exact pytest output each step should produce before the production change landed — down to 'NoneType' object is not subscriptable and the failed/passed counts. Prediction matched observation four times running, and the retro rated the practice "self-validating in a way no other discipline we've tried has been" — because every match is positive evidence that your model of the code is current, and every mismatch is an early warning you'd otherwise have received as a production surprise. The cost is one line per step, written when the reasoning is already loaded. Pin: an unpredicted failure proves nothing. Say what red you expect, then check you got that red.

Rule: Descope explicitly. Name what's out, and name why.

Why: A plan that only names what's in is a plan that quietly hopes everyone agrees on what's out. They won't. The items you silently omit will come back as surprise asks during implementation, as scope creep during review, or as "wait, I thought we were doing that" at the retro. Explicit descopes close those loops at plan time, when they're cheap, instead of at implementation time, when they're a conversation you didn't plan to have. How to apply: In every plan, write a dedicated "Descoped" section. List every item the plan considered and decided not to do. Give each one a named reason — "hardware-dependent, untestable in CI," "dependency too large for this release," "spec evolves faster than we can keep up," "low value for the complexity cost." A descope with a reason is a negotiable contract. A descope without a reason is an argument waiting to happen. The second case study's final release is the clean example: v1.0.0 cut four items from the plan, each with a one-line reason, and the retro treated those cuts as part of the release's value, not as a failure to ship everything. Descoping with reasons is how a plan stays honest about its own boundaries.

Field note — case study 2: Descoping the Final Four. The v1.0.0 release cut four items that had been on the roadmap, each with a one-line reason: one required physical hardware and was untestable in CI; one brought in a massive dependency with complex signaling; one depended on an enormous external codebase for a spec that evolves rapidly; one added significant complexity with no proportional value. Each descope was named with its reason in the release plan and in the retro. The release still shipped well above its original target, and the four cuts were part of the story, not a shortfall. A plan that can explain what it chose not to do is a plan that has actually been thought about. Pin: every plan has a descoped section. Every descoped item has a reason.

Second-edition clause: descoping has a change-sized sibling — the "things we deliberately didn't do" list. Descope operates at plan scale; the same honesty is worth keeping at the scale of an individual change, where the considered-and-declined edits are invisible in the diff. From its v0.11.0 retro onward, the fifth case study shipped every change with a list of the adjacent edits it chose not to make — didn't lower the triumph threshold, didn't add excited to the contentment cluster — each with its reason. The purpose is double. For the humans: "not adding things you considered adding is the half of the work that doesn't show up in the diff but does show up in the next person's trace." For the agents: a declined change with its reason attached is the only durable defense against a future session "helpfully" making it — the reason is what turns restraint from an absence into an instruction. Mid-cycle bug discoveries get the same treatment: the first case study found a dead method and a domain typo during an unrelated release and deliberately left both, writing it down plainly — "The bug is right there. It's trivial to fix. … And the right answer is still: not now." Pin: restraint that isn't written down will be undone by the next helpful editor — including you.

Field note — case study 2: The Config Test That Was Always One Release Behind. A single test in the project's config package asserted the exact number of registered adapters — a hardcoded integer that had to be bumped every single release because the test pinned a literal instead of deriving it. One retro caught it mid-flight: "TestDefaultConfig failed with the wrong expected count. The ghost of the previous release, hard-coded in an assertion, politely informing us that we had changed the thing we explicitly set out to change." The deeper lesson is that tests which pin literal facts about the codebase (counts, filenames, fixed lists) are decoupled from the facts they claim to pin, and every release has to remember to update them manually. A test that derives its assertion from the code is self-updating; a test that hardcodes it is a scheduled reminder disguised as an assertion. Pin: if a test has to be updated every time the code changes, the test is asserting the wrong thing.

§15 Audits

(New as a chapter in the second edition — the first edition carried this as a coda aside, and the evidence outgrew the placement.) An audit is a different shape of attention than review. A reviewer checks that a change is correct; an auditor asks what's wrong with what's already shipped — including, crucially, what's wrong with what's already written: the claims, the docs, the memory, the beliefs. Four case studies have now run scheduled audits, and the pattern held every time: the audit surfaced more user-visible and risk-relevant findings than the feature release before it, and almost none of the findings had ever generated a bug report. This chapter is the discipline for looking on purpose.

Rule: Schedule audits on purpose.

Why: Feature work, bug reports, and automated tests all organize attention around changes and complaints. Whole categories of defect generate neither: the security hole nobody has exploited yet, the UX seam every user silently routes around, the accessibility failure your suite scores as passing. The only way those get found is a human (increasingly: a human directing agents) deciding to walk a surface end to end with no goal except find what's wrong. How to apply: Put audits on the release calendar as first-class themed cycles — a security pass, a visual pass in both themes on both form factors, a docs-truth pass — each with its own retro. Pick one surface per audit. Do not attach the audit to a feature release; the audit is the release.

The fourth case study is the origin evidence, unchanged from the first edition and still decisive: a scheduled security audit surfaced fifteen category-level issues that a year of normal review had walked past (lost-update races, TOCTOU windows, missing UUID validation, regex injection, materialized views silently never refreshed), and a visual audit produced six UX fixes nothing else would have found. Both out-delivered the feature releases they followed. The second edition adds the mechanized form (rule 3 below) and the two rules the fifth case study's audit season forced into words (rules 2 and 4). Pin: every release that ships only what the bug reports asked for is a release that left work on the table. Audit on purpose.

Rule: Claims must be downstream of code, not upstream.

Why: Privacy policies, marketing copy, READMEs, PRDs — the project's public claims — have a failure mode all their own: they get written aspirationally, describing the system as it's intended to be, and then the implementation never catches up while the claim sits there accruing liability. A claim is a promise a user can hold you to; an unimplemented claim is a broken promise with a timestamp proving how long you didn't notice. How to apply: Treat every outward claim as an assertion requiring evidence, exactly like §8 treats "tests pass." Before publishing a claim, verify it against the running system. At audit time, walk every published claim and probe it. When the PRD and the code disagree, the tiebreak is the retro's question: which one can a user verify? — fix that side first.

The fifth case study's "ready for company" release wrote the polish — privacy policy, marketing copy — and its audit season then spent three patch releases un-writing it. The policy claimed paid-tier point-in-time backups (never purchased), claimed no third-party requests (a font CDN loaded on every page), and referenced an analytics provider (never installed). None were lies when read charitably; all were aspirations formatted as facts. The retro named the principle: "claims should be downstream of code, not upstream. Aspirational privacy text is a liability for exactly as long as it's unimplemented — which can be months when nobody's auditing." The same season caught the PRD asserting a different OAuth provider than the login page actually offered — and the code won the tiebreak, because the login page is the one a user can see. Pin: publish what the system does. If you want to publish something better, build it first.

Rule: The debt is in the seams. Diff what you wrote down against what's true.

Why: A codebase under the loop's discipline stays surprisingly healthy — the rot doesn't accumulate in the code. It accumulates in the seams between the artifacts: docs asserting what code no longer does, memory asserting what infra never did, dependency manifests drifting from lockfiles, licenses missing, claims aging. No single seam is anyone's job, which is exactly why they rot. And the audit for seams mechanizes beautifully: it's read-and-compare work, the shape agents are best at. How to apply: Every few weeks, dispatch a small fleet of parallel report-only agents — one per seam family: docs-vs-code, security posture, dependency and debt, devops config — each instructed to diff the written record against observed reality and report, not fix. Synthesize to one screen. File everything as tickets; fix in a themed cycle. Isolate the fleet from your working tree (§13).

The fifth case study's first audit fleet is the demonstration: four agents, about ten minutes of wall clock, four written reports, eighteen issues filed. The verdict on the code itself: "the codebase is in genuinely good shape… The debt was in the seams — between what we'd written down and what was true." The haul was pure seam: the unprotected main branch everyone believed was protected, the backup tier the policy claimed and nobody had purchased, the missing LICENSE, the font CDN leaking IPs, dependency drift. A release later, the retro added the operational note that makes the cadence sustainable: "The rot itemizes faster than it accumulates if you measure it at all." Ten minutes of fleet time per few weeks keeps the seam debt enumerated, which is most of the way to keeping it paid. Pin: the code is watched by the loop; the seams are watched by nobody. Send the fleet down the seams.

Rule: Probe the gate. A protection you've never tested is a protection you don't have.

Why: Protections — branch rules, backups, rate limits, alarms, permission boundaries — share a cruel property: in normal operation, a working protection and a missing one look identical. Nothing exercises them until the day something goes wrong, which is precisely the day you can't afford to learn they were never on. Beliefs about protections are therefore the single highest-value target for an audit, because they are load-bearing, untested by all ordinary activity, and (per §11's seventh failure mode) amplified by every document that repeats them. How to apply: Enumerate the protections the project believes it has. For each, run the probe that would prove it: the API call that returns the protection object, a test restore from the backup, a request that should be rate-limited, a deliberate error that should page. Attach each probe's command to the belief (§6) so the next audit re-runs it mechanically. A protection with no possible probe should be treated as absent in every risk decision.

The Lock That Wasn't (§6, Part V) is the canonical scar: weeks of confident, written, repeated belief in branch protection, dissolved by one API call returning 404: Branch not protected. The team's response is the part to copy — not just enabling the protection, but reclassifying the belief category: the README now treats an un-PR'd commit on main as a security incident, and the audit fleet's security agent probes the protection object instead of quoting the docs. Companion scars from the same season: the backup tier that existed only in the privacy policy, and the error tracker that had been CSP-blocked into silence for five releases (§8) — three protections, three probes that had never been run, three gates that were open the whole time. Pin: a gate you've never rattled is scenery. Rattle every gate on a schedule.

§16 The retro habit

Every loop ends with a retro. Not a changelog — a retro. The loop is explore → brainstorm → plan → TDD → verify → commit → retro, and the last step is the one that turns a cycle into cumulative learning. Without it, each release is isolated. With it, each release teaches the next one something the last one had to discover the hard way.

This is the second load-bearing chapter of the playbook, and it is paired with §8. Verification catches the failure; retros convert it into a rule. Skip verification and the failure ships. Skip the retro and the failure re-ships, forever. Every scar in Part V exists because one of those two disciplines was skipped. Every rule in the rest of this playbook exists because one of those two disciplines produced it. If you remember nothing else from this document, remember that pairing: verify the work to catch the bug; retro the cycle to kill the class. Two disciplines, one practice, and the reason the playbook has any rules at all.

Six projects have independently evolved this discipline. The first case study's thirty narrative retrospectives, the second case study's nine themed release retros, the third case study's nine-version retro-plus-workflow-document practice, the fourth case study's audit-driven retros, the fifth case study's twenty retros in three months — including one for a session that produced zero code (a pure elicitation session; it still had a "By the Numbers" section) — and the fleet's per-panel retros all converged on the same shape without any of them copying from the others. When six differently-shaped projects independently reach for the same tool, the tool is load-bearing, not incidental. The retro is not a ceremony; it is the mechanism by which agent-directed work becomes cumulatively smarter instead of cyclically forgetful. This chapter is load-bearing because the practice is load-bearing because the discipline is load-bearing. Everything else in the playbook is downstream of this.

Rule: Write retros in voice, not in bullet points.

Why: A dry changelog is forgotten in a week. A story is remembered in a year. The retrospective's job is to be rediscoverable — and humans rediscover stories, not checklists. Name the bugs by their nicknames. Describe what it felt like. Admit what surprised you. The difference between "Release 5.11 CI/CD Cleanup Summary" and "The One Where Five Small Fixes Grew Teeth" is not aesthetic — it is the difference between a document nobody opens and one somebody searches for by instinct two years from now. If your retro reads like a status report, it will be treated like one: filed, ignored, and eventually auto-archived by a sprint tool.

How to apply: Write like a person, not a project manager. If you spent three deploys fixing a bug that turned out not to exist, that's a story worth telling exactly that way. If a Slack integration blew up because Slack's format is named after Markdown the way ketchup is named after tomatoes, say that. The engineer who finds your retro eighteen months from now while Ctrl-F'ing for "Slack notifications broken" will thank you for the specificity. Titles are half the job: "The One Where We Solved The Wrong Bug For Three Rounds" is something someone will click. A version number followed by a feature list is not.

Field note — case study 1: Titles earn their clicks. The first case study has 24 retros, from the founding sprint through the most recent pipeline polish. Each one has a title that tells you what it's about. One is "The One Where Five Small Fixes Grew Teeth." One is "The Preflight That Would Have Saved Us a Week." Another is "Hotfix 1: The Month That Wasn't Thirty Days." The retros are the proof of concept for this rule — and they're the source material for many of the field notes in this playbook. Pin: "The One Where We Solved The Wrong Bug For Three Rounds" is a title someone will click in two years. "Release 5.11 CI/CD Cleanup Summary" is not.

Rule: Retros are how the loop learns.

Why: Without a retro, every cycle re-discovers the same mistakes. The lesson learned in retro N becomes the rule applied in loop N+1 — but only if retro N gets written. Skipped retros compound. The first skip is free; by the third, you've stopped seeing the pattern you're walking into because you never named it the first time. The instinct to skip is strongest on small releases. Those are exactly the ones that teach the most granular lessons — the kind that aren't worth a postmortem but are worth a paragraph.

How to apply: A retro is mandatory at the end of every release, even small ones. It doesn't have to be long. It has to be honest. An honest paragraph beats a dishonest page — and a missing retro is the most dishonest thing you can write, because its absence implies nothing went wrong, and something always went wrong. The Health Check That Wasn't root-cause analysis is three paragraphs and four sentences of "what we learned." That's enough. The Month That Wasn't Thirty Days retro was written in 20 minutes about a flaky test that only failed on certain calendar days. Six months later it was cited in a planning document. Paragraphs don't evaporate; meetings do.

Field note — case study 1: The One Where Five Small Fixes Grew Teeth. The Health Check That Wasn't cost three rounds of correct fixes to a non-bug because nobody stopped to ask what /health actually returned. The retro named it. The pattern — verify the verifier — is now a rule in §8. That transfer happened because the retro was written. Pin: skipped retros compound.

Rule: Write a retro at the end of every themed cycle, and right after every surprise.

Why: "End of every release" is a useful default only when your work has releases. When it doesn't — or when "release" is a blurry calendar thing rather than a shippable thematic cycle — the default collapses and retros start getting skipped for the wrong reasons. The rule that works across contexts is: write a retro whenever a cycle closes or whenever something surprises you. Cycles give you scheduled reflection. Surprises give you opportunistic reflection. Together they catch both the slow lessons (the accumulation of small rough edges over a themed slice of work) and the fast ones (the unexpected bug, the near-miss, the "huh, that wasn't supposed to happen"). A cycle without a retro leaks its slow lessons. A surprise without a retro leaks its fast ones. How to apply: Retro at three triggers.

At the end of any themed cycle with a shippable boundary. The §5 nested-cycles rule is the driver: task-level, release-level, and project-level cycles each deserve a retro at their own scale. A task-level retro can be three sentences and a lesson. A release-level retro is the full story. A project-arc retro is the meta-story of all the release retros.
At the end of any unit of work that took more than a day, even if no one calls it a release. If you spent a day or more on something, something happened that is worth a paragraph.
Immediately after any surprise — a major incident, a near-miss, a debugging session that went sideways, a deploy that rolled back, a test that failed in a way that made you say "huh." Surprise is the signal that the cycle already produced a lesson worth extracting now, not at the next scheduled boundary. Catch it while the context is hot.

Do not write retros on calendar intervals divorced from the work ("weekly retros regardless of whether anything happened"). A retro with nothing to say is worse than no retro, because it teaches the team that retros are performance, not learning. Retros follow cycles and surprises, not dates.

Field note — case study 2: Nine retros, one per release, none skipped. The second case study wrote a retro at the end of every single release — nine of them across the major arc, plus the follow-on point releases. Every one had a theme, a story, nicknamed bugs, and a "what we learned" section. The discipline wasn't "retros on Friday"; it was "the cycle closes, the retro gets written, then the next cycle can begin." The retros were the latch between cycles. Without the latch, cycle nine doesn't know what cycle three learned. Pin: the retro is the latch between cycles. No retro, no latch, no cumulative learning.

Rule: A retro has an anatomy. Use it as a scaffold, not a template.

Why: "Write in voice, name what broke, end with one extractable sentence" is a posture, not a structure. A reader who hasn't seen a good retro has nothing to start from. The posture rules (voice, honesty, extractable lesson) produce great retros once you know the shape they live in. The shape is the thing that makes a retro skimmable by a cold reader, re-readable by a future engineer looking for something specific, and parseable by an agent extracting lessons. Without the shape, retros drift into freeform memoir that is readable exactly once. How to apply: The scaffold, in five parts. Scale each part to what the cycle actually produced — not every retro needs every part, but the shape is what makes a longer retro legible.

The mission. One paragraph on what you were trying to do, why it mattered, and what "done" was going to look like. A cold reader eighteen months from now needs this to orient. Skip it and the rest of the retro is context-free.
What happened — the execution narrative. The story of the cycle. Nicknamed bugs. Surprises. The near-misses that didn't make the commit log. The parts that were harder than expected and the parts that were easier. Written in voice (§16 rule 1), not in bullet points. This is where the humans live.
The numbers. Dry metrics for the people who came for them: counts, times, sizes, test results, dependency deltas, lines added and removed. Not the whole build output — the handful of numbers that characterize the cycle. This section makes the retro skimmable by a reader with a specific question and no time for the story.
What we learned — the extractable lessons. Stated explicitly, not left for a reader to infer from the story. One sentence per lesson, with the lesson first and the context second. This is the section the agent and future-you will Ctrl-F for. It's also the section that feeds the next cycle's plan (§5 nested cycles). If a lesson isn't in this section, it isn't going to reach the next loop.
What's next. The handoff to the next cycle. What this retro is telling the next loop to do differently, to watch for, or to try. A one-line "what's next" is enough; the point is to make the lesson actionable on the very next cycle.

Not every retro needs all five. A one-day hotfix might be three sentences and a lesson. A release retro probably wants all five. The anatomy is a scaffold for when the retro is larger than a paragraph — it's what keeps a long retro from becoming an unstructured memoir that nobody re-reads. The second case study's nine release retros all follow roughly this shape: mission, waves (their execution-narrative section), numbers, what we learned, what's next. They read like stories because they have skeletons underneath.

Field note — case study 2: The retro as story with a skeleton. Every release retro opens with a one-sentence mission, walks through waves of execution with nicknamed adapters and discovered gotchas, lists the raw numbers in a "The Numbers" section, extracts lessons in a "What We Learned" section, and closes with "What's Next." The shape is unmistakable from retro to retro, but the voice is different every time — each has its own rhythm, its own jokes, its own arc. The skeleton is what lets the voice happen: when the structure is handled, attention goes to the prose. Pin: the anatomy is the scaffold that frees the voice.

Rule: A retro has three audiences: you next month, your team, and the agent next time.

Why: The retro you write today is read by future-you when you're trying to remember what you did, by teammates who weren't there, and by an agent that will be asked to read this retro as context for a future task. All three need different things — future-you needs the full story, teammates need the context they missed, and the agent needs a clean extractable lesson — but all three can get what they need from the same document if you include the lesson explicitly, not just as something implied by the narrative.

How to apply: Every retro ends with "what we'd do differently" — a sentence, not a section. That sentence is the thing the agent extracts. That's the sentence you'll Ctrl-F for next month. It doesn't have to be long. The Four Scoping Gaps retro's extract is: "Walk every spec section and name the task that implements it before starting the work." Trust Your Local Tests': "If local and CI aren't running the same test suite, fix the suite, not the gap." Those sentences are now rules in §14 and §8, respectively — they got there because they were written down explicitly, not left inside the story for a reader to infer. The story is for the humans. The explicit lesson is for everyone, including the agent that hasn't read the story yet. Pin: every retro ends with one sentence the agent can extract.

Rule: The retro unit is the arc, not the tag — and when a retro was wrong, write the reversal down.

Why: Two honest pressures emerged once the practice ran long enough, and both needed naming rather than denying. First: at real velocity, some tags are too small to teach anything alone — three same-day patch releases are one story, not three. Mechanically requiring a retro per tag produces padding, and padding teaches the team that retros are paperwork. Second, and more uncomfortable: retros themselves can encode wrong lessons. A retro that celebrates a decision feeds that celebration into the next loop with all the authority this chapter gives it — and if the decision was wrong, the loop learns the error as a lesson. How to apply: Let the retro unit follow the narrative arc: a cluster of same-day patches consolidates into one retro, provided the consolidation actually happens — the arc rule is a license to batch, never a license to skip. And when later work reverses something an earlier retro praised, the new retro says so explicitly: what we said, why we now think otherwise, and the principle that emerges. The retro directory is a lab notebook, not a trophy case; its value is that it's true, including about itself.

The fifth case study supplied both halves in one season. Three same-day patch releases got no individual retros; the next minor release's retro told their story as one arc, with the reasoning stated: "The small releases were small… the retrospectives directory is allowed not to be" a one-to-one record of every tag. Meanwhile, the project's v1.0 retro had celebrated its new marketing paragraphs as exactly the right transparency — and three of that release's claims turned out to be aspirational (§15), reversed across the following patches. The reversing retro didn't quietly move on; it stated the counter-norm: "The right thing to write in a retrospective is not always 'we got it right' — sometimes it's 'we got it wrong, here's what we did instead, and here's the principle.'" The same project also pressure-tested the boring end: when the agent recommended skipping a quiet release's retro, the human overruled, and the resulting document earned its keep with one line — "The retro is boring because the work was on schedule. We should not stop writing the boring ones." Pin: batch retros by arc, never skip the consolidation — and when the loop learned a wrong lesson, the correction is itself a retro-worthy lesson.

The first case study has thirty retros behind it, written across more than two years, each one a story the field notes in this playbook have been borrowing from. Part V collects the ones worth reading in full — alongside entries from the other five case studies — and here they are, ready when you are.