Reports Pipeline v2

In the first commit of Reports Pipeline, the commit message lists a multi-provider orchestration plugin under considered and rejected. The problem, it says, is reproducibility discipline at stage boundaries, not orchestration.

Five months later I built a second version of that pipeline that runs across three providers. It works for the reason I gave on the first day. The truth about a report lives in files and in one small state engine, so the model doing the work can be swapped. The same eight-stage workflow now runs in Claude Code, Codex, or Google Antigravity, and a project can change providers between stages.

The architecture states the rule in one line: model choice and inference can differ; stage contracts cannot.

What carried over, what changed

Version 2 began as an independent copy of Version 1’s scaffold, with none of its project data or git history. Version 1 itself is untouched. Comparing the two shows more than a feature list would, because the pipeline’s five months of failures are the part I most wanted to keep.

What carried over unchanged

Kept	Why it matters
The eight stages and four human gates	The shape of the work didn’t change. It’s now declared in one pipeline file instead of spread across agent files
A locked, hash-sealed dataset	Downstream stages read it and nothing recomputes it
A findings registry with stable IDs, and citation checks	The IDs survive revisions, so citations downstream keep resolving
Fresh context per stage	The system’s own instructions say not to pretend a long conversation provides stage isolation
Archive-and-cascade revisions	Version 1 hard-coded which stages were downstream. Version 2 reads that from the pipeline file and invalidates the changed stage and everything after it
The narrative, findings, coverage, scope, and voice checks	Identical to Version 1’s apart from line endings

That last row is the one I like most. The validators that took months of failures to accumulate, which I listed as a table of scars on the Version 1 page, came across without a rewrite.

What changed, and why

What changed	In Version 1	In Version 2	The recorded reason
Providers	Claude subagents only	Shared stage contracts, thin adapters for Claude Code, Codex, and Antigravity	Keeping my CLI setups in step by hand was a frustration I’d written down. Version 2 is the provider-neutral architecture
Model choice	Thinking stages pinned to an exact model version	Each wrapper inherits the model from the session	Which model to use is a per-vendor question, so it lives only in the vendor adapter. A workflow fingerprint now blocks a resume if the contract changed, which is the guarantee pinning was standing in for
Rules	Advisory prompts and warnings	Enforcement in code	A rule about deprecated tables sat in my memory store five times and was never applied when a query got written. A prompt can’t enforce anything, so the query helper now refuses the tables
The data lock	Hashed the dataset	Also hashes the raw inputs and the scripts that produced it, and fails if a lineage input is missing	No reason recorded. The effect: the lock now covers a dataset’s lineage, not only its output
Repo scope	Entity resolution grew to about 600 MB inside it	Started clean, with the same rule: copy outputs in and hash them, never read across projects	The side project had outgrown the repo
Windows	A POSIX-only file lock broke the test suite	A portable lock, and line endings pinned to LF	The hashes are byte-exact

Two guarantees in Version 2 don’t appear in that table, because I can’t point to a recorded failure behind them: the leases and the approvals bound to a snapshot. The nearest precedent is an editor in Version 1 that once saved from a stale buffer and wiped a rewrite. Everything in the first table was earned by failing. Those two are bets, and I’d rather label them that way.

One contract, thin adapters

Dashed boxes are things that come and go. Solid boxes are what the workflow’s guarantees rest on.

One design wrinkle: the orchestrator is a skill in the main conversation, not a subagent, because subagents can’t spawn subagents. So each provider’s skill runs the loop, calls the engine, and dispatches the stage workers.

Eight stages, four human gates

Stage	Name	Produces	Human gate
0	Brief	Brief, conventions, requirements	Yes (strict; optional in relaxed)
1	Data exploration	Source manifest, exploration notes	No
2	Data lock	Analytical dataset, hash-sealed manifest	No
3	Analysis	Charts, tables, analysis self-check	Yes (strict; optional in relaxed)
4	Findings and outline	Findings registry, outline	No
5	Narrative	Prose with a citation on every number	No
6	QA claims audit	Claims audit, corrected narrative	Yes (strict; optional in relaxed)
7	Assembly and verification	HTML deliverable, verification and retro notes	Yes (always mandatory)

The idea that carried over from Version 1 is unchanged: lock the data once, turn every claim into a registry entry, cite by ID, and let scripts check the contracts instead of asking another model whether it looks right. Under the strict policy, all four human checkpoints require explicit sign-off; under the default relaxed policy, Stages 0, 3, and 6 complete after passing validation without fabricating approvals, while the final deliverable checkpoint remains mandatory.

Who may write what

Each stage declares the files it owns. Before a stage runs, the engine snapshots file hashes. After, it compares them. If anything outside the stage’s ownership list changed, the stage fails. Stage 3 additionally protects the Stage 2 build scripts, so the analysis step can’t quietly “fix” the data it was supposed to read.

Two rules from the pipeline’s agent instructions do most of the work:

• Only the engine advances workflow state. Workers write artifacts. They never mark themselves done.
• A worker’s completion message is not a passing validation or a human approval. When a worker says “finished,” the engine runs the validators independently. The worker’s opinion of its own work is an input, not a result.

Taking turns: leases

Two AI tools editing the same project is how you get a corrupted workspace. The fix is old: one writer at a time.

Steps with a blue stripe are where the handoff could go wrong silently.

The fingerprint covers the shared workflow files themselves. If I edit the pipeline definition mid-run, a resume is blocked, because the run started under one set of rules and would finish under another. State is also written atomically behind a file lock, so a crash can’t leave a half-written record.

An approval is a statement about specific bytes

When a human approves a stage, the engine records the approval only if the fingerprint supplied with it equals the current snapshot of that stage’s artifacts. A browser tab left open from an hour ago, showing a document that has since changed, gets refused.

It sounds fussy until you imagine the alternative: you approve what you saw, the file changes underneath you, and the record says you approved the new version. A related rule keeps the audit honest. Editing the narrative prose at Stage 6 forces a new claims audit, because the audit was about the old words.

The browser is a witness, not a judge

Each human gate opens a local web review canvas. It supports comments anchored to blocks, spans, figures, and individual YAML items, plus a collapsible terminal drawer that opens a persistent shared tmux shell in the reviewed project (also attachable via a native terminal). Structured review feedback flows through a handoff queue addressed to the stage that owns the artifact. Edits made in place are logged, and cited numbers can’t be edited; to change a number you revise the finding it cites. Release is allowed only with zero open comments, and the canvas locks editing while a worker is active.

The canvas’s own design note is blunt about its limits: it does not own pipeline state, leases, validation authority, or human approval. A “release” click in the browser is evidence. The coordinator syncs the review, the engine checks it and issues a receipt, and only then can an approval be recorded.

Strict vs. relaxed invalidation

In the initial design (now preserved as the strict policy), any edit to an upstream stage automatically archived and invalidated every stage downstream of it. That rule is clean on paper and punishing in practice: fixing a wording ambiguity in the Stage 0 brief or clarifying a non-numeric note in Stage 4 forced wholesale reruns of downstream stages whose contracts hadn’t actually broken.

In October 2026, I introduced a relaxed default policy that separates re-running a worker from re-verifying the contracts:

• Targeted validator queuing: Editing an upstream artifact (brief, analysis, findings, or narrative) queues the specific affected downstream validators instead of archiving downstream outputs.
• Two gates never relax: First, if the narrative prose changes after Stage 6, the engine flags a mandatory recheck and requires a fresh Stage 6 claims audit before final sign-off. Second, Stage 7 final assembly refuses to complete until all dirty stages are resolved, all queued revalidations have passed, and a byte-bound final approval receipt is recorded.

What the validators check

The engine runs these, not the workers:

Validator	What it enforces
Scope	Every source named in the brief’s requirements appears in the source manifest, before the lock
Data lock	Dataset and input hashes match the sealed manifest; symlinks and non-local inputs are rejected
Plots	All figures share one canvas width, within a 2% tolerance
Findings	Required fields, unique IDs, valid confidence levels; low or medium confidence needs an explanation
Coverage	Every constituent claim of the brief’s one-sentence takeaway has at least one supporting finding
Narrative	Citations resolve, required findings are cited, uncited numbers are flagged, cited values match their findings
Requirements	Non-numeric brief requirements: anonymization, source coverage, section count, voice
No stale	No leftover _old copies, duplicate directories, or competing “canonical” artifacts
Cited spans	Values in the rendered HTML match the findings registry
HTML audit	Figure width parity and alt text in the final deliverable

None of these judge whether the analysis is good. That’s the human gates’ job. They check whether the structure the analysis claims to have is actually there.

One number, end to end

This example is synthetic. It’s the same shape the repo’s fixtures use.

# Stage 2: data/MANIFEST.yaml (sealed)
dataset: analytical.parquet
sha256: "9c1e…"
rows: 2

# Stage 4: findings/findings.yaml
- id: F001
  value: 2
  description: "The sample contains 2 device-test rows"
  source:
    dataset: data/analytical.parquet
    script: analysis/count_rows.py
  confidence: high

# Stage 5: narrative/narrative.md
"The sample contains 2 device-test rows [F001]."

# Stage 7: deliverable.html
<span class="cited" data-finding="F001">2</span>

Each hop has a different checker. The data-lock check proves the dataset is the one that was sealed. The findings check proves F001 has its source and confidence. The narrative check proves [F001] exists and that “2” matches its value. The cited-spans check proves the HTML still says 2. And the approval at the final gate is bound to the fingerprint of all of it.

Keeping it portable

A small read-only doctor script checks that the core Python modules import and that 19 bundled reference files still match their recorded SHA-256 hashes. That matters because the bundle is what makes a fresh checkout behave the same as mine.

It also runs natively on Windows, which took two unglamorous changes: the file lock uses the right primitive on each platform, and line endings are pinned to LF. Hashes are byte-exact, so a checkout that quietly converts line endings would fail verification for no visible reason.

What hasn’t run yet

• Live multi-provider production district cutover. All eight stages, provider handoffs (Claude <-> Codex <-> Antigravity), relaxed revalidation queues, and the embedded review terminal are validated across 555 automated tests and synthetic fixtures; cutting live client reports over from v1 is the next step.
• Controlled cross-machine pilot. Handing a sealed project directory between the Windows workstation and the work Mac mid-run.
• Parallel work. The pipeline is strictly sequential. Whether small parallel teams would help is an open question, not a feature.

What the stack looks like

Layer	What	Notes
Orchestration	Claude Code, Codex, or Antigravity skill	Runs the loop in the main conversation
Stage workers	Eight wrappers per provider	Model inherited from the session
State	One Python engine	File-locked, atomic writes, strict & relaxed policies
Validators	Python scripts	Run by the engine, not the worker
Review	Local web canvas + terminal drawer	Evidence only; dispatches comment batches to active CLI
Data	Files: parquet, YAML, markdown	Hash-sealed where it matters
Tests	555 passing (pytest)	Covers multi-provider handoffs, validators, relaxed policy, and canvas drawer

Why build this yourself?

Multi-agent frameworks tend to sell flexibility. The thing that mattered in this build was the opposite: deciding what the model is not allowed to own. State, approvals, and “done” belong to a small engine, because a model’s confidence isn’t evidence.

If you’re evaluating an orchestration platform, the questions I’d carry over are: who is allowed to say a step is finished? What happens when a second tool touches the same files? And when something is approved, what exactly was approved?

The most useful sentence in this project’s history is a rejection from its first commit. I turned down multi-provider orchestration because the problem was discipline at the stage boundary. It took five months of failures, eleven check scripts, and a rewrite to find out how literally that was true: when the boundary is files and a state engine, supporting a second provider is mostly adapters.

The pipeline itself is a specific shape for a specific job. What travels is the boundary: keep the guarantees one layer below the tool, and the tool can change.