Running a web feature with no session editing #

The aim: a web feature goes from a spec to a landed commit and a community announcement on folknet hardware with nobody editing anything. A person or an interactive session that edits files inside a feature's track is a failure of the system, not a step in it.

This document surveys how far the repository is from that, ranks what stops it, and proposes a design. Nothing here is built. Evidence is cited as file plus symbol, or as the overnight run of 2026-09-24/25 (the roster handoff.md on main → "Overnight ramp", and the tick log $HOME/.local/state/spec-watch/spec-watch.log). Every claim not checked against code or a run is marked unverified. Hosts and addresses stay in $HOME/rig-notes.

The principle #

The whole lifecycle of one feature is packaged as one job. The job is shipped to a build lane, started there, and runs to the end there. The only thing watched from outside is a commit appearing on a result branch. A live writer session in a development-box worktree, one per feature, is the failure state, because it needs something to keep it alive (a tick, a relaunch, a local rebase) and every such touch is a place a person is pulled in.

The development box does three things: submit a job, poll for its result commit, and land it.

The loop starts at an enabled spec and ends at a result commit. It does not write or enable specs: a person or a coordinator session does that, outside the loop, so spec authorship stays with a reviewer. Finished features are reviewed afterwards, asynchronously, and a review that finds problems sends the feature back through the loop as a new or amended spec.

The loop as it stands #

flowchart TD
  spec["1. spec written<br/>(person or session, by hand)"]
  tick["2-3. cron tick: model session creates<br/>the worktree, commits handoff.md"]
  launch["4. writer launched on a lane<br/>(local if every lane refuses)"]
  work["5. writer works on the lane:<br/>corpus, ground truth, loop, model batch,<br/>admit, commit, docs"]
  fetch["6. tick fetches the commits back"]
  rebase["7. tick rebases locally if behind"]
  gate["8. verify on a lane<br/>(local if no lane answers)"]
  review["9. spec-reviewer subagent<br/>inside the tick session"]
  land["10. bench/land.sh: gate again on a lane,<br/>fast-forward merge, tag"]
  after["11. bench/after-land.sh: dashboard and map update,<br/>announcement"]
  hand["12-13. by hand: push main;<br/>map rules only for a spec<br/>landed outside a job"]
  late["14. asynchronous review of the finished feature<br/>(person or coordinator session)"]
  spec --> tick --> launch --> work --> fetch --> rebase --> gate --> review --> land --> after --> hand
  after --> late
  late -. "problems: new or amended spec" .-> spec

#	Step	What runs it	Where	Person or interactive session needed
1	Write the spec and its touches, depends-on, enabled	a person or an interactive session; drafting rules are prose in the roster's "Queue" section	development box	yes, every feature
2	Decide a tick is due	cron, spec-watch-tick.sh then spec-watch-due.mjs (no model)	development box	no
3	Create the worktree, run tracks.json install, write and commit handoff.md	the tick: one claude -p '/spec-watch' session (prompts/spec-watch.md → "Start unstarted or stale work")	development box	no
4	Start the writer	spec-watch-launch.mjs → bench/lane-writer-launch.sh → the declared harness's writer on the lane with the lowest load	lane; local when every lane refuses	no
5	Build the feature: corpus, ground truth in three browsers, generation loop, model batch, admission, commit, docs	the writer, following the spec's prose	lane	no
6	Bring commits back	spec-watch-fetch.mjs → bench/lane-writer-fetch.sh, fast-forward only	development box	no
7	Rebase onto main when behind	the tick; a conflict outside handoff.md relaunches a writer locally ("A rebase conflict")	development box	no
8	Gate: bench/verify.sh	bench/on-lane.sh	lane; development box if no lane answers	no
9	Independent review	spec-reviewer subagent, four questions, inside the tick session; verdict saved in $HOME/.local/state/spec-watch, not in git	development box	no
10	Land	bench/land.sh, one track per tick; reruns verify and landChecks (bench/fragment-floor.sh); bench/after-land.sh then raises the floors	gate on a lane; merge in the primary checkout	no
11	Dashboard and map update, announcement	bench/after-land.sh → bench/dashboard-update.sh, bench/web-features-update.sh, bench/feature-announce.py; a landed job always posts once to #bez (what it admitted, its run time, and the features that moved or why none did), any other landing only when a feature moved to partial or complete	development box	no
12	Push main and spec/* tags	bench/land.sh pushes nothing ("Nothing is pushed"); jobs/scripts/poll-and-land.sh pushes a landed job's spec/<name>/<sha> tag to JOBS_TAG_REMOTE (default origin), not main	development box	yes for main, after each batch of landings; also for the tag of a landing the poller did not make
13	Floors, map rules for what landed	floors: bench/generated-floors-update.sh from bench/after-land.sh; map rules: a job's feature-map stage from its manifest's feature_rules, with a NOT ANNOUNCED line from bench/expected-features-check.py after landing when a declared feature did not move	lane (floors, the job's stage); development box (the line)	no for a job; yes, by hand, for a spec landed outside a job
14	Review the finished feature; on problems, write a new or amended spec that re-enters at step 1	a person or a coordinator session, outside the loop, after the result commit and without holding a landing	development box	yes, by design: spec authorship stays with a reviewer

Which parts already work unattended: the overnight run had the tick write, review and land seven specs with no hand intervention (dashboard-history-graph, text-metrics-oracle, testdriver-playwright-input, transforms2d, audio-null-sink, inline-atomic, tables-fixed-layout), and fixed-positioning landed by hand after bench/land.sh refused it. The independent review found two real defects in aspect-ratio. So steps 2 to 11 run without a person when a spec is well formed; what fails is the edges (steps 12 and 13) and the checking inside the steps. Steps 1 and 14 need a person or a coordinator session by design.

One correction to older text: the roster's "Resume Here" says an owning interactive session's loop drives ticks. spec-watch-tick.sh's header says no interactive session drives ticks, and the crontab runs it every five minutes for genera. The roster line is out of date.

Reviewing, stopping and rewinding a job's stages #

A job's stage results can be read and re-run from the development box without watching a lane directory.

• A commit per passed stage. After each stage passes, job-run commits the work tree as stage: <name> on the branch job-stages/<name> and pushes it to the --publish remote (result.rs commit_stage, publish_stage). Each commit's parent is the previous stage's commit, and the first stage's parent is the job branch head. The tree is the work tree minus git-ignored files and the manifest's keep_dirs; a kept directory appears as a Kept-dir: <dir> <tree hash> line in the message, because corpus caches are large. A landing keeps the branch on the tag remote as the tag stages/<name>/<run id> before it deletes it from the forge (jobs/scripts/poll-and-land.sh); when the tag cannot be pushed, the branch stays on the forge.
• Output under ignored paths. A stage's declared outputs (the stage cache's field) and its output_dirs (jobs/src/manifest.rs) are force-added to that stage's commit and kept in every later stage commit, even where git ignores them or they lie in a kept directory, so each stage commit holds what the stages so far produced (for aspect-ratio, a few MB of corpus/ per run). The dump stage of jobs/scripts/e2e-fixtures.py stage_commits_stages() names target/e2e-stage-dump in output_dirs. Until 2026-10-08 no step-pipeline run had a stage commit: git add refused the exclude of a kept directory that git ignores, and every stage logged "no stage commit".
• Why a separate branch. Landing fast-forwards the base branch to the tip of job/<name> and takes every commit reachable from it, so generated output on that branch would reach main. job-stages/<name> is never an ancestor of the job branch, and a landing deletes it with job/<name> and result/<name>.
• Stopping. JOBS_STOP_AFTER=<stage> jobs/scripts/submit.sh <name> makes job-run stop once that stage has passed: exit 13, state saved, job branch pushed, no result written. The poller records the job as stopped and does not resend it. A resend by the poller keeps the stop point (record field stop_after), so a dependency dropping out cannot carry a held job past it. A plain submit.sh <name> clears it and resumes at the next stage. A resend by the poller likewise keeps the run's other settings from its record (model, batch_models, rerun, pass_failing_cases), never the poller's own environment. A run stopped in the middle with bench/dagu-lane.sh stop-run job <run id> is recorded stopped too and not resent.
• Rewinding. JOBS_REWIND_TO=<stage> (job-run --rewind-to) restores the work tree from that stage's commit, resets the job branch to the Job-head: trailer the commit carries and makes the later stages pending. It applies once, on the send that asked for it: a resend by the poller resumes where the rewound run got to, and a send that died before job-run started loses its rewind and must be sent again. It exits 2 for an unknown stage, a stage that has not passed, or one with no commit, and combines with JOBS_STOP_AFTER only when the stop stage comes after the rewind stage.
• Stages with no commit. A job that resumes from a run that began before stage commits existed has passed stages with no commit. None is invented from the current tree, since that would put later content under an earlier stage's name; the next stage commit lists them as No-commit: <stage>. A resume in a fresh clone first fetches job-stages/<name> from the publish remote (result.rs adopt_stage_branch), so the earlier commits are chained on, not overwritten.
• Reusing a stage whose inputs did not change. A script stage may declare inputs (repo paths or globs it reads; a path inside an earlier stage's outputs stands for that stage's output), env (variable names whose values it depends on) and outputs (everything it writes). Before such a stage runs, job-run keys it by those inputs, its command, the {tools} files it names and its agent (jobs/src/cache.rs stage_key), and when the share (WEBENGINE_LANE_SHARED) holds an entry for that key under archive/stage-cache/, it restores the stored outputs and passes the stage without running it (message reused <key prefix>). A stage that runs and passes stores its outputs there. job-run --rerun <stage> forces a run and replaces the entry. A reuse rewrites the entry's last-used file (cache.rs mark_used()), and right after each store job-run removes that stage's old entries (cache.rs prune()): per job and stage it keeps the 2 most recently used, with no age floor, and removes the rest along with .tmp-* leftovers older than 14 days. This is the one retention rule every job-run folder follows (jobs/scripts/retention.sh, docs/infrastructure.md → "Retention of job-run folders"), which also applies it at the end of every job and on each lane sweep. An entry's last use is the newest modification time of its directory, meta.json and last-used; on the share that time comes from the NAS clock. A removed entry is renamed aside first, so another lane's lookup sees it whole or not at all. The prune runs in job-run too, rather than only at the end of the job, because a store is the only thing that adds entries; a prune that fails is logged and the stage still passes. job-check manifest checks that every repo path the command names lies inside an input or an output, that --out lies inside an output, and that no two stages' outputs overlap; what a script reads without naming it cannot be checked. JOBS_RERUN=<stage>[,<stage>] on submit.sh passes --rerun through bench/dagu/job.yaml (RERUN). A review is never restored; in a manifest with no writer stage it is passed without running when nothing was committed since the base commit (a store that found its candidate already stored), with the message nothing committed since base; no diff to review, and job-run writes the review.json a reviewer would have, a pass with that message as its notes, so the result passes job-check result (jobs/src/runner.rs rule 6). A script stage that declares outputs without inputs always runs and is never stored.
• Starting afresh instead of resuming. A resend whose spec changed, or of a job whose manifest gives any stage inputs when the manifest or a path some stage lists as an input differs between the recorded base and the base tip, goes at the base tip and starts the job afresh (jobs/scripts/submit.sh, "The base sent"): submit.sh tags and deletes the forge's job/<name> from the older base, and the lane's prepare moves the old job directory aside and clones again, so every stage runs again except those the stage cache restores. Any other resend resumes in place at the recorded base, as before; a needs-person resend with the spec unchanged always does, since it carries a person's fix. prepare no longer restores keep_dirs from the newest archived tar, which said nothing of what the directory was made from. specs/aspect-ratio.job.json declares inputs and outputs on every script stage but wpt (its checkout is pinned by oracle/suites/suites.json, which keys the stages that read it), store (it commits), findings-pages-prompt and findings-pages-keep, which declare outputs only: findings-pages-prompt reads the round's rework file, which no path names, and findings-pages-keep reads its own earlier output to carry the pages earlier rounds kept, so both run every time, and the stages reading their output are keyed by what they wrote. That manifest has no writer stage; its rework_from names findings-pages-prompt, so a review fail that names the inputs on which the candidate is wrong turns them into browser-measured train cases and repairs again, up to failed_rounds (tasks/job-review-rework.md). A fail that names no such input gets "rework": false from pi-review.sh and starts no round; its review is advisory, so once no round follows the job lands the stored candidate on the score bar (jobs/src/runner.rs rule 22). repair lists every folder and file bench/repair-loop.sh leaves beside the batch, for each of the twelve rounds its --rounds 12 allows (outputs take no globs, so a change to --rounds changes the list with it); the loop only reads the batch itself (each round's report.json goes in that round's -work folder), so the candidates stage's output stays its own.

bench/e2e-jobs.sh has two scenarios for this: e2e-stage-commits covers stage commits, output_dirs, rewind through submit.sh and the stage branch being gone after a landing; e2e-stage-cache edits one stage's input on the base after a failed run and expects the resend to start afresh with only that stage, the stage that reads its output and the stages without inputs run again. Unless given --keep, the run's cleanup (bench/e2e-lane-clean.sh) removes each scenario's stage cache entries (archive/stage-cache/<name>/), so gate runs leave none on the share.

Gaps, ranked by how often they stop or degrade the loop #

Counts come from the overnight run (eight specs, seven landed by the tick, one by hand) and are the roster's, not rechecked against the tick log line by line.

1. Specs are written by hand (stops every feature) #

Nothing creates a spec, and the loop is not meant to: specs are written by a person or a coordinator session outside the loop, so the gap to close is a malformed spec, not the lack of an author. The roster's "Queue" is drafted by a person or a session, and the drafting rules live as prose there: single-line front matter (bench/land.sh touchesOf() reads a multi-line list as empty), touches must name the spec's own test files, its provenance module, crates/provenance/src/lib.rs, docs/generated-functions.md and docs/feature-map.json, and a depends-on on the previous generated-function spec because they all edit fragment.rs. No check enforces any of it. A drafted spec cannot arrive by a track either: bench/land.sh refuses any diff under specs/ or tasks/.

Every feature job carries the same two things, and these are checked. Its manifest (specs/<name>.job.json) lists the web-features ids it moves as expected_features and the exact browser-compat-data keys it implements as feature_rules, each with the status and evidence of a docs/feature-map.json rule, and only keys the spec's scope covers (a feature's keys are listed in the pinned web-features package under oracle/node_modules; aspect-ratio's spec leaves out the <img> and <video> keys, so its manifest does not list them). Its spec's touches name docs/feature-map.json, since the job commits the rules there, and its prose must not forbid that file. job-check manifest refuses a manifest with features and no rules, or rules and no features; job-check manifest --spec, which jobs/scripts/submit.sh runs, refuses one whose spec's touches omit the map (gap 2).

2. Announcements needed hand-added map rules (closed for jobs) #

A feature moves past unknown only through an exact-key rule in docs/feature-map.json (docs/web-features-map.md → "The status rule"). The generated-function specs' prose forbade editing that file, so none of the seven landings announced anything until rules were added by hand at 6a3de27, and bench/after-land.sh could not tell "nothing moved" from "no rule exists", so a missing rule looked like a quiet success.

For a job, the rules now come from its manifest (feature_rules, gap 1), and no writer has to choose to add them. job-run adds a feature-map stage before review (jobs/src/runner.rs add_feature_map_stage): job-check feature-map writes the rules into docs/feature-map.json through job-check feature-rules and commits them, runs bench/web-features-update.sh --no-commit, and fails the job when a listed key or expected feature does not end partial or complete. After landing, bench/after-land.sh runs bench/expected-features-check.py, which prints a NOT ANNOUNCED line naming each declared feature no event moved that is not partial or complete, with the fix. The announcement no longer depends on the map: a landed job posts once whether or not a feature moved, and a map that did not move is one line of that post giving the reason (docs/web-features-map.md → "Announcements"). Nothing is owed for aspect-ratio, which ran before the stage existed: its job definition is removed, and a new one writes its rules through the stage.

3. Floors are added by hand (silent on every generated function) #

Resolved by the floors-follow-the-corpus track. bench/seam-floor.sh used to have four floors (floats, stacking order, relative positioning, block height); containing block, transform and inline atomic had none, and each landed without one, because adding a floor needed a change in bench/ (refused by bench/land.sh for a track), a score line in seam_floor_check.rs and a cache rebuild in the script's loop. Now every generated function is scored by bench/generated-scores.sh against <name>.floor.json, a new function is one entry in generated_score_check.rs FUNCTIONS, and bench/generated-floors-update.sh raises the floors after each landing.

4. Steps the spec prescribes in prose are not enforced (silent, once in seven) #

tables-fixed-layout landed with a hand-written crates/layout/src/generated/table.rs whose table.provenance.json records "model": "hand-written" and "cases_run": 0, because its three lane runs never happened, and the review passed it. The review's four questions (prompts/spec-watch.md → "Review work") ask whether the diff implements the spec, not whether the runs the spec requires happened, and bench/land.sh reads no provenance file. The map already treats such a record as not complete (docs/web-features-map.md: a complete key needs at least one case run and every case passed), so the evidence exists and nothing reads it at landing.

5. Model batches bypass the batch script (loses the numbers on every landing) #

transforms2d's spec prescribes cargo run --release -p generation --bin transform_real_model_run (tasks/transforms2d.md), not bench/absolute-mixed-batch.sh or bench/overflow-mixed-run.sh. Only those two scripts call bench/ledger.py (through bench/batch-lane.sh batch_measure_stop), so a batch run the spec's way writes no ledger line, and the announcement has no candidates, tokens or CPU-hours to post. The roster says no real batch has written a ledger line yet. The tick log says the transforms2d batch "ran locally"; the log also says its writer started on lane 1, so the batch most likely ran on the lane inside the writer's copy, not on the development box (unverified: which machine ran it). Either way it did not go through bench/batch-lane.sh, which is what enforces "a model batch fails rather than fall back to a local run".

6. Landing refusals hold a slot forever (stops one feature and its dependents) #

bench/land.sh refuses committed files outside touches, any diff under bench/, tracks.json, .github/, specs/, tasks/, .cargo/, rust-toolchain*, .env*, .gitattributes, .gitmodules, and a **-broad touches (header of bench/land.sh; isDenied()). A refused track stays awaiting-approval, holds a live slot and blocks its dependents. fixed-positioning hit this and landed by hand. The touches omissions in sandbox specs (ring-buffer) are the same failure. A person's /land is the only way out.

7. The tick is a model session on the development box that edits worktrees #

The tick creates worktrees, runs npm install for oracle/, writes and commits handoff.md, rebases, resolves handoff.md conflicts, and relaunches a writer on the development box when a rebase conflicts outside handoff.md ("Rebases are kept on this machine by design": a lane copy carries only track/* branches, so a lane writer has nothing to rebase onto). It also falls back to a local writer when every lane refuses. Recorded failures: a lane launch read as a failure and started three writers on one track (fixed in pi-tracks); a tick died with no exit line twice; a lane writer waiting on a silent subagent stayed working for over 30 minutes, and the stall check did not fire; lane copies diverge after a local rebase. These are the roster's "Spike findings", several marked fixed. Each is a place where a dev-box process has to stay healthy for a feature to finish.

8. Gates can fall back to the development box #

bench/on-lane.sh runs a command locally when no lane answers, unless --lane-only. bench/land.sh runs each landChecks command through it without --lane-only, and runs verify directly (tracks.json's verify even falls back to a local cargo test when bench/verify.sh is missing), so with both lanes down the floors could run on the development box. That contradicts the root AGENTS.md for everything except model batches and bench/verify.sh, which is lane-only already. Unverified: that it has ever happened. Fixed in bench/land.sh: with WEBENGINE_LANE_STRICT=1 the gate and every check go --lane-only (docs/infrastructure.md → "Build lane"). The scheduled poller gets the variable from tracks.json's specWatch.pollEnv ("Hand changes still needed", item 5); a landing run by hand, or by the model tick's specWatch.landScript, does not set it.

9. Pushing main is by hand #

The tick lands on main without pushing. The poller (jobs/scripts/poll-and-land.sh) pushes the spec/<name>/<sha> tag of each job it lands to JOBS_TAG_REMOTE (default origin), but not main. A person pushes main, and any spec/* tag the poller did not push, after each batch (docs/coordinator-on-folknet.md → "Who pushes"). This does not stop the loop, since a lane gets unpushed commits in a bundle (bench/remote-lane.sh bundles origin/main..HEAD), but the bundle grows with every unpushed landing and the public repository lags.

10. Smaller items #

• One review round: maxRounds is 1, so the second failed review parks a track (aspect-ratio used its one round overnight; a second failure parks it).
• awaiting-approval tracks were listed as waiting on a person in repositories that land them automatically (roster "Spike findings"; the prompt now excludes them).
• The coordinator's move to a folknet guest (docs/coordinator-on-folknet.md) is designed and not done, and needs a person.
• Not yet exercised: subscription-limit backoff, the stalled-writer relaunch, and lane capacity under many writers (roster).
• An open decision: how wide a feature spec's generated scope should be (roster, "Blocked on"). It shapes what a spec asks for, so it shapes the job's stages.

Proposed design #

What a job contains #

One job is a branch plus a manifest, and everything the run needs is either in it or pinned by sha.

Part	Content	Today
Spec	specs/<name>.md at a pinned sha; its touches, depends-on, writer model	the same, read by spec-status.mjs
Base	the sha of main the job starts from; the lane carries this commit so a rebase can run there	not carried: lane copies hold track/* only
Plan	the brief for the writer stages, and the ordered stage list below	handoff.md plus prose in the spec
Model and batch settings	provider (never metered), model pool string, per-model counts and concurrency, stop rule (two flat batches or six), escalation	prose inside the spec (tasks/transforms2d.md)
Floors	per generated function: which corpus binary rebuilds its cache, the score it must keep	generated_score_check.rs FUNCTIONS and <name>.floor.json, raised automatically
Announcement	the features expected to move, and the exact-key rules that move them; the landing itself triggers the post, so a job that moves no feature still posts, with the reason in its map line	the manifest's expected_features and feature_rules, written into docs/feature-map.json by the job's feature-map stage; bench/after-land.sh passes the landed job's result.json to bench/feature-announce.py
Limits	wall-clock cap, writer relaunches, review rounds	maxRounds in tracks.json, three relaunches in prompts/spec-watch.md

The manifest is one file beside the spec, read by a script, so nothing depends on prose.

Stages, in order. A stage is either a writer stage (a Claude session on the lane, subscription-billed, doing work that needs judgment) or a script stage (a fixed command whose output is checked). Only stages 1 and 6 and the review need a model; the rest are commands, because the run must not depend on a writer choosing to do them.

1. Writer: hand-written substrate, types, baseline, corpus recipes.
2. Script: ground-truth cache from three browsers (*_ground_truth_cache). Check: case count and drops recorded.
3. Script: the generation loop over the baseline (*_generation_loop), which seeds the regression set. Candidates are scored against the browser-measured cases alone, with no hand-written reference and no deliberately wrong fixtures (docs/generation-loop.md step 2 gives the reason).
4. Script: the model batches, always through a batch wrapper that records a ledger line. Check: at least one ledger line for the job's name; provider is not metered.
5. Script: admission and commit of the winner with its provenance record. Check: the record's model is not hand-written unless an escalation is recorded, cases_run is above 0, cases_passed equals cases_run, regression_free is true.
6. Writer: wiring, docs, floor entry. Not the docs/feature-map.json rules: stage 8 writes those from the manifest's feature_rules.
7. Script: rebase onto the carried base; verify; landChecks; the new function's floor. A conflict resumes writer stage 6 on the lane with the conflict named.
8. Script: the feature-map stage, which job-run adds before review when the manifest has feature_rules (job-check feature-map). It writes and commits the rules (job-check feature-rules), then runs the map update as a dry run (bench/web-features-update.sh --no-commit) and puts the snapshot and event log back. Check: every listed key ends partial or complete through its own rule, no further back than before, and every expected_features entry ends partial or complete; it need not move, so a job that extends an already partial feature passes. After landing, bench/expected-features-check.py prints a NOT ANNOUNCED line for a declared feature that still did not move.
9. Review: a fresh read-only session on the lane, given the spec, the diff and the stage results. It answers the tick's four questions plus "did each script stage produce its artifact".
10. Result: a commit on result/<name> in the lane's copy holding result.json: spec sha, base sha, branch tip, per-stage status and artifact hashes, review verdict, ledger lines, expected events. Written last, and only when every check passed. A failed job writes the same file with the failing stage, so a poll can tell "failed" from "still running". The poller keeps each result it reads (a landing, a failed stage, a job waiting on a person) on tangled as the tag result/<name>/<run id>, and deletes the forge's job/<name> and result/<name> after a landing only once that tag is pushed.

The runner and the stage checks are copied from main at submit time and run from that copy, never from the job's own tree. A job that edits them affects later jobs, not its own run, which keeps the reason bench/land.sh gives for protecting its own files (a track must not rewrite the judge it is judged by) without putting the runner under bench/.

Before that first stage, on every start and every resume, the runner checks that the lane actually has what the manifest is about to need: a resolvable git, python3, node and kill, cargo and the oracle's Playwright browsers when a writer is in the stage list, a reachable model gateway when a stage runs a pi session, disk space, the shared archive directory, and enough wall-clock time left across resumes to be worth starting another stage (jobs/src/preflight.rs). One unmet requirement prints every reason, records them in state.json, runs no stage and exits 9 rather than let a writer spend a long stage only to lose the next one to something as small as a missing PATH entry. A gateway or NAS share that is only down exits 12 instead, after a bounded wait, and is resent by the poller.

An agent of kind command that runs pi-writer.sh or pi-review.sh may set thinking (off, minimal, low, medium, high, xhigh or max); the runner passes it as --thinking, a fan_out block's writers inherit it, and the stage's thinking in result.json reports it (tasks/job-agents-setting.md). An agent that can run a deepseek* model may use only low, high, xhigh or max; the gateway answers the other levels with HTTP 400, so the manifest check refuses them.

A job survives a change in which models the gateway serves. An agent of kind command may list fallback models after its model, and a fan_out block may list fallback spares for its models (jobs/src/fallback.rs; no manifest on main lists any). At each start the runner picks, from the lane's snapshot, the first model of each list that is declared, not retired, listed and ready, and runs every stage of the role on it; the job is refused only when no model of a list is usable (exit 10 when every one is out for good, else exit 3 so it is resent). If a stage's command then exits 66 (pi-session.sh does when its first request is refused with 401, 403 or 404), the role moves to its next model and the stage runs again, without counting a failed attempt. result.json has model_choices: the chosen model (or a fan_out stage's resolved models) and each model passed over, with the reason and whether it was at start or while running; each stage's model is the one it ran on. The candidates batch's models is a set run together, not a preference list, and has no fallback.

What a job does when a dependency drops out #

A run pauses and continues by itself when a dependency it needs is down for a while. The poller's resends for these are uncounted (they do not use up the three resends) and capped: infra_resends in the job's record counts them in a row, JOBS_INFRA_RESEND_CAP (default 3) is the limit, and the next one records failed with failure=infra and a message saying so, which a person resends with jobs/scripts/submit.sh <name> once the dependency is back (that resend is uncounted too). The count starts again when a run ends any other way.

Dependency	How it shows	What happens	Caps
Forge, mirror or lane while prepare runs	git ls-remote or git fetch fails in bench/dagu/job.yaml prepare	retry() runs it again after a pause	10 tries, 15 s doubling to 75 s (PREPARE_TRIES, PREPARE_PAUSE); then the step fails and the run ends - before the run step began, a failure marked infra that a person resends
Gateway or NAS share at the start	job-run preflight, exit 12	the check waits, then the poller resends uncounted	10 tries, about ten minutes (JOBS_PREFLIGHT_WAIT_TRIES, JOBS_PREFLIGHT_WAIT_SECONDS), then 3 resends in a row
Gateway during a stage	the stage's command (pi-session.sh) exits 67 on connection refused or reset, a timeout, 500, 502, 503 or 504	job-run waits for the gateway (the same bounded wait) and runs the stage again, with no failure and no writer relaunch counted	3 reruns per stage; a gateway that does not answer, or a fourth outage, is an ordinary failed attempt
Model not served	pi-session.sh exits 66 on a first request refused with 401, 403 or 404	the role moves to the next model of its fallback list	the list
Forge at publish	job-run exit 7	the poller resends uncounted	3 in a row
Inventory snapshot stale or a model not ready	job-run exit 3	the poller resends uncounted	3 in a row
Runner stopped (other than by stop-run, below) or killed after the run step began	exit 8, or no exit code with run started	the poller resends uncounted; job-run resumes the interrupted stage (JOB_RESUMED)	3 in a row
Anything permanent (a missing binary or variable, a retired model, a stage's own failure, exit 1, 2, 6, 9, 10, 11)	those exit codes	recorded failed or needs-person for a person; not resent	none

A run a person stops with bench/dagu-lane.sh stop-run job <run id> is not resent: before asking Dagu to stop it, stop-run leaves the file stopped-runs/<run id> in the job state directory (JOBS_STATE_DIR, default $HOME/.local/state/webengine-jobs), and the poller, finding it, records the job stopped with the line "resend with jobs/scripts/submit.sh " whether the run ended with exit 8 or with no exit code, then deletes the file. The file is written on the machine that runs stop-run, so this holds only when that is the machine the poller runs on (the box). A stop any other way after the run step began is still taken as a killed runner and resent as in the table.

Every resend by the poller goes with the per-run settings the job's record keeps (stop_after, model, batch_models, rerun, pass_failing_cases) and never with a rewind, which the run that asked for it has already done.

A writer stage with a fan_out block runs several writers on several models, each in its own checkout. The runner starts at most WEBENGINE_JOB_WRITERS_AT_ONCE of them at a time (default: the lane's CPU count divided by 16, at least 1) and queues the rest; a killed runner reruns only the writers that had not ended. The branches of the writers not chosen are kept as archive/job-writer/<job>/<stage>-<n> and pushed with the result.

The block's choose says which writer continues: first-pass (the first to pass the stages through through); score (the block's score command runs in each writer's checkout and prints one number, its last numeric line; the highest continues, a tie goes to the smaller diff, and a writer whose command fails or prints none is out); and, on a review stage, majority (each reviewer runs the review in the job's checkout with its own state directory; the review fails when more than half fail, and the findings handed back are those two reviewers report in the same words apart from case, spacing and a final full stop, plus each finding of a failed review that names a file and a symbol). The step's detail in the progress events names the writer that continued and its score, or how many reviewers failed.

merge (a writer stage; it needs score) scores the writers as score does, then, when at least two passed and were scored, runs one more session on the best writer's model in a checkout of the job branch's tip. It is handed a file (JOB_MERGE_FILE) listing every branch with its score and diff, and integrates them. Its result is scored the same way and continues when it passed and scores at least as well as the best single branch; otherwise the best branch continues and the merge's branch is kept as archive/job-writer/<job>/<stage>-merge. The merge session is recorded as writer 0 in the stage's writers. With score or merge, when the writers' scores differ or some changed files others did not, the runner writes writer-notes.md in the job's state directory and every later stage gets JOB_WRITER_NOTES naming it.

Each writer (and reviewer, and the merge session) of a fan_out stage sends its own events (docs/job-events.md: a writer event when it starts and when it passes or fails, with its number, the stage's writer count and its model; pi-writer.sh passes the writer's number to pi-session.sh, which sends the progress heartbeat under it). The dashboard lists each writer of the current attempt with its state, turns and last activity, and jobs/scripts/status-line.sh prints the same for the running stage; a running writer with no activity for 10 minutes is shown as stalled. In result.json each stage has duration_seconds, and each writer of a fan-out stage has started_at, ended_at, duration_seconds, turns, tool_calls and tokens (input plus output, totalled from the usage.jsonl that pi-session.sh appends to in the writer's state directory; a reviewer's sessions are not counted, so its usage is 0).

result.json also records which model did what. Each stage that ran agent sessions has sessions (one entry per session: its stage, model, thinking level, fan-out writer and round, with turns, tool calls and tokens) and thinking (the distinct levels, default when pi's own was used; a DeepSeek model defaults to high). summary counts sessions by stage and by model (writers' included), the rounds used against the failed-round limit, stage retries, the batch stage's candidates and compiled per model, and the run's turns, tool calls, tokens, wall-clock seconds and energy. authorship lists the job branch's commits since base_sha with the model of each (the Model: trailer a hook in pi-session.sh adds to every commit a session makes). A commit a script stage makes carries a Stage: trailer instead (the feature-map stage's commit has Stage: feature-map) and is listed as {"model": null, "stage": "feature-map", "kind": "script"}, counted as script:feature-map; a commit with neither trailer is unknown. Each entry has sha, model, stage and kind (model, script or unknown). authorship also gives the commit count per model (or script:<stage>), and per file the same keys for the commits that touched it. These are per commit and file, not per line.

A feature job may split its substrate writer stage into phases, each its own named, timed stage: test-cases, test-fixtures and test-ground-truth-binary (the test cases, the baseline fixture, and the ground-truth binary, one writer stage each), ground-truth (script), runner (generation runner, prompt, admission record, the loop and batch binaries), generation-loop (script), build (script, cargo test --workspace --locked --lib --bins) and gate (script, bench/verify.sh), then batch, admit, test and review as before. A failed generation-loop, build or gate has on_fail: "rework", so it goes back to the nearest writer before it, runner. A manifest may keep one substrate stage; no manifest on main uses the split. No stage writes a hand-written reference implementation or deliberately wrong candidates: candidates are scored against the browser-measured cases alone (docs/generation-loop.md step 2 gives the reason).

The model prompt and its check #

What the model prompt of a generated function carries. The model_prompt command builds it from three stored inputs, with no model session and nothing written per feature (crates/generation/src/model_prompt.rs build(), over the parts in crates/generation/src/prompt_parts.rs; docs/geometry-verification.md → "Cases from a feature's existing WPT pages" has the command and its arguments). The first aspect-ratio run sent a paraphrase, so the rule that a minimum beats a maximum, in CSS 2.1 sections 10.4 and 10.7 and referenced by CSS Box Sizing 4, never reached a model and 41 of 43 candidates got it wrong. A prompt holds, in this order:

• The whole readable text of the specification section the function implements, never an excerpt or a paraphrase, with the part that holds the rule being implemented between RELEVANT_BEGIN and RELEVANT_END (SpecificationBundle::render()).
• Each cited section whole (the section only, not the specification around it), under its own heading. The cited sections are the references/ files of the folder the specification_text command writes, one per --reference argument given to it (docs/provenance.md → "The text files of one function"). That command reads the project's copy under corpus/specs and never the network, and refuses a section that is not cached. SpecificationBundle::from_dir() reads the folder: specification.txt, relevant.txt and every references/ file its sources.json lists, in the order listed, each checked against the SHA-256 recorded for it (read_listed()). A listed file that is missing or altered is an error naming the file, never an omission. A prompt holds at most 6 cited sections and 60000 characters of their text (MAX_REFERENCES, MAX_REFERENCE_CHARS); SpecificationBundle::new() refuses a larger set instead of trimming it, so a smaller set is named to specification_text. The folder's sources.json (each text's URL with its anchor, and its hashes) is copied beside the prompt unchanged, so a run can be reproduced.
• The types file, byte for byte, under # Implement this exact signature. Every field's doc states what the input means: its unit, which box it measures (content box, or the box's own box-sizing box with padding and border), whether it may be absent or auto, and whether it may exceed another input. A doc never contradicts the specification and states no rule about which of two limits wins; that is the function's job.
• # Contract (model_prompt.rs contract()), the one text that is not an input: the module a candidate imports the types from, that it redefines none of them, that a path the types file writes as crate::... is <crate>::... in a candidate, and the name and type of the one function called. The names come from the command's --function and --resolver arguments (Function::new()).
• Every kept case of the cases folder, in case id order (groundtruth.rs load_all()), each shown as, for each box the engine calls the function for, the inputs it hands the function and the browsers' content box for that box as the expected output (render_function_case(); docs/geometry-verification.md → "Building the prompt" says how the calls are recorded), with the case's page before the calls unless the command is given --no-page-html (prompt_parts.rs PageHtml). The feature jobs give it: the function never sees a page, only its inputs, and a WPT page carries text that points away from the rule (a <link rel="match"> to a reference page the model cannot see, a title naming the test's intent, <head> boilerplate). Measured on the aspect-ratio job's stored inputs (64 train cases, 16 held out, kimi-for-coding, 8 candidates each, then bench/repair-loop.sh for up to 6 rounds, the current function passing 59 of 64 as the bar): with the pages the prompt was 27,251 tokens, 6 of 8 candidates built, the best first reply passed 49 and the best repaired one 55 of 64 (16 of 16 held out, 5 rounds); without them 19,640 tokens, 7 of 8 built, the best first reply passed 53 and no repair rose above it (8 of 16 held out, 3 rounds). No reply was cut by a guard in either first batch. Two batches of 8 from one model are a small sample; the first-pass numbers, which set each run's ceiling, point to the form without pages, and the held-out number points the other way. Only a function whose calls the engine can record (model_prompt.rs recorder_for()) has a prompt. The contract ends by saying what a reply holds: one rust-tagged fenced block, the last one defining the function being the candidate (model.rs extract_rust_source()). A case whose calls the browsers agree on none of is refused, and so is a folder with no case. A case was once also shown as the whole page with every node's box under it (render_case(), kept for reading a cases folder); with that form no candidate compiled (0 of 13, against 4 of 11 with the calls), so no prompt uses it. sample_by_recipe() picks one case per recipe name (not per failure category) plus one per recipe of the regression set, for a builder that samples from a recipe corpus; the command does not call it.

The functions generated before the command keep their per-feature builders in crates/generation/src/prompt.rs (build_*); none of them is built by the command.

A reply is cut off only by a runaway: MAX_COMPLETION_TOKENS in crates/generation/src/model.rs is 64000, because a longer prompt gets longer replies, a reasoning model's reasoning counts against the cap (kimi-for-coding spent 15,000 to 32,000 tokens on a reply with about 2,000 tokens of code), and a reply cut off mid-source is a certain compile failure that says nothing about the model.

job-check prompt <prompt folder> <specification folder> (jobs/src/checks.rs prompt()) fails the job before any model call when the prompt lacks a specification text or an expected result. It reads prompt.txt and expected.txt from the first folder, where model_prompt writes them, and the specification texts from the second, which is the folder specification_text wrote: no copy of a specification text is kept beside the prompt. It fails (exit 1) when:

• the specification folder is not one to compare with (specification_texts()): sources.json is missing, is not JSON or has no texts list; an entry lacks file or text_sha256, names a file that is not specification.txt, relevant.txt or a file directly under references/, or names a file twice; a listed file is missing or does not hash to its text_sha256; or specification.txt or relevant.txt is not listed;
• the prompt lacks the whole of specification.txt, does not hold relevant.txt between RELEVANT_BEGIN and RELEVANT_END, or lacks the whole text of any cited section the folder lists (the message names the references/ file);
• the prompt lacks an expected-result block of expected.txt, or expected.txt holds no block.

It cannot run (exit 2) when prompt.txt or expected.txt cannot be read. Whitespace is ignored in every comparison (squeeze()). A stage's checks entry {"check": "prompt", "args": ["<prompt folder>", "<specification folder>"]} runs the check when the stage passes, each argument resolved against the job's working directory (jobs/src/runner.rs run_checks(); jobs/src/manifest.rs CHECKS gives the check its two arguments). job-check manifest refuses a manifest whose model batch (a stage running run-batch.sh) has no earlier stage with that check (manifest.rs prompt_check_violations()). No manifest under specs/ runs a model batch, so none carries the check.

Two things the check does not do. It compares text, so it cannot tell whether a types file's docs agree with the specification; the review carries that. And it compares the prompt with the specification folder it is given: the sources.json beside the prompt records, by URL and SHA-256, the texts the prompt was built from, and the check does not read it.

The types file's check. job-check types <file> (jobs/src/checks.rs types(), rules in jobs/src/types_file.rs check()) is the output test of the step that writes a generated function's types file, the part that needs no checkout. The file must parse as Rust (syn); declare pub types and exactly one resolver type (pub type X = fn(&Inputs) -> Outputs) and nothing that runs (a fn with a body, an impl, a mod, a macro, a const or static each fail, named with their line); open with a module doc that states Scope with an In: and an Out: list; and document every struct, enum, variant and field, where a number's doc names its unit (px, unitless, or "the same unit ... as" another field) and a flag's doc opens with Whether. The stored aspect-ratio types file passes (2 structs, 17 fields). Not checked here, because they need more than the file: that it compiles against the engine (cargo check -p layout with the file in place, the engine-wiring step's test) and that no sentence in it contradicts a measured case (a reading). A stage may name it as {"check": "types", "args": ["<file>"]}.

Request and score, each alone from stored inputs. request_candidates --prompt <prompt.txt> --count <n> --out <dir> (crates/generation/src/bin/request_candidates.rs) sends a stored prompt to the gateway the environment names (WEBENGINE_MODEL_URL, WEBENGINE_MODEL or WEBENGINE_MODELS, WEBENGINE_MODEL_API_KEY), one request per seed, and writes each reply as it came under replies/, the Rust it carries under sources/ (model.rs extract_rust_source()) and requests.json with each request's model, seed, tokens, wall clock and finish reason or error. score_candidates --function aspect-ratio --sources <dir> --cases <dir> --work <dir> --out <report.json> (crates/generation/src/bin/score_candidates.rs) takes such a sources/ folder, or any folder of <name>.rs candidates, writes them into one crate (candidate_crate.rs Seam::AspectRatio), compiles it and scores every candidate over the measured cases with aspect_ratio_runner.rs evaluate(), every other generated function running as landed; the report is runner::Report. Two hand-written candidates are stored under crates/generation/tests/fixtures/aspect-ratio-candidates/ (its NOTICE) so the scoring step runs with no model: over the 33 measured train cases the baseline passes 12 and a first attempt at CSS Box Sizing 4 §4.2 and §4.4 passes 32. Before the crate, score_candidates compiles each candidate on its own (candidate_crate::probe); one that does not compile is left out of the report and its compiler messages go to <work>/compile-errors/<name>.txt, so each batch wants a --work folder of its own.

Repair, alone from stored inputs. repair_candidates (crates/generation/src/bin/repair_candidates.rs) asks the model once more for each candidate that is wrong, in one user turn (compile_repair::user_turn): the prompt, the candidate's source alone in a rust block as code to fix (compile_repair::source_block; not the reply it came from, whose reasoning can run to ten times the code), and what is wrong (compile_repair.rs). Sent as the model's own earlier answer instead, the source came back from Kimi unchanged in most rounds. With --errors <work>/compile-errors it repairs every candidate that did not compile, showing its compiler errors by line and column (compile_repair::message); with --report <report.json> --cases <dir> --function <name> --resolver <path> it repairs every candidate that fails a case, showing each failed case's boxes that come out off the browsers' majority and, for a box the function is called for, the call's inputs and expected output (compile_repair::case_message; the scorer fills CandidateReport::failing_case_diffs, aspect_ratio_runner.rs vote_diff()). It writes replies/, sources/ and requests.json as request_candidates does, each candidate under its own name, so its output is the next score_candidates --sources and the next repair's --batch; given both --errors and --report, one run repairs both kinds. Each request moves the candidate's seed on (compile_repair::next_seed), since the same request with the same seed comes back the same, and each record in requests.json lists the keys of every repair request the candidate was sent (asked, compile_repair::request_key of the source and the message); a candidate whose next request would repeat one of them is dropped, since its last repair came back failing the same way. A repair reply's source is its closed rust block (model.rs extract_tagged_rust_source()); a reply with none keeps the candidate's previous source rather than compiling its prose, and a compile repair shows at most the first compile_repair::MAX_ERRORS_SHOWN errors with a count of the rest. A case repair often breaks compiling (5 of 12 on aspect-ratio), so each one is followed by a compile repair. bench/repair-loop.sh runs the rounds: it scores the latest folder, stops when a candidate passes every train case, when the best train score has not risen for 3 rounds (box run 034eWRT4E70xnVxCtwTumo reached 60 of 62 at round 8 and rounds 9 to 12 added nothing), or when a round limit is met, and otherwise repairs every wrong candidate into <batch>-round<K>, keeping each round's folders (its score report in <batch>-round<K>-work/report.json, the batch's in <batch>-work/); the held-out cases are scored for the final candidate alone. Each round repairs from the better of every candidate's previous and repaired versions: a repair that passes fewer train cases, or breaks compiling, is set aside (a tie keeps the repair) (<batch>-round<K>-work/kept/), and the next request for that candidate opens with compile_repair::SET_ASIDE_NOTE and shows the next slice of its failing cases (compile_repair::next_slice), so it is not a request already sent. On aspect-ratio with qwen3.8-27b-fast, a candidate that did not compile went 23, 30, 32 and 33 of 33 train cases over one compile repair and three case repairs, and passes 6 of the 7 held-out cases.

A repair asks for the whole source again, not SEARCH/REPLACE edits to it. Edits were measured on box replays of folknet run 034fcGTHF0EaocdU3L3eUC's first batch with kimi-for-coding (53 train cases, the engine's function 50): whole source reached 49 train and 8 of 10 held out; edits only, skipping an edit that did not match, 48 and 5 of 10 (edits applied in 35 of 36 replies); edits with whole source allowed, 46 and 8 of 10. In every form a repair of a source at 45 or better came back at 33 to 47, from a wrong fix rather than from rewriting lines it was not asked to touch, so edits confine nothing that matters. Not measured with a job model on folknet.

A job lands a candidate that improves on the function the engine has, even when it misses train cases: getting a partial function landed for a feature comes first, and later jobs raise it. Before round 0 the loop scores the engine's current function on the job's own train cases (crates/<crate>/src/generated/<module>.rs for --resolver <crate>::<module>::<Type>, or --current); when that file is missing or does not compile against the types file in place, it scores the hand-written fallback instead (<crate>::<module>::resolve_baseline when that module declares it, or --fallback). <batch>-work/current.json records which one set the bar (scored: current, fallback, or none when neither could be scored, which counts as passing no case) and the cases it passes. A candidate clears this landing bar when it fails no train case the scored function passes and passes more train cases than it, or every one. The <name>.floor.json floors are not the bar: they count cases on generated_score_check's corpus, not the job's train cases. The final candidate is the one clearing the bar with the most train cases, or, when none clears it, the one with the most. The loop exits 0 when the final candidate clears the bar and 1 when none does, and jobs/scripts/store-candidate.sh refuses (exit 1) a final candidate below it, so a job whose repairs fall short stops at repair. A candidate that clears the bar but misses train cases is stored as partial: its commit subject ends ; partial, train X of Y, current function B, its body has a Known failures: <case ids> line, and it writes <batch>-store.json for the review, whose --known-failures turns a fail resting only on those cases' inputs into a pass with verdict_was fail, since no round is needed for it (specs/aspect-ratio.md, the review stage). The review does not decide the landing of a feature job: its review stage is advisory ("advisory": true, jobs/src/runner.rs rule 22), so a fail that stands starts a round while one can add a case, and otherwise the job lands the candidate on the score bar, with the review's findings committed beside the function as known-failures/<stored name>.json (each wrong input, the failing train cases it matches, and whether it needs an input the inputs struct lacks), a list later jobs work through. Landing on the score bar alone puts nothing into the engine: the job's admit stage, after the review, runs only when the review ran and passed (jobs/src/runner.rs rule 23), and is passed without running otherwise. When it runs, jobs/scripts/admit.sh --stored-dir scores the stored candidate on the score check's own corpus against the committed function, and when it passes more cases commits it as the generated function with its types file, provenance record and doc rows (no floor file, which bench/land.sh would refuse); the feature-map stage after it writes the job's generated rules. A later round whose repair or store falls below the bar lands the candidate the earlier round stored. In a review round after the first, store replaces the candidate an earlier round of the same job stored: the paths under its folder that the job's earlier Stage: store and Stage: review commits (those beyond JOB_BASE_SHA) touched, and NOTICE, go back to the job's base before the new candidate is written, and the commit body has a Replaces: line, so a job branch carries one candidate, the one its last review judged. When such a run lands, jobs/scripts/poll-and-land.sh adds partial, train X of Y, N known failures to the landed record's message (and, after an advisory review that failed, review failed, landed on the score bar, N known failure(s) from the review), the run's history record runs/<name>/<run id>.json carries candidate (train_passed, train_run, misses, known_failures), and bench/after-land.sh raises the function's floor, or writes its first (bench/generated-floors-update.sh), so a later job cannot lower it.

A job sent from the development box with its models replaced may go on below the bar: JOBS_PASS_FAILING_CASES=yes on submit.sh (refused without JOBS_MODEL or JOBS_BATCH_MODELS, sent as PASS_FAILING_CASES=yes and exported to the stages by bench/dagu/job.yaml) means "store a candidate below the landing bar, never land the run". The loop then exits 0 whenever there is a final candidate, and store commits one below the bar with a NOTICE paragraph and a commit subject that say it misses cases (train X of Y) and was stored under JOBS_PASS_FAILING_CASES. Those runs, against Kimi with reasoning off, test the job's later stages and its lifecycle, and their landing is checked, not done, even when the candidate cleared the bar: jobs/scripts/poll-and-land.sh runs bench/land.sh --check for a passing run whose record has pass_failing_cases, merges and tags nothing, and closes the record as checked. A function goes into the engine only from a folknet job, which never carries the setting. A loop with no final candidate (none compiled) still exits 1; one stopped by a round that gave no source exits by the same rule as any other stop, on its final candidate so far.

How it is shipped and started #

Existing pieces, and where each falls short of an unattended run:

Existing	What it does	Falls short
bench/remote-lane.sh	ships a bundle of commits not on origin/main, a diff and untracked files in one ssh stream, and runs a command in a lane copy	carries the worktree's HEAD only, not a base ref; the command is foreground and ends with the connection unless the command detaches
bench/lane-writer-launch.sh	syncs a worktree, git checkout -B, pulls pi-tracks, runs its writer command (--tool from specWatch.lanes.tool) with the brief, prints {ssh, sshOpts, path}	starts one open-ended writer session with a brief, not a runner with stages; needs a clean local worktree (exit 5), so a dev-box worktree must exist first; refuses over unfetched lane commits, which forces a fetch before any relaunch
bench/lane-writer-fetch.sh	fetches the branch from the lane copy and fast-forwards	fast-forward only, so a local rebase makes every later fetch report "diverged"; fetches the working branch, so "done" is inferred from liveness and transcript age (spec-status.mjs writerFor(), isStalled()) instead of read from a commit
bench/lane-apply.sh	checks out the exact commit in a lane copy, applies the diff, exports WEBENGINE_ON_LANE=1	nothing stops a lane run from being cleaned under a live job (git clean -fdx); the writer launch guards it, a runner would need the same
spec-watch-launch.mjs	picks a lane, falls back to local	the local fallback is the failure state; exit 4 exists only because of it

The proposed submit does, from the development box and in this order: verify the spec sha is the one on main, verify depends-on specs are landed, bundle the job branch and the base tip, pick the lane with the lowest load, send the bundle and the manifest through the same single ssh stream bench/remote-lane.sh uses, and start the runner detached on the lane (its own session, a pid file and a lease file it touches every minute). Submit prints the lane path and returns. It never starts anything locally: with no lane answering, it exits non-zero and the job stays unsubmitted (the same rule bench/on-lane.sh --lane-only gives a model batch).

The runner is resumable: each stage writes its status into a state file in the lane copy, so a lane restart or a killed runner is handled by starting the runner again on the same copy, and it continues from the first unfinished stage.

What the development box does #

sequenceDiagram
  participant D as development box (script, no model)
  participant L as lane
  participant P as primary checkout
  D->>L: submit: bundle, manifest, start runner
  loop every few minutes
    D->>L: poll: lease, then git ls-remote result/name
  end
  L-->>D: result commit (fetch)
  D->>P: land: bench/land.sh name spec-sha tip
  P->>P: fast-forward, tag, map update, announcement

• Submit. One script per enabled spec whose dependencies are landed and whose slot count allows it.
• Poll. One ssh call per running job: read the lease, and git ls-remote the result branch. No lease refresh for ten minutes and no result: mark the job lost and submit it again (resumes). A result that says failed: record the stage, park the job.
• Land. Fetch the result branch and the job branch, check the result's spec sha, base sha and tip against what bench/land.sh will read, create the local worktree bench/land.sh needs from the fetched branch, run bench/land.sh <name> <spec-sha> <tip>, then bench/after-land.sh runs as it does today. bench/land.sh keeps its own gate rerun and its own protected-file list, so a wrong result cannot land anything the gate rejects.

The worktree in the primary checkout exists only for the length of the landing. Nothing writes a file there, so there is no handoff.md to conflict with the roster, no local rebase, and no local writer. This step needs no model session, so it is a plain script (cron) rather than a claude -p tick.

Left outside the job on purpose: writing and enabling specs (gap 1), which stays with a person or a coordinator session, and pushing main (gap 9), which needs a decision below.

How the gaps map onto the shape #

Gap	What it blocks in the shape	Closed by
1 Hand-written specs	a malformed spec gives the job nothing to run	a spec lint; specs stay outside the loop (below)
2 Map rules	the announcement stage	stage 8, and the manifest's feature_rules (done)
3 Floors by hand	stage 7 for every function	floors as data read by the checker
4 Prose not enforced	stages 2 to 5 being real	script stages with artifact checks
5 Batch bypass	the ledger, so the post's numbers	stage 4 always through the batch wrapper
6 Refusals	landing	lint on touches at submit; a job that would touch bench/ is not submitted
7 Dev-box tick	"only submit, poll, land"	the runner on the lane, review inside the job
8 Local fallback	"nothing heavy on the development box"	strict mode for unattended gates
9 Push by hand	public visibility only	a decision, not a blocker

Settled #

• The runner and its scripts may live outside bench/. The loop has to be unattended, and bench/land.sh refuses any diff under bench/, so a runner kept there could never be landed or changed without a person. The split is accepted: some scripts stay in bench/, the runner and its stage checks live elsewhere.
• A running loop does not write or enable specs. Specs are written by a person or a coordinator session, outside the loop. Finished features are reviewed afterwards, asynchronously, and a review that finds problems sends the feature back through the loop as a new or amended spec (a new spec sha, so a new job). Spec authorship stays with a reviewer; the loop's job is to take an enabled spec to a result commit. There is therefore no drafting stage in the job and no path for adding to specs/ from a track.
• Dagu ships, queues and supervises a job; job-run runs its stages as one step. A trial on a lane ran fixture jobs end to end through Dagu, and a real spec has since run the same way ("First real run" below) (bench/dagu/job.yaml, docs/infrastructure.md → "Dagu on a lane"): sent over its MCP server from the development box, working copy from the lane's mirror at a base on the forge, job/<name> and result/<name> published to the forge, a queue limit, a resume after a stop, and a restart of the lane. So the detached start, pid and lease handshake, ssh lease probe and lost status drafted in "How it is shipped and started" and "What the development box does" above are not built: Dagu starts and queues the run, its status replaces the lease probe, and a resend with the same parameters resumes the job. The queue's limit replaces specWatch.maxLive. job-run stays a single step rather than one Dagu step per stage, because a step per stage would rebuild in workflow YAML what jobs/src/runner.rs already does under tests: pinned tools, compiled-in checks, the per-role environment, the writer's relaunches, the summed wall-clock limit, the review verdict, the state file and the result commit and its push. What stays on the development box is tasks/job-submit-land.md: the repository's refusal checks before a submit, and the fetch, job-check result and bench/land.sh at landing.

First real run #

A real spec has since run the loop end to end, not only the fixture job the Dagu trial above used. Spec viewport-units (manifest specs/viewport-units.job.json: pi writer and review on deepseek-v4.1-flash through the gateway; stages write, test, measure, review) ran as Dagu run 034VMxPcXtPSIim2wljbuk on lane 0 on 2026-09-26 and ran every stage. The start-of-job check (jobs/src/preflight.rs) passed. write made two commits, one of them a new file, crates/layout/src/bin/viewport_units_check.rs, with 69 cases; test passed; measure compared all 69 cases across chromium, firefox and webkit, 69 passed and none dropped. review returned fail on a real defect: browser_verdict compares only adjacent browser pairs (names.windows(2)), so chromium and webkit are never compared directly, though the spec requires every returned dump to agree. job-run exited 1 and published job/viewport-units and a failed result/viewport-units; poll-and-land.sh --check reported the review failure. Logs landed in archive/job-logs/viewport-units/<run id>/ on the NAS share with exactly the files the archive handler in bench/dagu/job.yaml lists, and a scan of them found no keys, tokens, credentials, addresses, hostnames or personal home paths.

A first attempt at the same spec (034VKGQgP1MDBABX0D5rJB) wrote for 1899 s of a 2400 s limit, then failed at stage test: the run step of bench/dagu/job.yaml did not put ~/.cargo/bin on PATH, and job-run starts a script stage's command without a shell (jobs/src/runner.rs run_command()). The wall-clock limit counts across resumes, so the job was restarted from scratch with a 3600 s limit.

The two runs exposed gaps, each fixed since: no job-check on the development box (jobs/scripts/install-job-check.sh); bench/dagu-lane.sh without --lane reached a lane without Dagu, so it now defaults to WEBENGINE_DAGU_LANE or lane 0; bench/on-lane.sh --lane-only refused inside a job, so it and bench/remote-lane.sh now run in place under WEBENGINE_ON_LANE; cargo missing from the run step's PATH; a resend always went at the base tip while the lane resumes only at the base a job started from, so submit.sh now keeps base_sha in the record; requirements went unchecked until a stage needed them, so jobs/src/preflight.rs now exits 9 with every reason, read back by poll-and-land.sh through bench/dagu-lane.sh step-log; and a Cargo.lock missing a new dependency, which the job's --locked build refused while lane test runs had updated the lock file silently, so bench/verify.sh now tests with --locked. A restart from scratch was done by hand (deleting job/<name> and result/<name> on the forge, bench/dagu-lane.sh clean-jobs <name> for the lane's jobs-work/<name>/, bench/dagu-lane.sh prune-mirror, and moving the local record aside), so jobs/scripts/submit.sh --restart <name> now does all four and sends the job again at the base tip; clean-jobs itself never touches the forge.

A resend resumes at the failed stage inside the same round, at the base and spec blob the job started with, and it may carry a changed specs/<name>.job.json: a fixed stage command no longer forces --restart, which would delete job/<name> and discard every passed stage. When the manifest at the base tip differs from the one at the recorded base, submit.sh checks it with job-check manifest --spec, then sends MANIFEST_BASE=<base tip> along with the recorded base. The lane's prepare writes that manifest (the spec blob at it must be the recorded one) and job-run --changed-manifest compares it with its stored copy (jobs/src/runner/resume.rs check(), rule 19 of the jobs/src/runner.rs module doc). The new manifest replaces the stored copy only when every stage before the failed stage is the same in both, with the agent that stage ran under; the stage list from the failed stage on, those stages' commands, checks and agents, and every limit may differ. A stage that already passed and differs makes job-run exit 2 naming the stage and saying --restart is needed, and nothing runs. A changed spec blob needed --restart too, until a resend could start a job afresh with the stage cache ("Starting afresh instead of resuming" above).

Left open: jobs/tests/agents.rs reportedly fails inside a job because job-run exports JOB_AGENT_NAME, JOB_AGENT_MODEL and JOB_AGENT_KIND to every stage (unverified beyond the writer's own report); a failed review ends the job with review_rounds at 1, and a resend resumes at review, reviewing the same code again, so a finding does not yet reach a second writer round; submit.sh --restart has not yet restarted a real job; and the writer's claim to have run bench/verify.sh inside the job is unverified, since pi runs with --no-session and keeps only its closing text.

aspect-ratio: the first generated-function run, kept as a baseline #

The aspect-ratio job (its manifest, since removed, ran substrate, ground-truth, generation-loop, batch, admit, test, review) passed every stage in one round as Dagu run 034VztFIHkkBEpA6VFkhuR (base 34c51d5) and was landed by the scheduled poller at 96fe266, with its floor raised by bench/after-land.sh at ba2a910. The admitted function is a model candidate, model_28 from qwen3.8-27b-fast, at 109/109. Stage times: substrate 2h26m, ground-truth, generation-loop and batch under a minute together, admit 11 minutes, test 36 seconds, review 30 minutes; 3h09m from start to publish. The substrate took that long because its writer followed the spec's "Run, in order" list and ran the ground truth, the generation loop and the whole model batch itself (772 s of model time) before the job's own stages ran them again.

That run's commits (7d48c22 through ba2a910) are kept under tag archive/aspect-ratio-run1 and reverted on main, so the spec runs again from scratch as a comparison against writer stages that pass --task (jobs/tests/writer_task.rs), a brief that asks for narrow reading (jobs/scripts/pi-writer.sh BRIEF) and sessions that record their turns and tokens (jobs/scripts/pi-session.sh, events under state/pi/<stage>/events.jsonl). The first run's logs, its job record (job-record.json) and these stage times (baseline.txt) are under archive/job-logs/aspect-ratio/034VztFIHkkBEpA6VFkhuR/ on the NAS share; its result.json went with the forge's result/aspect-ratio, deleted at landing, and its content is in state/state.json and state/review.json there.

The rerun passed every stage in one round as Dagu run 034WA9rX2fx1UlV9szr3Bd (base 03948e5) and was landed by the scheduled poller at c362bc4. The admitted function is again a model candidate, model_34 from qwen3.8-27b-fast, at 109/109. Stage times, from the job event store, with the first run's in brackets: substrate 35m28s (2h26m), ground-truth 5 seconds, generation-loop 50 seconds, batch 5m48s, admit 18m32s (11 minutes), test 36 seconds, review 40m15s (30 minutes); 1h41m37s from start to publish (3h09m). The review session took 88 turns and 92 tool calls, with 392,574 uncached input, 43,221 output and 5,026,304 cache-read tokens, so the model server reuses the shared start of the context across turns. Admit grew because the substrate writer left the aspect_ratio entry in crates/generation/src/bin/generated_score_check.rs FUNCTIONS and a test in crates/generation/src/admit.rs for admit to restore: with them in place, bench/verify.sh would have run the ground-truth binary that the substrate's --task forbids. The writer stages' turns and tokens are not known, because the rerun's logs were lost: the archive script in bench/dagu/job.yaml failed to create its temporary folder on the NAS share, and step cleanup then removed the job folder, the only copy. cleanup now keeps the job folder until the archive succeeds, and the archive keeps each pi session's events.jsonl.

Proposed changes #

Each change lists what it makes harder later, and one tag: spec now (can be commissioned as a spec that lands through bench/land.sh), decision (needs a person to choose), action (needs a person to do something a track cannot).

1. Spec lint as a test. crates/harness/tests/spec_lint.rs, in the style of no_seam_word.rs, reads specs/*.md and tasks/*.md and enforces the drafting rules of gap 1 plus: touches never names bench/, specs/, tasks/ or the other protected paths, and a generated-function spec names docs/feature-map.json. Harder later: a wrong rule rejects every new spec until the test is fixed. Existing specs that break a rule need an allow-list entry, since a spec's sha must not change. Spec now.
2. Provenance check as a test. A test over every *.provenance.json under crates/layout/src/generated/ requiring a model other than hand-written unless an escalation is recorded, and cases_run above 0. Harder later: hand-written generated functions need a recorded escalation field. Unverified: whether the older hand-written modules would pass; an allow-list may be needed. Spec now.
3. Floors as data. Done differently from the plan below: one <name>.floor.json per generated function, read by generated_score_check.rs (which also names each function's cache binary), raised by bench/generated-floors-update.sh after each landing, and protected from a track by bench/land.sh refusing any diff that touches a *.floor.json (which replaces the comparison against main's table). bench/seam-floor.sh and seam_floor_check.rs are gone. tasks/floors-table.md is superseded.
4. Runner, stage checks and manifest. A new top-level directory, not bench/, so a spec can commission and land it, because bench/land.sh refuses any diff under bench/. Harder later: two places hold lane scripts, and the pinned-from-main rule must be kept or a track could edit its own judge. Spec now, in this order: manifest and stage checks first (also usable by hand), then the batch wrapper that always writes a ledger line, then the runner.
5. Submit and poll-and-land scripts. Submit reuses bench/remote-lane.sh's payload; poll-and-land wraps bench/land.sh. Harder later: bench/land.sh still expects a local track worktree, so the poller builds one at landing time; a later bench/land.sh change to land from a fetched ref would remove that. Spec now if placed with the runner (item 4); a change to bench/land.sh itself is an action.
6. Replace the model tick for genera. The cron entry runs the poll-and-land script instead of spec-watch-tick.sh. That is a change to pi-tracks (a shared package other repositories use), so it is opt-in by declaration in tracks.json, the way specWatch.landScript is. Harder later: the model tick's review and conflict handling are gone for these repositories, so every failure mode moves into the job result. Action (a pi-tracks change, and editing tracks.json, which bench/land.sh refuses).
7. Strict lane mode for unattended gates. bench/verify.sh and each landChecks command use bench/on-lane.sh --lane-only when a variable the poller sets is present, and submit refuses a local fallback. Harder later: with both lanes down nothing lands, which is the intended result. The landing half is done (WEBENGINE_LANE_STRICT=1, see "Hand changes still needed", item 2); the submit half waits for submit itself.
8. Asynchronous review and send-back. A person or a coordinator session reviews finished features after the result commit, outside the loop, and does not hold a landing. A review that finds problems writes a new or amended spec (linted by item 1), which re-enters the loop as a new job. Harder later: review lags the loop, so a problem found late is fixed by a follow-up spec on top of what already landed, not by holding the landing; and a reviewer has to be available for the loop's output to be checked at all. Nothing in the loop writes or enables specs, so specs/ and tasks/ stay protected by bench/land.sh. Action (a person or a coordinator session does the review; nothing to build in the loop).
9. Push main and tags. The poller already pushes a landed job's spec/* tag; main is still pushed by hand. Harder later: an unattended push publishes code no person has seen. Decision (docs/coordinator-on-folknet.md → "Who pushes" and "Open questions" hold the deploy-key design). Not needed for the loop.
10. Coordinator on folknet. With the development box reduced to submit, poll and land, moving those three duties to a guest is smaller than it was, and still needs a person to create it and sign in (docs/coordinator-on-folknet.md → "Cutover"). Action.
11. Fix the roster's stale tick text. The roster's "Resume Here" still says an owning session drives ticks. Nothing is made harder; the roster is on main and outside this track's scope. Action.

Suggested order: items 1 to 3 (Rust side) first, since they need no decision and make the present loop safer at once; then items 4 and 5; then actions 3, 6 and 7 together, since they change the same scripts. Item 8 runs alongside, as soon as the first job lands. The only open decision among these changes is item 9, which the loop does not need.

Drafted specs #

Each spec below is a draft with enabled: false; a person reviews it before it is enabled. Each does the least that stands on its own, and none changes an existing spec, a bench/ file or tracks.json. They are chained with depends-on in this order because they share crates/harness/, crates/generation/ and the workspace manifest.

Spec	Change above	What it adds
tasks/spec-lint.md	1	crates/harness/tests/spec_lint.rs, the drafting rules as a test, with a closed allow-list for older specs
tasks/provenance-check.md	2	crates/harness/tests/provenance_check.rs, a model and cases-run check over every committed record, with a closed allow-list
tasks/floors-table.md	3, Rust side	superseded by floors-follow-the-corpus (<name>.floor.json files); enabled: false
tasks/cutoff-reporting.md	(from the roster)	per-model counts in candidate_crate.rs that separate replies cut off at the token cap from complete replies that failed to compile
tasks/job-manifest-checks.md	4, first	a new jobs/ crate: the manifest and the checks on a stage's output
tasks/job-batch-wrapper.md	4, second	jobs/scripts/run-batch.sh, a batch that cannot end without a ledger line
tasks/job-runner.md	4, third	the resumable runner and the result commit
tasks/job-submit-land.md	5	submit and poll-and-land scripts around Dagu, which starts and supervises the runner
tasks/job-agents-setting.md	(swappable agents)	the model or agent behind each role as a setting of the job, not code

The runner lives in a workspace crate at the top level because the track gate is cargo test --workspace, so only a crate's tests run in it, and because bench/land.sh refuses a diff under bench/. A job's manifest is specs/<name>.job.json, written by a person with the spec; a feature job's manifest lists the keys it implements as feature_rules and its spec's touches name docs/feature-map.json (gap 1); spec-status.mjs lists only specs/*.md, so it ignores the file (unverified for any other reader of specs/).

Hand changes still needed #

Changes that need an edit under bench/ or to tracks.json cannot be a spec, because bench/land.sh refuses those paths in a track. Each is an action for a person (or an interactive session) after the spec named has landed; none is done.

1. Done: floors are files, not a hand-edited script. See "Floors as data" above; no action left.
2. Done: a strict-lane mode for unattended gates. With WEBENGINE_LANE_STRICT=1 in its environment, bench/land.sh runs verify through bench/on-lane.sh --lane-only and every landChecks command --lane-only, so with no lane answering the landing is refused and nothing of the gate runs on the development box (docs/infrastructure.md → "Build lane"; bench/land-test.sh scenario s1). bench/verify.sh was lane-only already. Interactive use, with the variable unset, keeps the checks' fallback. The unattended caller exports it through tracks.json's specWatch.pollEnv (item 5).
3. One announcement per feature, for landings that are not jobs. A feature job carries one feature, so a job landing posts once and that post is the feature's announcement (docs/web-features-map.md → "Announcements"); this item does not apply to it. A landing that is not a job (a plain track) can move several features, and bench/feature-announce.py bundles them into one message. Post one message per feature for such a landing, each with the track's numbers once. Why: the first real post (2026-09-25) bundled several features, which reads as one event.
4. bench/ledger.py carries the token-cap counts. Read the compile outcome: line that tasks/cutoff-reporting.md adds and add cut_off and complete_failed to each model's entry in a ledger line, and print them in the announcement. Why: an announcement that says "16 of 32 compiled" should say how many of the rest were cut off by the cap. Needs tasks/cutoff-reporting.md landed.
5. The scheduled run is the poller, not the model tick. tracks.json declares specWatch.pollScript: "jobs/scripts/poll-and-land.sh" with specWatch.pollEnv: {"WEBENGINE_LANE_STRICT": "1"}. pi-tracks bin/spec-watch-tick.sh reads the key: for a repo that declares it, each scheduled tick runs that script from the primary checkout under the tick's per-repo lock, with pollEnv exported, and starts no due check and no model session, so nothing creates worktrees or launches writers any more. The crontab line bin/spec-watch-schedule.sh installs is unchanged (still one per repo, still naming spec-watch-tick.sh); the poller's output lands in $HOME/.local/state/spec-watch/spec-watch.log between === poll and === exit lines. A malformed declaration, or a script that is missing or not executable, is logged as refused and runs no model tick in its place. Left, as actions: land the pi-tracks change (branch track/poller-cutover there) together with this tracks.json, then reinstall the schedule and restore this repository's cron line, which is commented out by hand while jobs run one at a time.
6. Landing from a fetched ref. bench/land.sh needs a local track worktree and branch; jobs/scripts/poll-and-land.sh builds both at landing time and removes them after a refusal. A bench/land.sh that lands from a fetched ref would remove that step. Optional.
7. Map rules for specs landed outside a job. A job writes its own rules (gap 2). Exact-key rules for anything that landed without that stage are added by hand, or for a job by job-check feature-rules specs/<name>.job.json on main; no job is owed this now that the aspect-ratio job definition is removed. The field list in tasks/job-manifest-checks.md does not yet name feature_rules.
8. The roster's stale tick text (item 11 above) and the ground-truth, generation-loop and map-update-dry-run stage checks, which tasks/job-manifest-checks.md leaves out until their artifact formats are read. The checks are specs later; the roster line is a person's edit.

Later stages (design only) #

Three larger pieces come after the job runs a real feature. None is drafted as a spec: each is either unbounded as it stands or waits on something not yet known, and a spec written now would commission code against a guess.

Performance runs #

A generated function is chosen on its score against the browsers' answers; two candidates that both score in full are not told apart. A performance run would tell them apart on cost: several candidates that are known correct on the same corpus, each measured for CPU time, wall-clock time and peak memory over many scenarios.

Shape, if built: a runner takes a function's committed candidate and the other admitted candidates from that function's batches, builds each into its own process, and calls the function in a loop over a scenario set (the function's own corpus plus randomly generated inputs from the function's cases module, whose random(seed, count) already exists). Each candidate runs in a fresh process pinned to one CPU with the first run discarded and a fixed repeat count, and reports user plus system CPU time and peak resident memory from the process's own accounting, and wall-clock time. It runs on a lane, alone, since a shared lane's load would be the noise. The output is one table per function.

Why not a spec yet:

• The cost of one call of a leaf function is microseconds, and what matters is its share of a whole layout, so the right scenarios are whole documents, not isolated inputs. Which documents represent real pages is not decided (decision).
• What the table is for is not decided: choosing between equal candidates, a regression check on later batches, or a number for the announcement. Each needs a different tolerance (decision).
• Whether losing candidates' sources survive a batch is unverified: each batch writes sources/ under its run directory on the lane, and nothing shows whether the run directory is kept.

The bounded first slice, once those are answered: one function, its committed candidate against the baseline fixture, wall-clock and CPU only, reported and not enforced.

Adversarial review #

The review stage asks whether the diff implements the spec. It does not test whether the winning candidate is right in cases the corpus does not contain. One model could review another's winning candidate: given the spec text, the function's signature and the candidate's source, and not the scores, it proposes inputs it believes the candidate answers wrongly.

Shape, if built: an opinion never blocks a landing, and only a counterexample the browsers confirm counts. The reviewer's inputs are run through the same ground-truth path as the function's corpus (*_ground_truth_cache, three browsers) and through the candidate; a finding is an input where the browsers agree with each other, the candidate disagrees with them, and the reviewer's claim matches the browsers. Confirmed findings go into the job's result and are not applied: turning them into corpus cases and a new or amended spec is for a person or a coordinator session outside the loop, since the loop does not write specs. The reviewer is a review role with its own agent (tasks/job-agents-setting.md), required to differ from the candidates' agent.

Why not a spec yet:

• Each generated function has its own case type in its own *_cases.rs, so a reviewer's output has no common format to be written in. The roster's Queue item seam-kit, which factors the per-function runners into one driven by a table, is where a common case format would come from; drafting the review before it means writing the format twice.
• Whether a counterexample should ever hold a landing (not only be reported) is a policy choice (decision); reporting only is the reversible default.
• The browser oracle's cost for many extra inputs is not measured (unverified).

The long-running learning loop #

The idea: across many features, record what worked (which models, counts and prompts led to admitted candidates, at what cost) and use it to set the next job's package, so the loop improves without a person tuning it.

Only the recording half is known enough to build, and the drafted specs already do it: the ledger line per batch (tasks/job-batch-wrapper.md), the token-cap counts (tasks/cutoff-reporting.md), the provenance record per function and the result commit per job. What to do with the record is not known: no real batch has yet written a ledger line, there are about a dozen generated functions to learn from, and it is not known which knob matters (the model pool, the count, the prompt, the token cap). A loop that changed its own settings would edit a job's manifest under specs/, which the loop must not write; so any tuning is a proposal for a person or a coordinator session to turn into a spec.

Next step, not a spec: after the first several real jobs land, read the ledgers and results by hand (an action) and decide which one number would have changed a decision. A read-only summary over the same files comes after that.

What this survey did not check #

• The design was unbuilt when this survey was written. Items 4 and 5 have since been built and a real feature has run through them ("First real run" above), which was the test this survey proposed.
• The roster's counts (seven landings, one by hand, no announcements, no ledger line) are taken from the roster; the tick log was read only for its tail and the transforms2d lines. Rechecking against the whole log needs no decision (action, a read of $HOME/.local/state/spec-watch/spec-watch.log).
• Where the transforms2d batch ran (development box or lane copy): unverified. Settled by reading the writer's transcript on lane 1 (action, needs lane access).
• Whether the lane's mirror can hold a base ref and be rebased there: bench/remote-lane.sh bundles origin/main..HEAD and bench/lane-apply.sh checks out the exact commit, but neither was read for what refs the copy keeps. Settled inside item 5 (spec now).
• Whether older hand-written provenance records would pass the check in item 2: unverified, settled by running it (spec now, inside item 2).
• bench/land.sh past its option parsing and helper functions (the gate, rebase and finish steps) was read from the header comments, not line by line; refusals and their order are as the header states them.
• Checks that do not exist yet. The proposed stage checks name artifacts (cases_run, ledger lines, map events) that are read from committed formats, but no checker for them exists; their fields were read from crates/layout/src/generated/table.provenance.json and bench/ledger.py's callers only.
• The drafted specs were not built or run. They were written against the files they cite, read while drafting (bench/land.sh, bench/on-lane.sh, bench/remote-lane.sh, bench/batch-lane.sh, bench/ledger.py, candidate_crate.rs, the provenance records). Their test lists are proposals; whether each is enough is settled when a track builds it (spec now). Two facts they rely on are unverified: that the lane's mirror keeps result/<name> and the job branch fetchable after a lane copy is swept, and that the lane's shared store is writable for the ledger at the time of a real batch.