This repository has no description
bez docs generation-loop.md
5.3 kB

The generation loop #

How one layout rule goes from spec text to committed engine code. The unit of work is a generated function: one rule pulled out of the hand-written engine into a small plain-data function (docs/generated-functions.md). Everything below happens once per generated function, and again whenever the spec or the browsers change.

flowchart TD
  spec["1. Spec section<br/>(e.g. CSS 2.1 §10.3.3)"] --> types["2. Types and corpus<br/>(hand-built)"]
  types --> truth["3. Ground truth<br/>3 browsers, cached once"]
  types --> prompt["4. Prompt"]
  prompt --> gen["5. Model pool writes<br/>N candidates"]
  gen --> score["6. Compile and score<br/>against the cache"]
  truth --> score
  score --> admit{"7. Admit?"}
  admit -- "no: failures go<br/>into the next prompt" --> prompt
  admit -- yes --> commit["8. Commit source<br/>+ provenance record"]
  commit --> land["9. Land on main<br/>behind score floors"]
  1. Spec section. One algorithm from one spec section is the unit, because a whole algorithm is the smallest thing a browser can check. The spec text travels with the generated function and is hashed, so a later spec change can find the code derived from it (docs/provenance.md).
  2. Types and corpus. A person (or agent) writes the function's input and output types, a baseline that reproduces the old behaviour, a set of recipes (small HTML page generators, one per case the spec distinguishes), and two or more deliberately wrong candidates. The wrong ones must be rejected; if they are not, the corpus cannot tell right from wrong and needs more cases. This is the expensive, hand-made step.
  3. Ground truth. Every corpus page is rendered once in Chromium, Firefox and WebKit, and each page's box geometry is saved to a gitignored cache (crates/generation/src/groundtruth.rs). Nothing after this starts a browser. How the three answers become one reference is in docs/geometry-verification.md.
  4. Prompt. The spec text, the function's Rust types, and one failing case per failing category (crates/generation/src/prompt.rs).
  5. Candidates. A model draws N independent versions of the function — 8 per batch from a paid API, 32 at once from the self-hosted model pool (crates/generation/src/model.rs, docs/rig-capacity.md). Many cheap tries beat one careful one, because the browsers say which tries are right.
  6. Compile and score. Each candidate is compiled into a small probe crate against the function's types and run over every cached case inside the real engine, in under a millisecond per case. A case passes when every measurement matches the browsers' vote within 1/60 px.
  7. Admit. Two gates, both required (next diagram). A rejected batch's failures seed the next prompt; a batch loop stops itself when per-category scores stop improving or after six batches, and may escalate once to a stronger model first (specs/automated-repair-iteration.md, specs/escalation-tier.md).
  8. Commit. The winner is written under crates/layout/src/generated/ with a provenance record (spec URL, text hash, model, prompt hash, seed, score). From then on it is ordinary engine source; no model runs at build or run time.
  9. Land. The track merges to main through bench/land.sh, which runs cargo test --workspace and the score of every generated function against its own floor (bench/verify.sh), and bench/fragment-floor.sh (the recipe corpus must stay at 227/227). A generated function's floor is its <name>.floor.json, which bench/after-land.sh raises after a landing and nothing lowers.

Admission #

flowchart LR
  cand["candidate<br/>scores"] --> reg{"loses any case in<br/>the regression set?"}
  reg -- yes --> reject["rejected"]
  reg -- no --> best{"beats the committed<br/>candidate's score?"}
  best -- "no (tie or lower)" --> keep["committed one stays"]
  best -- yes --> admit["admitted and<br/>committed"]
  • Regression set. Every case any admitted candidate has ever passed. A new candidate may not lose one, so the loop cannot trade old correctness for new (crates/generation/src/admit.rs).
  • Beats the committed candidate. Strictly more cases passed; a tie keeps what is already committed (admit::beats_committed).
  • Margin collapsing only: a third gate checks the candidate's intermediate decisions against Datalog rules in crates/logic, because right geometry can come from wrong reasoning (docs/margin-collapse-experiment.md).

Where it runs #

flowchart LR
  subgraph cpu["CPU servers"]
    oracle["browser oracle<br/>(builds the cache)"]
    verify["compile and score"]
  end
  subgraph gpu["GPU model pool"]
    sample["candidate generation"]
  end
  sample -- "candidate source" --> verify
  verify -- "failing cases" --> sample
  oracle -- "cached ground truth" --> verify

Scoring is fast (272,930 cases/s at sixteen threads, docs/rig-capacity.md); writing candidates is the bottleneck. Hardware, gateway and tiers: docs/infrastructure.md.

When things change #

A spec edit, a browser release or a WPT update re-enters this loop at different points: see roadmap.md → "Ongoing development lifecycle".