# How layout output is checked against real browsers The engine's layout is judged by one question: for a given page, does every box come out the same size and in the same place as in real browsers? This page explains how that is answered, starting from nothing. The exact wire format is in `docs/dump-format.md`. ## The idea in one picture ```mermaid flowchart LR page["test page
(HTML + CSS)"] --> chr["Chromium"] page --> ff["Firefox"] page --> wk["WebKit"] page --> bez["this engine"] chr --> vote["vote per
measurement"] ff --> vote wk --> vote vote --> cmp{"engine within
1/60 px of the vote?"} bez --> cmp cmp --> result["pass / fail
+ per-box error"] ``` The browser side runs once per page and is cached; only the engine side runs on every candidate. ## Step by step **1. Dump.** Each browser loads the page through Playwright (`oracle/oracle.mjs`) at a fixed 800×600 viewport, device pixel ratio 1, and walks every element in document order. For each it records the element's box rectangles (`getClientRects()`), a subset of computed style, and whether its content overflows. That list is a **dump**. The engine produces the same dump from its own fragment tree (`crates/harness/src/dump.rs`). **2. Compare shape first.** Both dumps must list the same elements in the same tree shape (tag, depth, parent). If they do not, the difference is in parsing, not layout, and the case stops there. **3. Compare sizes and relative offsets.** For each box: its width and height, and its position *relative to its parent*. Relative offsets mean one misplaced box shows up as one wrong number instead of shifting every box below it, so the first wrong box names the rule that is wrong. **4. Vote.** With three browsers, each measurement is classified (`crates/harness/src/compare.rs` `vote()`): ```mermaid flowchart TD m["one measurement,
three browser values"] --> q1{"all three agree
within 1/60 px?"} q1 -- yes --> u["unanimous:
that value is the reference"] q1 -- no --> q2{"two agree,
one differs?"} q2 -- yes --> o["2–1: the pair's value is the
reference, the third is named outlier"] q2 -- no --> s["split: no reference,
measurement skipped"] ``` Two browsers could only report "they disagree" and throw the case away; three can keep it and name who is off. **5. Discard what cannot be trusted.** A whole case is dropped rather than scored when the browsers disagree about whether some box has a scrollbar, because a scrollbar takes a platform-dependent width from everything inside it (`compare::scrollbar_dispute`, `docs/dump-format.md` → "Scrollbar probe"). Unanimous agreement that a scrollbar *is* there does not discard the case. **6. Score.** A case passes when every trustworthy measurement of the engine is within 1/60 px of the reference. 1/60 px is the coarsest grid any of the browsers rounds to (Firefox's); Chromium and WebKit use 1/64 px. The raw difference is kept too, as a finer-grained signal than pass/fail. ## A worked example Twelve nested boxes, each `width: 83.333%` of its parent, starting from 800 px (the `nesting-depth-12` recipe). At the innermost box: | source | width at depth 12 | | --- | --- | | Chromium | 89.671875 px | | WebKit | 89.671875 px | | Firefox | 89.66667175292969 px (its own 1/60 px grid) | | vote | unanimous — all three within 1/60 px; reference **89.671875 px** | | engine, exact arithmetic | 89.72098558887357 px — off by 0.049 px, **fail** | | engine, rounding to nearest 1/64 px at each level | 89.71875 px — **fail** | | engine, truncating to 1/64 px at each level | 89.671875 px — exact, **pass** | The browsers' shared answer told the engine exactly how they round: truncate toward zero on a 1/64 px grid at every step (`crates/dom/src/px64.rs`, `tasks/device-pixel-rounding.md`). On other boxes of the same shape Firefox's coarser grid drifts up to 0.0375 px from the other two, past the tolerance; there the vote is 2–1 and Firefox is named the outlier. That drift turned out to be a real web-compatibility bug (`docs/firefox-app-units.md`). ## Where the test pages come from ```mermaid flowchart LR recipes["recipe corpus
227 cases"] --> gate["development signal
and landing floor"] seamc["per-function corpora
104–109 cases each"] --> gate grammar["grammar corpus
200 cases"] --> held["held-out exam"] wpt["WPT normal-flow pages
7 usable"] --> held fwpt["a feature's own WPT pages
aspect-ratio: 40 cases"] -. "four pages in five: 33" .-> gen["cases for generating
that feature"] fwpt -. "one page in five: 8" .-> held ``` The dotted arrows are built and measured, and no feature job reads those cases yet ("Cases from a feature's existing WPT pages" below). - **Recipe corpus** (`crates/harness/src/cases.rs`). About thirty **recipes**, each a small page generator for one branch of the spec's case analysis (auto margins on one side, a negative margin collapsing with a positive one, a percentage against an auto-height parent...). Each recipe is a **category**, so scores are reported per category. 27 fixed cases plus 200 seeded-random draws from the same recipes (seed `0xC0FFEE`) make 227. Values are deliberately fractional (`33.333%`, 1/64 px steps), because round numbers hide rounding bugs. - **Per-function corpora** (`absolute_cases.rs`, `float_cases.rs`, `relpos_cases.rs`, `stacking_cases.rs`, `block_height_cases.rs`, `overflow_cases.rs`). The same recipe idea, one set per generated function, kept separate so adding cases does not reshuffle everyone else's random draw. - **Grammar corpus** (`crates/harness/src/grammar.rs`). Pages composed freely from the same properties, with no recipe steering them, so it contains shapes no recipe thought of. Never used to generate code — only to check it. - **WPT normal-flow pages.** The `css/CSS2/normal-flow` pages that the engine can read (no text, no external stylesheet; a no-break space counts as text): 7 of 1056 (`oracle/suites/normal-flow-pages.mjs`). An exam only: scored the same way, and no prompt or case set for generation holds one. - **A feature's own WPT pages.** The pages of the WPT folder written for one feature, selected and measured by code. Four in five are cases for generating that feature; the fifth is held out and only ever scored. The next section is the rule, the two commands that make the cases and the command that builds a prompt from them. - **SMT-sampled cases** (`crates/harness/src/smt.rs`). A handful of margin-collapse cases whose values a solver chose to hit rule boundaries. ### Cases from a feature's existing WPT pages A feature's WPT folder is a source of cases for generating that feature, not only an exam for it. Two commands turn the folder into ground-truth cases, a third builds the model prompt from the cases for generation, and none of the three runs a model: ```mermaid flowchart LR folder["WPT folder of one feature
pinned checkout"] --> select["feature-pages.mjs
no browser, no model"] select --> pages["corpus/NAME-pages/
rewritten pages + manifest.json"] pages --> train["pages_ground_truth_cache
--list train"] pages --> heldout["pages_ground_truth_cache
--list held-out"] train --> gt["corpus/NAME-wpt-ground-truth/"] heldout --> hgt["corpus/NAME-wpt-held-out-ground-truth/"] gt --> build["model_prompt
no browser, no model"] text["specification_text folder"] --> build types["types file"] --> build build --> out["prompt.txt, expected.txt,
sources.json"] ``` The rule, and why: - **The pages that already exist are used before any case is written.** In `aspect-ratio` job run `034aWbHrNquwSP4TyfOSr5` a model session took about 40 minutes to write a 993-line case generator, while the feature's WPT folder held 289 pages; selecting and measuring those pages takes under ten seconds and gives 40 cases. - **A page supplies its markup and nothing else.** The expected result of a case is what the three browsers measure on the page as it was rewritten, by the same vote as every other corpus. A WPT assertion or reference page is never read, because it says whether one test passes, not where every box is, and because the page measured is no longer the page the assertion was written for once its text is removed. - **One page in five is held out**, so each feature keeps an exam of pages that no prompt and no case set for generation has held. A page is held out when the first byte of the SHA-256 of its file name is divisible by 5 (`feature-pages.mjs` `isHeldOut()`), so the choice is the same on every run and does not move when the deny list changes. The two lists are measured into two folders, so nothing that reads the first can see a held-out page. - **The pages are what the suite's authors wrote.** They are not stratified by the specification's case analysis and not chosen for fractional values; the recipe corpora above are what covers those. No feature job reads the two case folders or runs the third command yet, and `aspect-ratio` has no job definition: the one that named a writer-built case generator and `corpus/aspect-ratio-ground-truth/` is removed (tag `archive/job-aspect-ratio` holds its last run's commits). **Selecting pages.** No browser and no model: ``` node oracle/suites/fetch.mjs wpt node oracle/suites/feature-pages.mjs --feature oracle/suites/features/aspect-ratio.json \ --out corpus/aspect-ratio-pages ``` The first command lands the pinned WPT checkout; the feature's folder must be listed in `oracle/suites/suites.json` `wpt.sparse`. The feature file (`oracle/suites/features/.json`) holds `source`, the folder's path inside WPT; `deny`, regular expressions for what the feature's job definition puts out of scope (for `aspect-ratio`: flex, grid, absolute and fixed positioning, floats, `writing-mode`, every inline and table display, multi-column, `contain-intrinsic-*`, the intrinsic sizing keywords `min-content`, `max-content` and `fit-content`, which the engine has no sizing for, and scripts); and `exclude`, optional, page name to reason, for a page out of scope for a reason its markup does not state (for `aspect-ratio`: the two pages where the automatic minimum width grows a ratio-derived width to a child's, which needs a min-content width the engine does not compute). Each page goes through five steps, and a page is recorded under the first step that rejects it: 1. A page with no doctype is rejected (`no-doctype`): browsers lay it out in quirks mode, and the engine always lays out in no-quirks mode (`crates/dom/src/parse.rs` `set_quirks_mode()`). 2. Every `

` holding the test's pass instruction ("Test passes if...") is removed, because the engine lays out no text and step 5 rejects a page with any. The browsers measure the page after this rewrite, so the expected result is true of the page the engine is given. 3. A page named in `exclude` is rejected with its reason (`excluded:`). 4. A page matching a `deny` pattern is rejected (`denied:`). 5. The rules of `oracle/suites/normal-flow-pages.mjs` `classify()` apply: no text, no external stylesheet, no image, no script in the body, no element other than `html`, `body`, `div`, `p` and `span`, and no `span` that is not shown to be `display: block`. Reference pages (`-ref`, `-notref`) and the `support/`, `resources/` and `reference/` folders are skipped, not rejected. The output folder holds the accepted pages, flat (a page in a subfolder takes its path with `/` replaced by `_`), and `manifest.json`: `source`, `wpt_commit`, the `train` and `held_out` lists, and `rejected` (page name to reason). The folder is removed and rebuilt on every run, and refused when it exists and holds no `manifest.json`, so a mistyped path deletes nothing else. **Measuring pages.** The three browsers, no model; once per list: ``` cargo run --release -p generation --bin pages_ground_truth_cache -- \ --pages corpus/aspect-ratio-pages --list train --out corpus/aspect-ratio-wpt-ground-truth cargo run --release -p generation --bin pages_ground_truth_cache -- \ --pages corpus/aspect-ratio-pages --list held-out \ --out corpus/aspect-ratio-wpt-held-out-ground-truth ``` The flags are read in that order. Each page of the list is dumped in every browser and written as one case in the format every `*_ground_truth_cache` binary writes (`crates/generation/src/groundtruth.rs` `CachedCase`), so `load_all()` and `prompt_parts.rs` `render_case()` read the folder unchanged. A case's id is the page's file name (`block-aspect-ratio-001.html`), its seed is 0 and its one category is `wpt`. `groundtruth.rs` `drop_reason()` decides whether a measured page is kept; the text before a reason's first `:` names the rule: | rule | a page is dropped when | |---|---| | `too-few-engines` | fewer than three browsers produced a dump, so there is no vote | | `shape-mismatch` | the browsers list different elements, so there is no vote | | `split` | on some measurement no two browsers agree | | `scrollbar` | the browsers disagree whether some box's content overflows it (`compare::scrollbar_dispute`) | A measurement two browsers agree on and the third does not is kept: the pair's value is the expected one. A dropped page is still written in full, as a case file under `/dropped/`, with every reason in `/dropped/reasons.json`, so the numbers behind a drop can be read; `load_all()` skips subfolders and so reads the kept cases only. The output folder is removed and rebuilt on every run, and refused when it exists and holds no `dropped/reasons.json`. The `scrollbar` rule drops a page on any such disagreement, including one about a box that cannot scroll: `scrollbar_dispute` compares content width with box width on every element whatever its `overflow` value. Seven `aspect-ratio` pages were dropped that way before the intrinsic sizing keywords were denied, and one (`block-aspect-ratio-038.html`) still is, each because Firefox alone reports content wider than a box whose `overflow-x` is `visible` in all three browsers. The rule is kept as it is because the cases left are enough to bring up the steps that will read them; counting only boxes that can scroll would keep that page. **Measured on `aspect-ratio`** (`css/css-sizing/aspect-ratio/` at the commit `suites.json` pins; Chromium 153, Firefox 155, WebKit 26.6; the commands above, on the development box): | command | in | out | wall time | |---|---:|---|---:| | `feature-pages.mjs` | 289 pages scanned | 42 selected (35 for generation, 7 held out), 247 rejected | 0.08 s | | `pages_ground_truth_cache --list train` | 35 pages | 33 cases, 2 dropped | about 4 s | | `pages_ground_truth_cache --list held-out` | 7 pages | 7 cases, 0 dropped | about 3 s | So 289 pages give 40 measured cases, 33 for generation and 7 held out, in about 7 s of measuring, of which about 2 s per run is starting the browsers. The binary builds in 1 min 26 s from an empty target folder. The 247 rejections are mostly layout modes outside the job's scope (flex 57, grid 42, intrinsic sizing keywords 35, absolute or fixed positioning 26) and replaced elements (18 with an image). Before the intrinsic sizing keywords were denied, the same run selected 63 pages and measured 55 cases, 47 for generation and 8 held out. Seven held out of 42 is 17%, under one in five, which follows from these file names. Of the 2 dropped pages, one is a `scrollbar` page above (`block-aspect-ratio-038.html`) and one is a `split` (`small-aspect-ratio-crash.html`: the root's height is 8, 16 and 33554432 px in the three browsers). All 40 kept cases are unanimous: none is kept on a 2–1 vote. **Building the prompt.** No browser, no model, no network; it reads three stored inputs: ``` cargo run -p generation --bin model_prompt -- \ --specification crates/provenance/tests/fixtures/specification-text/expected-output \ --types crates/generation/tests/fixtures/model-prompt/aspect_ratio_types.rs \ --cases crates/generation/tests/fixtures/pages-ground-truth/train \ --function resolve --resolver layout::aspect_ratio::AspectRatioResolver \ --out ``` Each case is shown as, for each box the engine calls the function for, the inputs it hands the function and the content box the browsers measure for that box, as the expected output (`prompt_parts.rs` `render_function_case()`), with the page before the calls unless `--no-page-html` is given, as the feature jobs give it (`docs/autonomous-loop.md` → "The model prompt and its check" has the measurement). The calls are recorded with the baseline answering, never the resolver the engine ships (`layout::fragment::layout_recording_aspect_ratio_calls()`, `aspect_ratio_runner.rs` `record_calls()`). A box whose `height` is not definite is asked twice, before and after its children; when the children's height depends on the first answer (a child's percentage height against it), the case shows both calls, the first with no expected output and the second with that height written as a rule in the first answer, measured by laying the page out with the first answer held at three values. The module doc of `aspect_ratio_runner.rs` says why ("A box asked twice"). For `aspect-ratio` the 33 train cases have 33 ratio boxes, every one with an agreed browser size, two of them shown with both calls, and the baseline already gives the expected output for 8 of them. A case was once also shown as its whole page with every node's box under it; with that form `qwen3.8-27b-fast` wrote no candidate that compiled (0 of 13, against 4 of 11 with the calls), so the command no longer builds it. Those are the inputs stored for the tests; a feature's own run names its `specification_text` folder, its types file and its `corpus/-wpt-ground-truth` folder. All six arguments are required, in any order, and none may be given twice; `--no-page-html` takes no value (`crates/generation/src/bin/model_prompt.rs` `main()`): | argument | what it names | |---|---| | `--specification

` | A folder the `specification_text` command wrote (`docs/provenance.md` → "The text files of one function"): `specification.txt`, `relevant.txt`, the cited sections under `references/`, and the `sources.json` that lists them. | | `--types ` | The function's types file. The prompt carries it byte for byte. | | `--cases ` | A folder of measured cases, such as one `pages_ground_truth_cache` wrote. Every kept case is shown, in case id order (`groundtruth.rs` `load_all()`); the cases under `dropped/` are not read. | | `--function ` | The name a candidate gives the function (`resolve`). | | `--resolver ::::` | The full path of the function's type as a candidate crate names it: the module a candidate imports the types from, then the type's name (`model_prompt.rs` `Function::new()`). Only a function whose calls the engine can record is accepted (`recorder_for()`: `aspect-ratio`). | | `--out ` | The folder the three files below are written into; it is created when missing. | `crates/generation/src/model_prompt.rs` `write()` writes three files into `--out`, in this order: | file | content | |---|---| | `prompt.txt` | The prompt (`build()`): the specification text with its relevant part marked, every cited section, the types file, the contract and every case with its calls and their expected outputs. `docs/autonomous-loop.md` → "The model prompt and its check" describes each part. | | `expected.txt` | Each call's expected output followed by a line `---` (`prompt_parts.rs` `expected_calls_file()`). | | `sources.json` | The `--specification` folder's own, byte for byte: which texts, by URL and SHA-256, the prompt carries. | No specification text is copied into `--out`: `cargo run -p jobs --bin job-check -- prompt ` reads `prompt.txt` and `expected.txt` from the first folder and the texts from the second, and fails when the prompt lacks one. Every input is read and checked before the first file is written, so a refused run writes nothing. Refused: a case with no three-browser vote (the message names the case), a `--cases` folder that is missing or holds no case, a types file with no text, a specification folder that `prompt_parts.rs` `SpecificationBundle::from_dir()` refuses (a listed text missing or not hashing to its recorded SHA-256, among others), and a `--function` or `--resolver` that is not a name or not a path with a module. A file of another name already in `--out` is left there. `--cases` is given the folder of the `train` list. The command reads whichever folder it is given and does not refuse a held-out one. For the stored inputs (one cited section, one case) the prompt is 22,764 characters and the built binary writes the three files in under 0.01 s on the development box. **Tests.** `node --test oracle/suites/feature-pages.test.mjs` runs the selector on real WPT pages stored under `oracle/suites/fixtures/feature-pages/`, one per outcome, and compares the manifest, the rewritten page and the printed summary; `bench/verify.sh` runs it. `crates/generation/tests/pages_ground_truth.rs` tests the keep-or-drop decision on stored cases with no browser, under `cargo test`. `bench/pages-ground-truth-test.sh` measures the fixture pages again in the real browsers and compares pages kept, pages dropped with their rule, and each kept page's majority boxes with the stored cases; it needs the three browsers and is run by name, not by the gate. `crates/generation/tests/model_prompt_command.rs` runs the `model_prompt` binary on the stored inputs under `cargo test`: the order of the prompt's parts, the three files written, two runs writing the same files, and each refusal. Its `the_stored_prompt_the_prompt_check_is_tested_on_is_what_the_command_writes()` fails when the binary's `prompt.txt` and `expected.txt` differ by a byte from `jobs/fixtures/prompt/model-prompt/`, the folder `jobs/tests/prompt_check.rs` runs the prompt check on; that folder's `NOTICE` holds the command that writes the two files again. ## Floors A **floor** is a recorded score that a change may not go below; landing on `main` runs them (`bench/land.sh`): - `bench/fragment-floor.sh` — the recipe corpus through `layout()` must pass 227 of 227. - `bench/verify.sh` (`bench/generated-scores.sh`) — every generated function must pass at least as many cases of its own corpus as its `.floor.json` holds, and the corpus must not shrink. Floors are raised when more cases pass (by `bench/generated-floors-update.sh` after a landing), never lowered to let a change through. ## Current numbers - Recipe corpus: **227/227**, worst per-box error 0.0000 px. - Held-out WPT normal-flow pages: **7/7**. Grammar corpus: **61/164** scored, 36 discarded for scrollbar disagreement. The gap between 227/227 and 61/164 is the honest measure of how much the recipes still miss. - Browser agreement: 699 of 705 browser pairs agreed across 235 documents; all six disagreements were the twelve-deep percentage shape above. History of these numbers: `roadmap.md` → "Proof of concept".