diff --git a/docs/geometry-verification.md b/docs/geometry-verification.md index b210fe42..6fbbb146 100644 --- a/docs/geometry-verification.md +++ b/docs/geometry-verification.md @@ -96,7 +96,7 @@ flowchart LR seamc["per-function corpora
104–109 cases each"] --> gate grammar["grammar corpus
200 cases"] --> held["held-out exam"] wpt["WPT normal-flow pages
7 usable"] --> held - fwpt["a feature's own WPT pages
aspect-ratio: 55 cases"] -. "four pages in five: 47" .-> gen["cases for generating
that feature"] + fwpt["a feature's own WPT pages
aspect-ratio: 40 cases"] -. "four pages in five: 33" .-> gen["cases for generating
that feature"] fwpt -. "one page in five: 8" .-> held ``` @@ -153,7 +153,7 @@ The rule, and why: - **The pages that already exist are used before any case is written.** In `aspect-ratio` job run `034aWbHrNquwSP4TyfOSr5` a model session took about 40 minutes to write a 993-line case generator, while the feature's WPT folder held 289 pages; selecting and - measuring those pages takes under ten seconds and gives 55 cases. + measuring those pages takes under ten seconds and gives 40 cases. - **A page supplies its markup and nothing else.** The expected result of a case is what the three browsers measure on the page as it was rewritten, by the same vote as every other corpus. A WPT assertion or reference page is never read, because it says whether @@ -184,21 +184,27 @@ node oracle/suites/feature-pages.mjs --feature oracle/suites/features/aspect-rat The first command lands the pinned WPT checkout; the feature's folder must be listed in `oracle/suites/suites.json` `wpt.sparse`. The feature file -(`oracle/suites/features/.json`) holds `source`, the folder's path inside WPT, and +(`oracle/suites/features/.json`) holds `source`, the folder's path inside WPT; `deny`, regular expressions for what the feature's job definition puts out of scope (for `aspect-ratio`: flex, grid, absolute and fixed positioning, floats, `writing-mode`, every -inline and table display, multi-column, `contain-intrinsic-*`, scripts). Each page goes -through four steps, and a page is recorded under the first step that rejects it: +inline and table display, multi-column, `contain-intrinsic-*`, the intrinsic sizing +keywords `min-content`, `max-content` and `fit-content`, which the engine has no sizing +for, and scripts); and `exclude`, optional, page name to reason, for a page out of scope +for a reason its markup does not state (for `aspect-ratio`: the two pages where the +automatic minimum width grows a ratio-derived width to a child's, which needs a +min-content width the engine does not compute). Each page goes through five steps, and a +page is recorded under the first step that rejects it: 1. A page with no doctype is rejected (`no-doctype`): browsers lay it out in quirks mode, and the engine always lays out in no-quirks mode (`crates/dom/src/parse.rs` `set_quirks_mode()`). 2. Every `

` holding the test's pass instruction ("Test passes if...") is removed, - because the engine lays out no text and step 4 rejects a page with any. The browsers + because the engine lays out no text and step 5 rejects a page with any. The browsers measure the page after this rewrite, so the expected result is true of the page the engine is given. -3. A page matching a `deny` pattern is rejected (`denied:`). -4. The rules of `oracle/suites/normal-flow-pages.mjs` `classify()` apply: no text, no +3. A page named in `exclude` is rejected with its reason (`excluded:`). +4. A page matching a `deny` pattern is rejected (`denied:`). +5. The rules of `oracle/suites/normal-flow-pages.mjs` `classify()` apply: no text, no external stylesheet, no image, no script in the body, no element other than `html`, `body`, `div`, `p` and `span`, and no `span` that is not shown to be `display: block`. @@ -243,11 +249,12 @@ output folder is removed and rebuilt on every run, and refused when it exists an The `scrollbar` rule drops a page on any such disagreement, including one about a box that cannot scroll: `scrollbar_dispute` compares content width with box width on every -element whatever its `overflow` value. Seven `aspect-ratio` pages are dropped that way, -each because Firefox alone reports content wider than a box whose `overflow-x` is -`visible` in all three browsers. The rule is kept as it is because the 55 cases left are +element whatever its `overflow` value. Seven `aspect-ratio` pages were dropped that way +before the intrinsic sizing keywords were denied, and one (`block-aspect-ratio-038.html`) +still is, each because Firefox alone reports content wider than a box whose `overflow-x` +is `visible` in all three browsers. The rule is kept as it is because the cases left are enough to bring up the steps that will read them; counting only boxes that can scroll -would keep the seven and give 62 cases from 63 pages. +would keep that page. **Measured on `aspect-ratio`** (`css/css-sizing/aspect-ratio/` at the commit `suites.json` pins; Chromium 153, Firefox 155, WebKit 26.6; the commands above, on the @@ -255,18 +262,21 @@ development box): | command | in | out | wall time | |---|---:|---|---:| -| `feature-pages.mjs` | 289 pages scanned | 63 selected (54 for generation, 9 held out), 226 rejected | 0.08 s | -| `pages_ground_truth_cache --list train` | 54 pages | 47 cases, 7 dropped | 4.1 s | -| `pages_ground_truth_cache --list held-out` | 9 pages | 8 cases, 1 dropped | 2.8 s | +| `feature-pages.mjs` | 289 pages scanned | 42 selected (35 for generation, 7 held out), 247 rejected | 0.08 s | +| `pages_ground_truth_cache --list train` | 35 pages | 33 cases, 2 dropped | about 4 s | +| `pages_ground_truth_cache --list held-out` | 7 pages | 7 cases, 0 dropped | about 3 s | -So 289 pages give 55 measured cases, 47 for generation and 8 held out, in about 7 s of +So 289 pages give 40 measured cases, 33 for generation and 7 held out, in about 7 s of measuring, of which about 2 s per run is starting the browsers. The binary builds in -1 min 26 s from an empty target folder. The 226 rejections are mostly layout modes outside -the job's scope (flex 57, grid 42, absolute or fixed positioning 26) and replaced elements -(28 with an image). Nine held out of 63 is 14%, under one in five, which follows from -these file names. Of the 8 dropped pages, 7 are the `scrollbar` pages above and one is a -`split` (`small-aspect-ratio-crash.html`: the root's height is 8, 16 and 33554432 px in -the three browsers). All 55 kept cases are unanimous: none is kept on a 2–1 vote. +1 min 26 s from an empty target folder. The 247 rejections are mostly layout modes outside +the job's scope (flex 57, grid 42, intrinsic sizing keywords 35, absolute or fixed +positioning 26) and replaced elements (18 with an image). Before the intrinsic sizing +keywords were denied, the same run selected 63 pages and measured 55 cases, 47 for +generation and 8 held out. Seven held out of 42 is 17%, under one in five, which follows +from these file names. Of the 2 dropped pages, one is a `scrollbar` page above +(`block-aspect-ratio-038.html`) and one is a `split` (`small-aspect-ratio-crash.html`: the +root's height is 8, 16 and 33554432 px in the three browsers). All 40 kept cases are +unanimous: none is kept on a 2–1 vote. **Building the prompt.** No browser, no model, no network; it reads three stored inputs: diff --git a/docs/oracle-coverage.md b/docs/oracle-coverage.md index a3849d56..e44efebf 100644 --- a/docs/oracle-coverage.md +++ b/docs/oracle-coverage.md @@ -191,7 +191,7 @@ WPT supplies test inputs; expected outputs come from live-browser runs, never fr assertions or reference pages. A feature's own WPT pages are case input for generating that feature, because they already exist and need no model to write them: the pages are selected by `oracle/suites/feature-pages.mjs` and measured in the three browsers by -`pages_ground_truth_cache` (289 `aspect-ratio` pages give 55 cases in about 7 s). One page +`pages_ground_truth_cache` (289 `aspect-ratio` pages give 40 cases in about 7 s). One page in five, chosen by a hash of its file name, is a fixed held-out partition used only for scoring, so an exam of pages that generation has not seen remains. The rule, the two commands and the drop rules: `docs/geometry-verification.md` → "Cases from a feature's diff --git a/oracle/suites/feature-pages.mjs b/oracle/suites/feature-pages.mjs index aee04086..5b4d7d1f 100644 --- a/oracle/suites/feature-pages.mjs +++ b/oracle/suites/feature-pages.mjs @@ -6,8 +6,10 @@ // // Usage: node oracle/suites/feature-pages.mjs --feature [--source

] --out // --feature the feature file (oracle/suites/features/.json): `source`, the WPT -// subpath of the feature's folder, and `deny`, regular expressions for what -// the feature's job definition puts out of scope +// subpath of the feature's folder; `deny`, regular expressions for what +// the feature's job definition puts out of scope; and `exclude`, optional, +// page name to reason, for a page out of scope for a reason no pattern in +// its markup states // --source read pages from this folder instead of the pinned checkout's `source` // (`oracle/suites/fetch.mjs wpt` lands it; the tests pass their fixture folder) // --out the folder to write; removed and rebuilt on every run, and refused when it @@ -19,9 +21,10 @@ // 2. every `

` holding the test's pass instruction is removed (the engine lays out no // text; the browsers later measure the page as rewritten, so the expected result // stays true to what is measured); -// 3. the feature file's `deny` patterns, in order and case-insensitive, against the +// 3. a page named in the feature file's `exclude`: `excluded:`; +// 4. the feature file's `deny` patterns, in order and case-insensitive, against the // whole rewritten page: `denied:`; -// 4. the rules of `oracle/suites/normal-flow-pages.mjs` `classify()`. +// 5. the rules of `oracle/suites/normal-flow-pages.mjs` `classify()`. // // Reference pages (`-ref`, `-notref`) and the `support/`, `resources/` and `reference/` // folders are not pages under test and are skipped, not rejected. @@ -61,6 +64,13 @@ if (!featureFile || !outDir) { const feature = JSON.parse(readFileSync(featureFile, 'utf8')); const sourceDir = flag('source') ?? join(HERE, 'checkouts', 'wpt', feature.source); const deny = feature.deny.map((pattern) => ({ pattern, re: new RegExp(pattern, 'i') })); +const exclude = feature.exclude ?? {}; +for (const [name, reason] of Object.entries(exclude)) { + if (typeof reason !== 'string' || !reason.trim()) { + console.error(`${featureFile}: exclude["${name}"] has no reason; each excluded page says why`); + process.exit(2); + } +} if (!existsSync(sourceDir)) { console.error(`no pages at ${sourceDir}: add "/${feature.source}" to suites.json wpt.sparse and run oracle/suites/fetch.mjs wpt`); @@ -90,10 +100,12 @@ function withoutCommentsAndDoctype(html) { return html.replace(//g, '').replace(/^\s*]*>/i, ''); } -/// `{ ok: true, html }` with the rewritten page, or `{ ok: false, reason }`. -function select(original) { +/// `{ ok: true, html }` with the rewritten page, or `{ ok: false, reason }`. `name` is the +/// page's flattened name, which `exclude` keys on. +function select(name, original) { if (!/ { + withOut((out) => { + const feature = { ...JSON.parse(read(FIX, 'feature.json')), exclude: { 'small-aspect-ratio-crash.html': 'needs a min-content width' } }; + const featureFile = join(dirname(out), 'feature.json'); + writeFileSync(featureFile, JSON.stringify(feature)); + const r = spawnSync( + process.execPath, + [join(HERE, 'feature-pages.mjs'), '--feature', featureFile, '--source', join(FIX, 'pages'), '--out', out], + { encoding: 'utf8' }, + ); + assert.equal(r.status, 0, r.stderr); + const m = JSON.parse(read(out, 'manifest.json')); + assert.equal(m.rejected['small-aspect-ratio-crash.html'], 'excluded:needs a min-content width'); + assert.ok(!existsSync(join(out, 'small-aspect-ratio-crash.html'))); + assert.ok(![...m.train, ...m.held_out].includes('small-aspect-ratio-crash.html')); + assert.match(r.stdout, / 1 excluded:needs a min-content width\n/); + + // An excluded page with no reason is refused before anything is written. + writeFileSync(featureFile, JSON.stringify({ ...feature, exclude: { 'small-aspect-ratio-crash.html': '' } })); + const refused = spawnSync( + process.execPath, + [join(HERE, 'feature-pages.mjs'), '--feature', featureFile, '--source', join(FIX, 'pages'), '--out', join(dirname(out), 'other')], + { encoding: 'utf8' }, + ); + assert.equal(refused.status, 2); + assert.match(refused.stderr, /has no reason/); + assert.ok(!existsSync(join(dirname(out), 'other'))); + }); +}); + test('a missing --feature or --out prints the usage and exits 2', () => { const r = spawnSync(process.execPath, [join(HERE, 'feature-pages.mjs'), '--out', 'unused'], { encoding: 'utf8' }); assert.equal(r.status, 2); diff --git a/oracle/suites/features/aspect-ratio.json b/oracle/suites/features/aspect-ratio.json index e0d10d91..5a80f7a3 100644 --- a/oracle/suites/features/aspect-ratio.json +++ b/oracle/suites/features/aspect-ratio.json @@ -10,6 +10,11 @@ "display\\s*:\\s*table", "\\bcolumns?(-count|-width)?\\s*:", "contain-intrinsic-", + "\\b(min|max|fit)-content\\b", "