burrito.space / bez
This repository has no description
Star
Fork 0
Watch
bez docs web-features-map.md
21 kB
Markdown
at track/substrate-composition

The web-features map #

One committed, versioned file, docs/web-features-map.json, says for every feature in the web-features package and every browser-compat-data key it groups how much of it this engine implements, and the evidence behind each status. A second committed file, docs/web-features-events.jsonl, records what changed at each regeneration as JSON lines. Both are data only: a widget, a dashboard or a bot reads them; nothing here renders or announces anything.

Design: what already existed, and what was missing #

  • crates/wpt-votes (docs/wpt-votes.md). features.rs reads WPT's own WEB_FEATURES.yml files at a pinned WPT revision (WebFeaturesTree, a blobless clone under corpus/wpt-votes/wpt-repo) and the web-features package's compat_features (WebFeatures, fetched by URL at version 3.39.0). bin/join_features.rs rolls the wpt.fyi vote table up per browser-compat-data key (subtests, majority, frontier). What it provides is the WPT-to-feature join and the browsers' results; what it does not have is any result for this engine, which does not run WPT yet (docs/platform-areas.md -> "Tooling and embedding"). The vote table also needs about 2.4 GB of downloads and is built from whatever wpt.fyi reports as current, so it cannot be an input that regenerates on every merge.
  • crates/content-analyser/src/bcd.rs walks browser-compat-data's html and css trees for content analysis. It measures what pages use, not what the engine implements, so it is not an input here.
  • oracle/suites/dashboard.mjs with docs/feature-map.json (docs/dashboard.md) is the one place that records what the engine implements: an ordered rule map from browser-compat-data key patterns to generated / hand-written / linked / oracle only, each with an evidence pointer that must resolve. It counts leaf keys per area; it has no notion of a web-features feature, of "complete", or of history.
  • oracle/suites/entry-coverage.mjs measures how much of an area's browser-compat-data a conformance suite could check (tests that mention an entry). It is about oracle reach, not implementation, and needs suite result files that are not committed, so it is not an input either.
  • Generated functions carry a committed .provenance.json next to their source with a verification record (cases_run, cases_passed, regression_free). That is the only machine-readable pass record for anything the engine implements today.
  • docs/platform-areas.md names the areas; nothing mapped keys to them.
  • specs/dashboard-history-graph.md (disabled) adds history for the dashboard's overall percentages. This map keeps its own history as events rather than percentages and leaves docs/dashboard.json and the README untouched, so the two do not collide.

Missing, and built here: a per-feature and per-key status with a written rule for complete; WPT evidence per feature that is reproducible from a pinned revision alone; an area per feature; pinned package versions in a lockfile; a diff that emits events; and regeneration on every merge.

Inputs, all pinned or committed #

Input Where it is pinned Recorded in the snapshot as
web-features 3.39.0 oracle/package.json (exact) and oracle/package-lock.json inputs.webFeatures
@mdn/browser-compat-data 8.0.4 same inputs.browserCompatData
WPT revision bc93a431dd... docs/web-features-wpt.json inputs.wptRevision
The rule map docs/feature-map.json (shared with the dashboard) per key: via, rule, evidence
Generated-function verification crates/layout/src/generated/*.provenance.json per key: seam
Key-to-area rules docs/web-features-areas.json per feature: area
Engine commit the checkout the generator ran in (git rev-parse HEAD) inputs.engineCommit

The web-features version matches WEB_FEATURES_PACKAGE_VERSION in crates/wpt-votes/src/bin/join_features.rs; the browser-compat-data version is the one the dashboard already used. Bumping either is a change to oracle/package.json and a regeneration, and shows up as an input event.

WPT evidence. cargo run --release -p wpt-votes --bin feature_wpt_rollup <revision> writes docs/web-features-wpt.json: per web-features id, how many WPT test files WPT's WEB_FEATURES.yml assigns it, and the directories whose WEB_FEATURES.yml made the assignment. It needs only the WPT tree at that revision (a blobless fetch, no file contents beyond the rule files), so it reproduces exactly. "Test file" is wpt_votes::testfiles::is_test_file, an approximation of WPT's manifest rules (markup tests and .any.js / .window.js / .worker.js wrappers, outside support and reference material); it counts files, not subtests. At bc93a431dd: 66,682 test files, 49,496 of them under a web-features id, 858 ids. The engine's own WPT results would go in wpt.enginePassed, which is null for every feature because no such run exists yet.

Areas. docs/web-features-areas.json is an ordered list of { "pattern", "area" } rules, matched like docs/feature-map.json's patterns (first match wins; a pattern matches its key and everything under it; a segment may be * or end in *). Every area must be a heading in docs/platform-areas.md; the generator refuses to run otherwise. A feature's area is the one most of its keys map to (ties broken by name), so a permission name or an allow token a feature also lists does not move it; unmapped when none of its keys map. The rules are coarse by design: 66 features stay unmapped in the first snapshot (the Navigator grab bag, on-device AI APIs, and similar that have no row in docs/platform-areas.md).

The status rule #

Computed only from the committed inputs above. Ambiguity resolves toward partial or unknown, never complete.

Per browser-compat-data key, from the first docs/feature-map.json rule that matches it (oracle/suites/web-features-map-lib.mjs keyEntry):

  • none — no rule matches, or the rule is oracle only: nothing claims an implementation.
  • unknown — an implementing rule (generated, hand-written, linked) matches only because its pattern is a parent of the key or a wildcard. The parent is claimed; this refinement of it (css.properties.width.fit-content under a css.properties.width rule) is not individually evidenced.
  • partial — an implementing rule names the key exactly, but no committed pass record shows every case passing. Hand-written and linked rules carry no pass record at all, so they are always partial; a generated function whose own verification passed fewer cases than it ran is partial too.
  • complete — a generated rule names the key exactly, its evidence is a .rs file with a .provenance.json beside it, and that record ran at least one case, passed every case it ran, and was regression-free.

Per feature (featureStatus), from the counts of its keys' statuses:

  • complete — it lists at least one key and every key is complete.
  • partial — otherwise, at least one key is complete or partial.
  • none — every key is none.
  • unknown — anything else: it lists no keys (21 features in 3.39.0), or its only claimed keys are unknown ones. Flexbox is the example: display.flex sits under the hand-written display rule, which is not evidence that flexbox is started.

moved and split entries in web-features are redirects, not features, and are left out. A key web-features lists that the pinned browser-compat-data does not have (141 in 3.39.0 against 8.0.4) is kept and marked inBcd: false.

Consequence today: no key is complete, because every generated function whose verification passed every case (height, overflow, stacking, float, relpos) is not what docs/feature-map.json names for its keys, and the ones it does name (width, margin, absolute, percentage) passed fewer cases than they ran. The first rule that points an exact key at a fully passing generated function makes that key complete; a feature follows once all its keys do.

Schema, version 1 #

docs/web-features-map.json, top-level fields in this order:

  • schemaVersion — 1. Any change to a field's meaning or to the status rule bumps it.
  • statusRule — docs/web-features-map.md#The status rule.
  • inputs — webFeatures, browserCompatData, wptRevision, wptTestFileRule, engineCommit.
  • totals:
    • webShare — the one number: { complete, measurable, percent }, complete features out of measurable ones (every feature not unknown), percent to one decimal.
    • features — { total, complete, partial, none, unknown }.
    • keys — { total, complete, partial, unknown, none }, over the distinct keys all features list.
    • byBaseline — high, low, false (the web-features Baseline status), each a feature breakdown as above.
    • byArea — per docs/platform-areas.md area (and unmapped), a feature breakdown.
    • keysByDashboardStatus — the keys counted by the dashboard's own five classes, so the two views can be reconciled.
  • features — one line per feature id, sorted: { name, group[], baseline, status, counts: { keys, complete, partial, unknown, none }, area, wpt: { testFiles, directories[], enginePassed }, keys[] }.
  • keys — one line per browser-compat-data key, sorted: { status, via, rule?, evidence?, seam?: { provenance, casesRun, casesPassed, regressionFree }, inBcd? }. via is the dashboard's classification of the key (unreached when no rule matches), computed by the same code (oracle/suites/feature-map-rules.mjs), so a key's via here and its class in docs/dashboard.json never disagree.

One line per feature and per key makes every status change a one-line diff. The file is about 2.3 MB.

Where consumers read it #

  • A dashboard or widget reads totals.webShare for the headline, totals.features and totals.byArea for the breakdown, and features.<id> for a per-feature row. The README's coverage table (oracle/suites/dashboard.mjs) still counts browser-compat-data leaf keys over the whole of browser-compat-data rather than per feature; its classification code is the same module the map uses, and totals.keysByDashboardStatus is the bridge between the two. Rendering a line from totals.webShare into the README is a change to dashboard.mjs's README block, which specs/dashboard-history-graph.md also edits, so it is left for after that spec.
  • A bot reads docs/web-features-events.jsonl, below.
  • The landing announcement reads the events each landing appends and posts to the community Discord when a feature moves forward ("Announcements", below).

Events #

node oracle/suites/web-features-diff.mjs BEFORE AFTER prints one JSON object per line. BEFORE may be empty or missing (no earlier snapshot: every feature is reported as new). Every event has format (1), type, and at (the engineCommit of AFTER). Order: input changes, then totals, then features by id.

type Fields When
input input, from, to schemaVersion, webFeatures, browserCompatData, wptRevision or wptTestFileRule differs
totals from, to: each { webShare, features, keys } any of those moved
feature-status feature, name, from, to, counts a feature's status changed; from is null for a new feature, to and counts are null for a removed one
feature-counts feature, name, status, from, to its key counts moved but its status did not

A feature becoming complete is feature-status with to: "complete"; becoming partial is to: "partial"; losing completeness is from: "complete". Fixtures that exercise every type: oracle/suites/fixtures/web-features-map/ (before.json, after.json, expected-events.jsonl, rebuilt by make-fixtures.mjs).

docs/web-features-events.jsonl is the committed log: every regeneration that changed the snapshot appends its events. It starts with the events from no snapshot to the first one, so replaying it from the top reconstructs every feature's status.

Example: a bot that keeps a running count and announces newly complete features.

import { readFileSync, writeFileSync } from 'node:fs';
const log = readFileSync('docs/web-features-events.jsonl', 'utf8').trim().split('\n');
const state = JSON.parse(readFileSync('bot-state.json', 'utf8')); // { seen: 0, complete: 0 }
for (const line of log.slice(state.seen)) {
  const e = JSON.parse(line);
  if (e.type !== 'feature-status') continue;
  if (e.to === 'complete') { state.complete++; announce(`${e.name} is complete (${state.complete} features so far, at ${e.at.slice(0, 10)})`); }
  if (e.from === 'complete') state.complete--;
}
state.seen = log.length;
writeFileSync('bot-state.json', JSON.stringify(state));

It remembers how many lines it has read, so each run handles only the lines appended since; the count comes from the events alone and agrees with totals.features.complete in the snapshot at the same commit.

Keeping it current #

One entry point: bench/web-features-update.sh [--no-commit], run in the checkout to update. It regenerates the snapshot; if the content changed (anything except engineCommit), it appends the diff's events to the log and commits exactly those two files as web-features map: regenerate at <sha> (<n> events). Unchanged content writes and commits nothing, so a merge that touches no input adds no commit. On any failure it puts both files back and exits 1 with the reason on stderr.

  • Unattended landings. bench/land.sh runs it in the primary checkout after every merge, after the roster step, through bench/after-land.sh (MAP_ON_LAND, in finish_landing), which runs bench/dashboard-update.sh (the README's coverage table, docs/dashboard.md) before it and the announcement below after it. It is tried once; a failure of either update is named on the one output line (owed: web-features map or dashboard not updated), never undoes the merge and does not change the exit status. While the primary is dirty or off base it is owed like the roster step and a rerun tries it. bench/land-test.sh w1 covers the stand-in being called once per landing and never under --check, the real script committing a regenerated snapshot and its events after a merge that changes docs/feature-map.json, and a failing regeneration leaving the merge and a clean primary.
  • Hand landings. tracks.json declares bench/after-land.sh as its afterLand command, so the /land command (which lives outside this repository) runs the map update and then the announcement once in the primary checkout after every merge. A failed map update is reported in the landing report and leaves the merge in place; a failed announcement is one line and never fails the step.
  • By hand. node oracle/suites/web-features-map.mjs rewrites the snapshot; --check exits 1 when the committed one is stale against its pinned inputs.

Announcements #

Every landing that moves a web feature forward to partial or complete posts one message to the community Discord's #nev channel. bench/after-land.sh (both landing paths, above) notes how long the event log is, runs bench/web-features-update.sh, and hands only the lines that run appended to bench/feature-announce.py. A feature-status event counts when it moves a feature forward: from none, unknown or new to partial or complete, or from partial to complete. A regression (complete to partial) is not announced. No such event, no post.

Rules come with the job. A feature moves only through an exact-key rule (above), so a job that implements one lists the rules in its manifest: feature_rules in specs/<name>.job.json, each an exact browser-compat-data key with its status and evidence in this file's rule shape, beside expected_features (the web-features ids that must end partial or complete). The job's spec names docs/feature-map.json in its touches, or jobs/scripts/submit.sh refuses the job and bench/land.sh would refuse its landing. job-run adds a feature-map stage before review: job-check feature-map writes the rules into docs/feature-map.json (through job-check feature-rules, each directly above the first rule that would otherwise match its key), commits them, runs this map's update as a dry run, and fails the job when a listed key or expected feature does not end partial or complete. A missing or wrong rule therefore fails the job, not the announcement. job-check feature-rules specs/<name>.job.json run by hand on main writes and commits the same rules, for a job that landed without the stage. After the announcement, bench/after-land.sh runs bench/expected-features-check.py on the landed job's result.json: a declared feature that no event moved and that is not partial or complete gets one NOT ANNOUNCED line naming the fix; one already partial or complete gets an "extended ... nothing to announce" line.

The post names the features (linked to their webstatus.dev pages) and adds what the work behind them took, from the landed track's ledger (below): per model the candidates drawn and compiled, the best score and the completion tokens; the wall time from the track's first ledger entry or first commit, whichever is earlier, to the landing; lane CPU-hours; GPU-hours (each model's draw seconds times the cards serving its pool); and an energy estimate (GPU-hours times card watts plus CPU-hours times a per-CPU figure). Card counts and watts come from the gitignored rig inventory bench/rig-inventory.local.json (model_pools, lane_watts_per_cpu; shape in bench/rig-inventory.example.json), since hardware counts stay out of the repository; a model it does not list is left out of GPU time, and without it there is no energy figure.

The ledger. Every model batch appends one JSON line to ledger/<track>.jsonl on the lanes' shared NAS directory (bench/batch-lane.sh batch_measure_stop runs bench/ledger.py): start and end time, lane CPU-seconds, and per model the counts, tokens, draw seconds and best score, parsed from the batch binary's own output. No host names or addresses. The announcement reads it from NEV_LEDGER_DIR, or $NEV_LANE_SHARED/ledger where that is mounted, or else asks a build lane over ssh. The track is the landed worktree's name: bench/land.sh passes it; a hand landing finds the one track/* branch that the merge brought in, and without one the post names the features only.

Posting. POST <NEV_ANNOUNCE_URL>/announce with Authorization: Bearer <NEV_ANNOUNCE_TOKEN>, both from the environment or the gitignored .env; the bot (repository phenomena, announce.js) checks the body against Discord's embed limits and posts it. With either unset or empty the post is skipped with one line. A failed post is one line naming the HTTP status or that the service could not be reached; it never fails or undoes the landing, and neither the address nor the token is printed.

Trying it. bench/feature-announce.py --dry-run --events <file> --track <name> prints the JSON body instead of posting; bench/after-land.sh --dry-run does the same after a real map update. bench/feature-announce-test.sh checks the ledger line and the body against bench/fixtures/feature-announce/ and posts to a local stand-in server; bench/land-test.sh a1 covers a landing that posts, one that does not, and a refused post that leaves the landing in place.

Existing files changed #

  • oracle/package.json, oracle/package-lock.json — web-features 3.39.0, exact.
  • oracle/suites/dashboard.mjs — its rule loading and matching moved, unchanged, into oracle/suites/feature-map-rules.mjs, which it now imports.
  • crates/wpt-votes/src/features.rs — list_tree_files and WebFeaturesTree::deciding_dir, with a test; crates/wpt-votes/src/lib.rs — the new testfiles module.
  • bench/land.sh — the map step in finish_landing; bench/land-test.sh — a stand-in map script in the harness and scenario w1.
  • docs/dashboard.md — a pointer to this map.

How to verify #

  • node --test oracle/suites/web-features-map.test.mjs — the key and feature status rules, the area choice, the snapshot's shape, the diff tool against the committed fixtures, and that the committed snapshot reproduces from its pinned inputs.
  • node oracle/suites/web-features-map.mjs --check — the last of those on its own.
  • cargo run --release -p wpt-votes --bin feature_wpt_rollup bc93a431ddd2e301dece2c6346eb23858aa487a8 — rewrites docs/web-features-wpt.json byte-identically.
  • bench/land-test.sh w1 — the landing step.
  • cargo test --workspace — includes wpt_votes::testfiles and deciding_dir.