This repository has no description
bez docs dashboard.md
11 kB
Markdown
at main

The coverage dashboard #

Explains what README.md's coverage table means, how docs/feature-map.json decides it, and how to keep both in sync after a change.

The five statuses #

Every browser-compat-data leaf key under css, html, api, javascript, svg, webassembly, http and mathml gets exactly one of:

  • generated — implemented by an admitted model-written candidate under crates/layout/src/generated/. Each generated function's .provenance.json names the spec section it was generated against; docs/feature-map.json's rule for the key points at the function's source file.
  • hand-written — implemented by substrate code the project wrote by hand: box tree construction, the DOM/cascade boundary onto Stylo, and the reflection/tree-mutation work in crates/dom.
  • linked — implemented by a real dependency the project integrates rather than writes, using docs/platform-areas.md's own word for it: Stylo for CSS cascade and computed values today, and a JavaScript engine once one is chosen and actually wired in. It covers only the feature surface the dependency itself computes. The project's glue and adapter code around the dependency — the DOM/cascade boundary in crates/dom/src/style.rs, the reads in crates/dom/src/computed.rs — is hand-written, and a key whose behavior beyond the cascaded value (layout, box generation) is done by this project's own code keeps that code's status: display and position are hand-written, width and the top/right/bottom/left offsets are generated, even though Stylo computes their values. The linked rules today are the remaining properties of the style subset compared against getComputedStyle (docs/dump-format.md), which docs/platform-areas.md → "Style, layout and rendering" reports as fully matched. Properties Stylo cascades but that no comparison checks stay unreached until one does, and javascript.* stays unreached while no JavaScript engine exists.
  • oracle only — not implemented, but a three-engine-comparable probe or conformance suite exists for it (docs/oracle-coverage.md, docs/conformance-oracles.md).
  • unreached — none of the above: no rule in docs/feature-map.json matches the key. This is the default, and today it accounts for most of the surface — see "Expect small numbers" below.

The rule map #

docs/feature-map.json is the one hand-maintained input: an ordered array of rules, for example:

{ "pattern": "css.properties.width", "status": "generated", "evidence": "crates/layout/src/generated/width.rs" }
  • pattern is a browser-compat-data key or a prefix of one, dot-separated (css.properties.width matches that key and everything nested under it, such as css.properties.width.calc). A segment can be * (matches any one segment) or end in * (matches a segment starting with that text — api.GPU* matches api.GPUAdapter, api.GPUDevice, and so on).
  • Rules are checked in file order; the first match wins. A key no rule matches is unreached.
  • evidence must resolve to something real: a repo-relative file path, path/to/doc.md#Heading Text naming an actual heading in that file, or corpus/<feature>-triage.json#<key> naming a feature triage record in which every page tagged for that key passes (docs/web-features-map.md → "The status rule"). oracle/suites/dashboard.mjs checks every rule's evidence on every run and refuses to compute anything if one doesn't resolve.

Updating the table #

flowchart LR
  map["docs/feature-map.json<br/>(hand-edited rules)"] --> dash["dashboard.mjs --render"]
  bcd["browser-compat-data"] --> dash
  dash --> json["docs/dashboard.json"]
  dash --> hist["docs/dashboard-history.json"]
  hist --> svg["docs/dashboard-history.svg"]
  dash --> svg
  dash --> readme["README.md table<br/>and graph"]
  json --> test["readme_dashboard.rs<br/>and --check"]
  hist --> test
  svg --> test
  readme --> test
  1. Edit docs/feature-map.json — add, remove or repoint a rule, and commit it with the work it describes. This is the only step a spec does by hand.
  2. Landing does the rest. After every merge, bench/after-land.sh runs bench/dashboard-update.sh in the primary checkout. It checks whether docs/dashboard.json, docs/dashboard-history.json, docs/dashboard-history.svg and the tables in README.md still match a fresh computation (node oracle/suites/dashboard.mjs --check; a different date alone does not count), and if not runs node oracle/suites/dashboard.mjs --render and commits the files that changed as dashboard: regenerate at <sha> on the base branch. Numbers that did not move add no commit. The computation is offline and takes seconds. It refuses, and writes nothing, when any of the four files has uncommitted changes; on any failure it puts them back and the landing's output line names it as not done (web-features map or dashboard not updated), and the next landing tries again because it recomputes from scratch.
  3. To refresh by hand (a track that wants the numbers before landing, or after a failed landing step), run npm run dashboard from oracle/ (equivalent to node suites/dashboard.mjs --render from that directory, or node oracle/suites/dashboard.mjs --render from the repo root) and commit the four generated files together, even when the history did not gain an entry, because the SVG and the README block are rewritten from it on every --render.

crates/harness/tests/readme_dashboard.rs fails cargo test --workspace if the README table, the JSON, the history and the graph disagree with each other, and node oracle/suites/dashboard.mjs --check fails if any of them is stale against a fresh computation from browser-compat-data. bench/dashboard-update-test.sh covers the landing step: numbers moved gives one commit, numbers unchanged gives none, and an uncommitted change in one of the four files is refused.

The history graph #

README.md carries a small graph above the table, between <!-- dashboard-history:start --> and <!-- dashboard-history:end -->. It is a committed file, docs/dashboard-history.svg, embedded with <img>: GitHub's README sanitizer strips inline <svg>, <script>, <style> and style=, so a committed image is the only form that renders at a size the page sets while staying reviewable in a diff.

One bar per recorded run, oldest on the left, the newest 60 at most. The x-axis is entry order, not time, so a long gap between runs looks the same as a short one. Each bar stacks generated, hand-written, linked and oracle only from the bottom up, in the table's column order. unreached is the remainder and is not drawn: at around 93% it would flatten the other four into a line at the bottom. It is still named in every bar's tooltip, so nothing is hidden, only unplotted.

The y-axis runs from 0 to the smallest multiple of 5 that covers the tallest stack, never less than 5, and the top edge carries its label (10% today). The label is there to be read: the axis does not go to 100, so the bars look far taller than the shares they stand for.

Behind the bars, a grey line with a faint fill under it plots the number of leaf keys reached (every status but unreached) for each run, one point at each bar's centre. It has its own scale, from 0 to the current leaf-key total, labelled on the right edge (17.3k today) so it is not read against the % on the left; the legend names it. Against the full total the line sits low and nearly flat, which is the point: it shows how much of the platform is covered in absolute terms, where the bars exaggerate. With a single run the line is a short flat segment across that bar. Every bar's tooltip also gives the reached count and the total.

docs/dashboard-history.json is the data. It is a JSON array, oldest first, one compact object per line so a new entry is a one-line diff, and it records only overall — there is no per-area history. Besides the five percentages, an entry records reached and total, the absolute counts behind the line, so a later change in the total does not move old points. A run appends an entry only when one of the five overall percentages, or either count, differs from the last entry's; re-running on unchanged inputs leaves the file byte-identical, on the same day or a later one, even though docs/dashboard.json's date moves. Entries are never rewritten, merged or removed, and several may share a date. The first entry predates the linked status, which did not exist when it was recorded, so its linked is 0. Entries recorded before the counts existed have no reached or total; the graph derives reached as (100 − unreached) × current total / 100, rounded, and the tooltip marks it "about", and the last entry's percentages alone decide whether a run appends while it is one of those.

Expect small numbers #

The generated, hand-written and linked shares are expected to be small single-digit percentages for a long time. linked could be made large overnight by mapping every css.properties key to Stylo, but Stylo's cascade only counts once a comparison shows the value matching, so it grows with the compared style subset, not with Stylo's own property list. The dashboard's job is to be accurate about what's implemented, not to look impressive — an inflated number would mean a bug in docs/feature-map.json, not a feature. oracle only and unreached will dominate every area's row for the foreseeable future; that reflects the plan, not a shortfall the table should hide.

Per-feature progress #

The same rule map also drives docs/web-features-map.json, a per-feature view over the web-features package: each feature's status (complete, partial, none, unknown), its keys, and the evidence behind each, regenerated after every merge. The rule matching is shared code (oracle/suites/feature-map-rules.mjs), so a key's class here and its via there always agree. A dashboard reads that file's totals for per-feature progress; format and status rule: docs/web-features-map.md.

When a candidate is admitted #

Every spec that lands a generated function already says what to commit (the source file, its .provenance.json, the pub mod line, the entry points). Add one more step to that list, to copy into the spec's own "When a candidate is admitted" section: add the function's compat keys to docs/feature-map.json as generated rules pointing at the new file, and commit that alongside the function. The dashboard's numbers, the history and the README table follow from landing ("Updating the table"); the spec does not run npm run dashboard or commit its output.