enabled: true model: claude-fable-5-1 touches: [oracle/suites/, docs/platform-economics.md, docs/oracle-coverage.md, corpus/] depends-on: [] #

Sample 200 compat keys and record whether a three-engine probe exists for each #

docs/platform-economics.md → "Experiments that would move the estimate most" names the cheapest measurement in the repo: "sample 200 random compat keys and record whether a three-engine-comparable probe can be written for each, to tighten the oracle fraction". The oracle fraction is the denominator of the whole thesis (roadmap.md → "Open questions": does the economics work across the real web surface, not just layout), and it is currently an estimate built from coverage of whole suites, not from a sample of features. This track produces the sample and the number.

Write oracle/suites/compat-sample.mjs beside oracle/suites/entry-coverage.mjs, using the same pinned browser-compat-data version that file uses:

Draw 200 compat keys uniformly at random from the leaf entries under api, css, html, javascript.builtins, svg, webassembly, http and mathml, with a fixed seed written into the output so the draw is reproducible. Record each key's top-level area and whether it is standard-track and unflagged in the current stable release of all three engines per the compat data.
Classify each key against a written rubric, committed at the top of the script and copied into the doc: probe-able (a script-observable, three-engine-comparable check exists: a computed value, a DOM/API return value, a layout rect via the existing dump, a trace event via the trace protocol in docs/dump-format.md), probe-able with new oracle work (a check exists but needs an oracle kind the repo lacks, named: pixels, audio, input, network, timing), or not probe-able (nothing observable without pixels or a human). Every classification carries a one-line reason naming the observable. Where the existing per-subtest three-engine results in docs/wpt-votes.md or crates/wpt-votes already cover a key, cite them instead of judging afresh.
Write the sample and classifications as corpus/compat-sample/sample-<seed>.json and a summary table into docs/oracle-coverage.md: the fraction in each class, overall and per area, with a 95% confidence interval for a sample of 200.

Then update the oracle-fraction figure in docs/platform-economics.md from the sample, keeping the previous estimate beside it with the method that produced each, and shorten that experiment's entry under "Experiments that would move the estimate most" to say it has run and what it found.

No browsers run and no model batches run; this is a reading of compat data against a rubric. The rubric is the thing a later reader will argue with, so it is stated in full before a single key is classified, and no key's classification changes it afterwards (a rubric change means restarting the sample with a new seed and saying so).

cargo test --workspace must pass (nothing under crates/ changes). node oracle/suites/compat-sample.mjs --seed <n> reproduces the committed JSON byte for byte.