# How much of WPT needs no script, and which JavaScript engine to link

Answers two open questions behind `roadmap.md` "Later phases": how far the engine can get
through WPT before any JavaScript engine is linked, and which engine to link when one is.
Measured on 2026-10-19 at WPT commit `26f417a8c09fbfcc0b4fcf55760f31090ef2b772` (the
`wpt.commit` pinned in `oracle/suites/suites.json`). Every number below is in
`docs/js-engine-choice.json`; `crates/harness/tests/js_engine_choice.rs` checks that the
engine rows match it and that the WPT counts add up.

## Recommendation

**Link nothing yet.** 40.1% of WPT test files outside test262 run with no script, and
66.6% of `css` does: 25350 test files, most of them reftests. The layout work has that
many tests to pass before a script engine is the limit. `html` (6.4%) and `dom` (0.0%)
are where script becomes necessary.

**When an engine is linked, link Boa first.** It is the only candidate written in Rust:
it builds with no C or C++ compiler and downloads no prebuilt library, so a build on any
lane is the same build from crates.io sources; it builds for both WebAssembly targets the
engine cares about (`wasm32-unknown-unknown` needs `boa_engine`'s `js` feature); and at
95.6% of test262 it trails V8 and SpiderMonkey by two to three points, about half of them
in `intl402`, while QuickJS-NG passes 83.5%. Its cost is size: 12.2 MiB over an empty Rust
binary, against QuickJS-NG's 1.2 MiB.

What would change it:

- **The WebAssembly build has to stay free of JavaScript glue.** `crates/webengine` builds
  a plain `wasm32-unknown-unknown` module with no `wasm-bindgen`. Boa's `js` feature
  reaches the platform's random source through `wasm-bindgen` imports. If supplying
  `getrandom` another way (its custom backend) does not work, QuickJS-NG builds for that
  target with a wasm-capable clang and no glue, and becomes the first choice.
- **Script speed starts to dominate WPT run time.** Boa is a bytecode interpreter with no
  JIT; QuickJS-NG is a small, quick-starting interpreter. Execution speed was not
  measured here. A WPT run that is slow because of script points to QuickJS-NG, and a
  real-site workload to V8 or SpiderMonkey.
- **Binary size gets a budget.** At 12.2 MiB, Boa is ten times QuickJS-NG; a budget under
  that points to QuickJS-NG.
- **Conformance at the top end matters more than a Rust-only build.** SpiderMonkey
  (98.5%) and V8 (97.7%) lead, but both are C++ linked through a prebuilt static library
  downloaded at build time (or a from-source build of a browser engine), neither builds
  for either WebAssembly target, and they add 33.2 MiB and 45.6 MiB.
- **QuickJS-NG gains Temporal.** Its gap to Boa is mostly the Temporal tests (0 of 6715
  pass) and `intl402`; without `intl402` it passes 89.1%.

## Method

### WPT without script

`oracle/suites/wpt-script-free.mjs` reads a blobless clone of WPT at the pinned commit in a
temporary directory (`git fetch --depth 1 --filter=blob:none`, then `git checkout`), builds
WPT's own manifest there (`./wpt manifest`), and classifies every test file the manifest
lists. It refuses a clone at any other commit and never writes to
`oracle/suites/checkouts/` or `oracle/suites/suites.json`.

```sh
node oracle/suites/wpt-script-free.mjs /tmp/wpt-clone > counts.json
```

The rule, the same text as at the top of the script:

- A test file is a path the manifest lists under a test type. Support files are not tests.
  A file with several variants (`?query`, `.any.js` globals) counts once.
- testharness, test262, wdspec and aamtest files always need script: their verdict comes
  from script in the page or from a driver that runs script in it.
- reftest, print-reftest, crashtest, visual and manual files need script when the test
  file, or any reference it names (followed through references that name their own
  references), contains a `<script>` element or an inline event-handler attribute
  (`onload=`, `onclick=`, ...), or carries the `reftest-wait` or `test-wait` class. Text
  inside `<!-- -->` comments is ignored. A reference that is not on disk (one a server
  handler generates) is counted as needing script, since nothing shows it can run
  without.
- Everything else about a file is ignored: a frame or image it loads is not followed, so a
  script-free file whose iframe document has script counts as script-free.

The last point makes the script-free counts an upper bound. One reference was missing
on disk at this commit and counts as needing script. Manual tests are counted like the
other scanned types, but they need a person to judge them; the 1577 script-free manual
files are not reachable by automation either way.

test262 is ECMAScript's own conformance suite, vendored into WPT under `third_party/`. It
measures the JavaScript engine, not the browser engine, so the browser-side fraction is
also given without it.

### Engines

Each engine has a standalone probe crate under `experiments/js-engine-probe/<engine>/`
with its own empty `[workspace]` table and `Cargo.lock`, so `cargo test --workspace` never
builds it. Each probe registers one host function written in Rust (`hostAdd`), evaluates
`hostAdd(40, 2)` and prints the result. `experiments/js-engine-probe/empty/` is the same
program with no engine, the size baseline.

`experiments/js-engine-probe/measure-all.sh <scratch-dir>` measures them one after another
on the development box (16 cores, rustc 1.97.1). For each probe,
`experiments/js-engine-probe/measure.sh`:

- fetches crate sources first, so no build counts the crates.io download;
- builds with `cargo build --release --locked` from an empty `CARGO_TARGET_DIR` under the
  scratch directory, with the compiler cache off (`RUSTC_WRAPPER` cleared, which also
  stops cc-rs from using it);
- records the size of the release binary, stripped by the probe's profile
  (`strip = true`), and subtracts the empty probe's size;
- keeps the lines of build-script output that mention a download, to tell whether the
  build fetched a prebuilt engine library;
- tries `wasm32-unknown-unknown` and `wasm32-wasip1` release builds and keeps their logs.

Build times come from `experiments/js-engine-probe/time-builds.sh <scratch-dir>`: three
cold release builds per probe, one at a time, each from a fresh empty `CARGO_TARGET_DIR`
with the compiler cache off, and each started only once the one-minute load average was
below 2 (it was between 1.56 and 1.87 at every start). The tables give the median of the
three; the samples and start loads are in the JSON.

test262 pass rates are not run here. They come from test262.fyi
(`https://data.test262.fyi/index.json`, per-feature counts from
`https://data.test262.fyi/meta.json`), generated 2026-10-19 against test262 commit
`d53ebc8`. test262.fyi runs newer engine builds than the crates measured (Boa at git
`9871775`, QuickJS-NG at `081d50c`, V8 15.8.21, SpiderMonkey 160.0a1), so a crate's own
pass rate can be a little lower. rquickjs bundles QuickJS-NG, not Bellard's QuickJS, so
its row uses test262.fyi's `qjs_ng`.

## WPT without script

| Directory | Test files | Need script | No script | No script, share |
|---|---:|---:|---:|---:|
| all of WPT | 122073 | 94603 | 27470 | 22.5% |
| all of WPT without test262 | 68428 | 40958 | 27470 | 40.1% |
| `third_party` | 53625 | 53625 | 0 | 0.0% |
| `css` | 38049 | 12699 | 25350 | 66.6% |
| `html` | 10044 | 9397 | 647 | 6.4% |
| `svg` | 1887 | 998 | 889 | 47.1% |
| `referrer-policy` | 1392 | 1391 | 1 | 0.1% |
| `content-security-policy` | 879 | 877 | 2 | 0.2% |
| `dom` | 752 | 752 | 0 | 0.0% |
| `webdriver` | 641 | 641 | 0 | 0.0% |
| `mathml` | 610 | 392 | 218 | 35.7% |
| `fetch` | 495 | 485 | 10 | 2.0% |
| `navigation-api` | 453 | 453 | 0 | 0.0% |
| `wasm` | 408 | 408 | 0 | 0.0% |
| `mixed-content` | 388 | 388 | 0 | 0.0% |
| `editing` | 347 | 347 | 0 | 0.0% |
| `shadow-dom` | 346 | 342 | 4 | 1.2% |
| `xhr` | 332 | 332 | 0 | 0.0% |
| `webvtt` | 323 | 323 | 0 | 0.0% |
| `webaudio` | 307 | 307 | 0 | 0.0% |
| `service-workers` | 302 | 299 | 3 | 1.0% |
| `core-aam` | 277 | 275 | 2 | 0.7% |
| `scroll-animations` | 276 | 270 | 6 | 2.2% |
| `infrastructure` | 184 | 138 | 46 | 25.0% |
| `encoding` | 157 | 153 | 4 | 2.5% |
| `html-ruby-extensions` | 84 | 0 | 84 | 100.0% |
| `preload` | 71 | 70 | 1 | 1.4% |
| `forced-colors-mode` | 63 | 30 | 33 | 52.4% |
| `jpegxl` | 61 | 20 | 41 | 67.2% |
| `accessibility` | 58 | 44 | 14 | 24.1% |
| `webauthn` | 56 | 55 | 1 | 1.8% |
| `quirks` | 51 | 26 | 25 | 49.0% |
| `png` | 44 | 33 | 11 | 25.0% |
| `compat` | 38 | 17 | 21 | 55.3% |
| `html-longdesc` | 23 | 1 | 22 | 95.7% |
| `xml` | 23 | 18 | 5 | 21.7% |
| `density-size-correction` | 22 | 2 | 20 | 90.9% |
| `entries-api` | 18 | 17 | 1 | 5.6% |
| `html-media-capture` | 15 | 12 | 3 | 20.0% |
| `content-dpr` | 6 | 2 | 4 | 66.7% |
| `contenteditable` | 4 | 3 | 1 | 25.0% |
| `avif` | 2 | 1 | 1 | 50.0% |

The table lists every top-level directory with at least one script-free test file or at
least 300 test files. The other 229 directories (8960 test files) have no script-free test
file; their counts are in `docs/js-engine-choice.json` under `wpt.directories`.

By test type, across all of WPT:

| Type | Test files | Need script | No script | No script, share |
|---|---:|---:|---:|---:|
| aamtest | 98 | 98 | 0 | 0.0% |
| crashtest | 2005 | 1276 | 729 | 36.4% |
| manual | 3047 | 1470 | 1577 | 51.8% |
| print-reftest | 431 | 22 | 409 | 94.9% |
| reftest | 27939 | 5805 | 22134 | 79.2% |
| test262 | 53645 | 53645 | 0 | 0.0% |
| testharness | 31552 | 31552 | 0 | 0.0% |
| visual | 2715 | 94 | 2621 | 96.5% |
| wdspec | 641 | 641 | 0 | 0.0% |

Within `css`, 20713 of 24921 reftests (83.1%) need no script; every one of its 7712
testharness files does.

## Engines

| Engine | Crate | License | test262 | test262 without `intl402` | Binary over empty | Cold release build | Prebuilt download | `wasm32-unknown-unknown` | `wasm32-wasip1` |
|---|---|---|---:|---:|---:|---:|---|---|---|
| Boa 0.22.0 | `boa_engine` 0.22.0 | MIT OR Unlicense | 95.6% | 96.7% | 12.2 MiB | 49.2 s | no | fails by default; builds with feature `js` | builds |
| QuickJS-NG 0.16.2 | `rquickjs` 0.14.0 | MIT | 83.5% | 89.1% | 1.2 MiB | 42.8 s | no (the `wasm32-wasip1` build downloads WASI SDK 24) | fails here (no clang); builds with a wasm clang | builds |
| V8 15.2.124 | `v8` 152.2.0 | MIT (bindings), BSD-3-Clause (V8) | 97.7% | 97.6% | 45.6 MiB | 26.9 s | yes | fails | fails |
| SpiderMonkey 153.4.0 | `mozjs` 0.26.9 | MPL-2.0 | 98.5% | 98.4% | 33.2 MiB | 16.2 s | yes | fails | fails |

Build times are medians of three cold builds with 16 cores. The empty probe is 366352
bytes stripped and builds in 0.2 s. V8 and
SpiderMonkey build fastest only because their engine arrives as a prebuilt static library;
building either from source was not timed and takes far longer.

WebAssembly build errors, as recorded in the JSON:

- **Boa, `wasm32-unknown-unknown`:** getrandom 0.4.3 refuses the target by default ("you
  may need to enable the "wasm_js" crate feature"). With `boa_engine`'s `js` feature the
  probe builds, and the module then imports `wasm-bindgen` glue.
- **QuickJS-NG, `wasm32-unknown-unknown`:** the `rquickjs-sys` build script needs a clang
  and the development box has none. With `CC_wasm32_unknown_unknown` pointed at WASI SDK
  24's clang the probe builds: `rquickjs-sys` vendors a wasm libc for this target.
- **V8, both targets:** the build script finds no prebuilt binding for the target
  (HTTP 404), and V8 has no WebAssembly port.
- **SpiderMonkey, both targets:** the first failure is `libz-sys` needing a clang.
  `mozjs_sys` does not support `wasm32-unknown-unknown`; it supports WASI only through
  `WASI_SDK_PATH` and a from-source SpiderMonkey build, which was not attempted.

## Reproducing

```sh
# WPT counts (writes MANIFEST.json into the clone on first run)
git init /tmp/wpt-clone
node oracle/suites/mirror.mjs git https://github.com/web-platform-tests/wpt.git \
  -C /tmp/wpt-clone fetch --depth 1 --filter=blob:none '{url}' 26f417a8c09fbfcc0b4fcf55760f31090ef2b772
git -C /tmp/wpt-clone checkout FETCH_HEAD
node oracle/suites/wpt-script-free.mjs /tmp/wpt-clone

# Engine probes (needs the wasm32-unknown-unknown and wasm32-wasip1 Rust targets)
experiments/js-engine-probe/measure-all.sh /tmp/js-engine-probe
# Build times: three cold builds per probe, each waiting for a load average below 2
experiments/js-engine-probe/time-builds.sh /tmp/js-engine-timing
```