# Dump format

The contract between the oracle service, the comparator, and eventually the engine under
test. Frozen: changing it requires changing every consumer at once.

## Oracle protocol

The oracle is a long-lived process speaking newline-delimited JSON on stdio. One JSON
object per line in each direction, no pretty-printing, no interleaved text on stdout.
Diagnostics go to stderr.

Browsers stay open for the process lifetime. Nothing spawns a browser per case.

### Request

```json
{"id": "case-0001", "html": "<!doctype html>...", "engines": ["chromium", "firefox", "webkit"]}
```

`engines` is optional; omitted means every engine the service launched. `script` is
additive (`docs/feature-cost-experiment.md`): optional JavaScript run in the page after
`html` loads and before the dump walk. Omitted, a request behaves exactly as before --
this is what lets DOM tree-mutation ground truth reuse the ordinary dump response
unchanged, rather than needing a new shape: the mutation happens, then the same
`domDump` walk that has always run answers the request.

```json
{"id": "case-0002", "html": "<div id=a><span id=b></span></div>", "script": "document.getElementById('a').appendChild(document.createElement('b'));"}
```

### Response

```json
{"id": "case-0001", "dumps": {"chromium": {...}, "firefox": {...}}, "errors": {"webkit": "message"}}
```

`dumps` holds one dump per engine that succeeded. `errors` holds a message per engine
that failed, and is omitted when empty. A response always carries the `id` it answers.

### Control

`{"id": "...", "op": "ping"}` responds
`{"id": "...", "ok": true, "engines": [...], "versions": {"chromium": "140.0.7339.5", ...}, "profile": "default"}`.
`versions` maps each launched engine's name to its `engine_version` (see below); `profile`
is the launch profile active for the process (see "Launch profiles" below).
`{"id": "...", "op": "shutdown"}` closes browsers and exits.

## Dump

```json
{
  "engine": "chromium",
  "engine_version": "140.0.7339.5",
  "viewport": {"width": 800, "height": 600},
  "overflowed": false,
  "nodes": [
    {
      "index": 0,
      "parent": -1,
      "tag": "HTML",
      "depth": 0,
      "bounding": {"x": 0.0, "y": 0.0, "w": 800.0, "h": 100.0},
      "rects": [{"x": 0.0, "y": 0.0, "w": 800.0, "h": 100.0}],
      "style": {"display": "block", "...": "..."}
    }
  ]
}
```

- `engine_version` is the exact browser build that produced the dump, from Playwright's
  `browser.version()` at launch — not from `navigator.userAgent`, which the in-page walk
  could report but which is imprecise. Required, since ground truth without a version is
  exactly what this field exists to rule out.
- `nodes` is every element in document order, starting at `<html>` at index 0. Document
  order means `index` increases monotonically and a parent always precedes its children.
- `parent` is an index into `nodes`, or `-1` for the root.
- `tag` is the uppercase tag name as the DOM reports it.
- `rects` is `getClientRects()`, one entry per box fragment, in content order. An empty
  array means the element generates no box — that is the signal, not a null `bounding`.
- `bounding` is `getBoundingClientRect()`, always present, zeroed when `rects` is empty.
- Coordinates are viewport-relative CSS pixels as doubles, serialized at full precision.
- `overflowed` is true when the document scrolls in either axis. Overflow means a
  scrollbar may have stolen viewport width by a platform-dependent amount, so comparisons
  on an overflowed case are not trustworthy and the case is discarded.

## Style subset

A fixed, ordered list. Present on every node, values exactly as `getComputedStyle`
returns them.

```
display position box-sizing direction writing-mode float clear
width height min-width max-width min-height max-height
margin-top margin-right margin-bottom margin-left
padding-top padding-right padding-bottom padding-left
border-top-width border-right-width border-bottom-width border-left-width
overflow-x overflow-y
```

A mismatch here indicts the style pipeline rather than layout, which is the second
attribution axis.

## Comparison

Comparison never uses absolute coordinates.

1. **Shape.** The sequence of `(tag, depth, parent)` must match. A mismatch is a parse
   divergence and the case reports that and stops — no geometry is compared.
2. **Size.** Each node's `bounding` width and height, and each fragment's width and
   height pairwise.
3. **Relative offset.** Each node's `bounding` origin minus its parent's `bounding`
   origin. The root compares against the viewport origin. A single wrong offset then
   appears once rather than corrupting every descendant.

Tolerance is 1/60 px (0.01667), the coarsest engine quantization — Gecko app units.
Blink and WebKit quantize at 1/64. Differences below tolerance are quantization noise.

Both an exact verdict (within tolerance or not) and the raw signed delta are reported.
The verdict gates pass/fail; the delta is the dense reward signal and needs no threshold.

## Trace protocol

Additive: everything above (request, response, dump, style subset, comparison) is
unchanged. `trace` is a new request operation with its own response shape, for what
script can observe beyond geometry — return values, thrown errors, event order, promise
settlement order, object structure, pixel and audio buffers — the oracle for
`docs/oracle-coverage.md` → "Four kinds of oracle that need no geometry vote" item 4.

### Origin

`Engine.run()` (the geometry path) uses `page.setContent`, which leaves the page on
`about:blank`; many APIs this unlocks need a secure context or a real origin. The oracle
starts one local HTTP server on `127.0.0.1` for its own lifetime and serves trace pages
from `http://localhost:<port>/trace/<id>`. Browsers treat `localhost` as a secure
context regardless of scheme, so `isSecureContext` is `true` without TLS. Every trace
records the page's `origin` and `isSecureContext`.

### Request

```json
{"id": "t1", "op": "trace", "script": "trace.value('x', 1); trace.done();", "html": "<button id=btn></button>", "engines": ["chromium", "firefox", "webkit"]}
```

`script` is required: JavaScript text run in the served page, after the recorder (see
below) is installed and after `html` (optional; omitted means an empty body) has
loaded. `engines` is optional, same meaning as the geometry request.

### Response

```json
{"id": "t1", "traces": {"chromium": {"...": "..."}}, "errors": {"webkit": "message"}}
```

`traces` holds one trace per engine that completed; `errors` holds a message per engine
that failed (including "engine not launched"), omitted when empty.

### Trace

```json
{
  "engine": "chromium",
  "engine_version": "153.0.8010.12",
  "profile": "default",
  "origin": "http://localhost:41673",
  "is_secure_context": true,
  "timed_out": false,
  "records": [
    {"seq": 0, "kind": "value", "label": "x", "value": 1}
  ]
}
```

- `profile` is the launch profile active for every engine in this run (see "Launch
  profiles" below).
- `timed_out` is true when the recorder's fixed completion timeout ended the trace
  rather than the script itself calling `trace.done()`.
- `records` is every observation the recorder pushed, in the order it pushed them; `seq`
  is that order as an explicit integer. Order is part of the result: two traces with the
  same records in a different order are not the same trace.

### The recorder

A script installed via Playwright's `context.addInitScript`, so it runs before any
script the trace page's own content carries — including inline `<script>` tags, which
load in document order after it. It defines `window.trace`:

- `trace.value(label, v)` — records a canonicalized value.
- `trace.event(target, types)` — listens for `types` (a string or array of strings) on
  `target` and records one event per firing, labeled from `target`'s identity
  (`window`, `document`, or `TAGNAME` / `TAGNAME#id` for an element).
- `trace.settle(label, promise)` — records how and when `promise` settles.
- `trace.error(label, err)` — records a caught error directly.
- `trace.done()` — ends the trace early. A trace that never calls this still ends, at a
  fixed completion timeout (5000ms, not configurable per request), with `timed_out: true`.

### Canonicalization

Values leave the page as JSON, so the recorder canonicalizes each one before recording
it:

- Booleans and strings serialize as-is.
- Numbers serialize as-is, except `NaN`, `Infinity`, `-Infinity` and `-0`, which have no
  JSON representation and are encoded explicitly as `{"$num": "NaN"}` (and so on).
- `undefined` becomes `{"$undefined": true}`; a `bigint` becomes `{"$bigint": "<digits>"}`.
- A thrown `Error` (wherever one ends up — a `trace.error` call, a rejected
  `trace.settle`) reduces to its name: `{"$error": true, "name": "TypeError"}`. The
  message is not recorded; the name is what a test asserts on.
- A typed array, `DataView` or `ArrayBuffer` reduces to base64 of its bytes plus element
  type and length: `{"$buffer": true, "elementType": "Float32Array", "length": 2, "base64": "..."}`.
- `ImageData` reduces the same way, plus `width`/`height`; the channel data is
  `Uint8ClampedArray`. `AudioBuffer` reduces to `{"$audioBuffer": true, "numberOfChannels", "length", "sampleRate", "channels": [...]}`,
  each channel a `Float32Array`-shaped buffer.
- Anything else (a plain object, an array, an IDL result) reduces to an own-property
  key/type structure walk, bounded to depth 4 and 64 array elements per level, so an
  unbounded or circular object cannot hang the trace: `{"$shape": "object", "keys": [...], "props": {"k": {"type": "...", "value": ...}}}`
  for objects, `{"$shape": "array", "length": N, "elements": [...]}` for arrays.

### Comparison

`crates/harness/src/trace.rs` compares two traces record by record, in order: an
unequal record count is reported outright (`CountMismatch`), since there is no sound way
to pair up the remainder. Otherwise every position is compared: `kind` and `label`
exactly; an `Event`'s `type` and an `Error`'s `name` exactly; a `Value` or `Settle`'s
`value` structurally exactly (object keys, array lengths, non-numeric leaves) except a
`$buffer`/`$imageData`/`$audioBuffer` payload, whose samples are decoded and compared
within a tolerance (`BufferTolerance`: relative and absolute, looser than the geometry
tolerance since two engines' DSP or rasterizer rounding differs more than their layout).
The three-engine vote (`trace::vote`) mirrors `compare::vote()`: pairwise-agreement
clustering per record, naming an outlier on a 2-1 split, `Split` (untrustworthy) on a
three-way disagreement.

### Launch profiles

Chosen once at oracle startup with `--profile=<name>` (default: `default`), applied to
every engine the process launches, reported in the `ping` reply's `profile` field and in
every trace's `profile` field.

- `default` — no extra launch args, no prefs, no permission grants. Exactly what
  geometry dumps have always used; cached ground truth keeps its meaning.
- `media` — makes hardware-backed APIs deterministic:
  - Chromium: `--use-fake-device-for-media-stream`, `--use-fake-ui-for-media-stream`,
    `--enable-unsafe-swiftshader`, plus `context.grantPermissions(['camera', 'microphone'])`
    (needed even with the fake device flags, or `getUserMedia` rejects with
    `NotAllowedError`).
  - Firefox: prefs `media.navigator.streams.fake`, `media.navigator.permission.disabled`.
  - WebKit: no launch args (Playwright's WebKit build has none for this); the same
    `grantPermissions(['camera', 'microphone'])` as Chromium.
  - Verified by observation, not assumed: `getUserMedia({video: true, audio: true})`
    resolves in all three engines under `media`, each delivering a fake track whose
    `getSettings()` reports a plausible fixed resolution/frame rate/sample rate (the key
    set differs per engine — that is a real platform difference `probe`'s
    `fake-camera-track-settings` result surfaces, not a harness bug).
  - **Known gap**: the profile also sets Firefox's
    `webgl.disable-fail-if-major-performance-caveat`, meant to allow WebGL on software
    rendering. Observed on the development host, this is not sufficient: Firefox's
    WebGL context creation still fails ("exhausted GL driver options"), where Chromium
    and WebKit both succeed via their bundled software renderer. Left in place as
    directionally correct and harmless; not claimed to work. A host that lets Firefox's
    GPU process start may not hit this.
- CDP-only emulation (virtual sensors, virtual authenticator, Bluetooth) is
  Chromium-only and is not implemented; it is a later step, not a partial one.

### Probes

`crates/harness/src/bin/probe.rs`, modeled on `crosscheck.rs`: hand-written scripts
across `OfflineAudioContext` rendering, a canvas `getImageData` readback, DOM event
bubble order, promise/microtask settlement order, an IDL shape walk (`URL`'s prototype
getters), and a fake-camera track's settings, each run through every engine, compared
pairwise and voted three-way, reporting agreement per probe.

### Not yet supported

- **CDP-only emulation** (virtual sensors, virtual authenticator, Bluetooth). These
  exist only in Chromium, so they need their own launch profile or a per-request flag,
  and the trace response needs a field saying which engines applied the emulation.
  Without it, a three-engine vote compares emulated Chromium output against unemulated
  Firefox and WebKit output.
- **Trace inputs mined from suites.** Scripts drawn from WPT, Chromium `web_tests` and
  WebKit `LayoutTests`, seeded by the text baselines identical between Chromium and
  WebKit (`docs/oracle-coverage.md`), would run through `Oracle::trace()` and
  `trace::vote()`, with mismatches routed to test-case generation.

## Reflection protocol

Additive, the same way the trace protocol is: everything above is unchanged, and a new
request operation carries its own response shape. This is the reflection half of
`docs/feature-cost-experiment.md`'s measurement -- content-attribute and reflected-IDL
state -- where "Request" above's additive `script` field is the DOM tree-mutation half.

### Request

```json
{"id": "r1", "op": "reflect", "html": "<input id=a>", "script": "document.getElementById('a').setAttribute('value', '5');", "probes": [{"selector": "#a", "attr": "value", "idl": "value"}], "engines": ["chromium", "firefox", "webkit"]}
```

`html` is optional body markup, same meaning as a trace request's (omitted means an
empty body). `script` is optional: JavaScript run after `html` loads and before the
probe walk -- the mutation sequence, when the case has one. `probes` is required, one
entry per (selector, content attribute, IDL property) triple to read: `selector` is a
CSS selector (every case this repository scores against restricts it to a bare `#id`,
`docs/feature-cost-experiment.md`); `attr` is the content attribute name; `idl` is the
IDL property name and defaults to `attr` when omitted (most reflected attributes share
their content and IDL name; `for`/`htmlFor`, `class`/`className` and similar exceptions
name it explicitly). `engines` is optional, same meaning as the geometry request's.

### Response

```json
{"id": "r1", "states": {"chromium": {...}}, "errors": {"webkit": "message"}}
```

`states` holds one reflection dump per engine that succeeded; `errors` holds a message
per engine that failed (including a probe's selector not resolving), omitted when empty.

### Reflection dump

```json
{
  "engine": "chromium",
  "engine_version": "153.0.8010.12",
  "observations": [
    {"selector": "#a", "attr": "value", "attr_value": "5", "idl": "value", "idl_value": "5"}
  ]
}
```

One observation per probe, in the request's probe order. `attr_value` is the content
attribute's string value, or `null` when absent. `idl_value` is the IDL property's value,
canonicalized: booleans, strings and finite numbers as-is; `NaN`/`Infinity`/`-Infinity`
encoded explicitly the same way the trace protocol's canonicalization does
(`{"$num": "NaN"}` and so on) since they have no JSON representation; a property the
element does not carry (an IDL name typoed, or the wrong element for that attribute)
comes back as `{"$undefined": true}` rather than `null`, so a probe naming a property
that does not exist is visible rather than indistinguishable from one that resolved to
`null`. A reflected IDL property is scalar by construction, so this is a much smaller
canonicalization than the trace protocol's general one -- no buffers, no object shape
walk.

### Comparison

`crates/harness/src/reflect.rs` votes three-way per observation: `attr_value` and
`idl_value` must both match, exactly -- reflection has no quantization to tolerate the
way geometry does. The clustering itself mirrors `compare::vote()` and `trace::vote()`:
pairwise agreement per observation, naming an outlier on a 2-1 split, `Split`
(untrustworthy) on a three-way disagreement.

### Scope decisions

- **`#id` only.** Every probe's `selector` in this repository's own cases is a bare
  `#id`; general CSS selector matching is not needed for the two attribute types and one
  mutation sequence step 1 chose, and `crates/dom::mutable::MutableTree` (the candidate's
  side of the same cases) implements only `#id` lookup to match.
- **No JS interpreter on the candidate side.** A case's `script` is hand-written
  JavaScript for the oracle to run; the equivalent instruction for a Rust candidate to
  replay against `crates/dom::mutable::MutableTree` is a separate, hand-written
  `crates/dom::mutable::TreeOp` value in the same case, not derived from `script`
  automatically. Keeping both in sync by hand is the cost this scope decision accepts, in
  exchange for not building a JavaScript-to-Rust mutation translator for a five-case
  experiment.