okikio.dev / selector-benchmark-harness
[READ-ONLY] Mirror of https://github.com/okikio/selector-benchmark-harness.
Star
Fork 0
Watch
TypeScript 90%
10%
HTML <1%
1 1 0
.vscode
bench
docs
fixtures
results
scripts
src
deno.json
package-lock.json
package.json
README.md
README.md

Kaiju live selector benchmark harness #

This is a reproducible validation harness for the live-browser selector choices before implementing the full CLI collector.

It tests the selector strategy as a three-part provenance model:

CSS selector
  best for browser replay

XPath selector
  best for structural/source traceability

Accessibility selector
  best for human and agent interpretation

The harness follows the attached instruction bundle:

  • Deno v2, strict TypeScript, ESM, explicit file extensions.
  • @std/testing/bdd, @std/expect, and fast-check for tests.
  • mitata benchmarks with do_not_optimize().
  • Realistic, behavior-driven tests instead of implementation-only assertions.
  • Durable JSONL/artifact corruption checks.

Why this exists #

Before adding the full live browser collector, we need evidence that the selector tooling is reliable enough to store as provenance. This project gives us a repeatable way to test selector behavior under realistic failure modes:

  • renamed classes
  • removed ids
  • removed stable attributes
  • renamed accessible names
  • hidden/non-rendered nodes
  • invalid CSS identifier characters
  • generated-looking ids/classes
  • corrupted JSONL
  • missing/truncated/hash-corrupted artifacts
  • feature flag conflicts

Requirements #

  • Node.js 22+
  • npm
  • Chromium-compatible browser

The harness uses the npm-distributed Deno binary so it can be reproduced in sandboxes where Deno is not globally installed.

If Chromium is not found automatically, set:

export CHROMIUM_PATH=/path/to/chromium

Run everything #

npm install
npm exec --yes --package deno@2.9.1 -- deno task verify
npm exec --yes --package deno@2.9.1 -- deno task report

Faster iteration #

npm exec --yes --package deno@2.9.1 -- deno task build:browser
npm exec --yes --package deno@2.9.1 -- deno task test
npm exec --yes --package deno@2.9.1 -- deno task bench

Outputs #

results/selectors_bench.json
results/selector-validation-report.md

The benchmark JSON is deliberately simple so it can be committed, compared, or pasted into a technical decision record.

Expected conclusion #

The expected conclusion is not that one selector type wins. The expected conclusion is that selector provenance needs all three selector families:

  • CSS selectors are usually fastest and best for direct browser replay.
  • XPath selectors are useful for structure and source traceability.
  • Accessibility selectors survive many visual/layout changes and are much easier for agents to interpret.

The tests also prove that each selector family can fail under a plausible mutation, which is why the live collector should store them together rather than choosing only one.