diff --git a/.isu/issues.json b/.isu/issues.json index 0944dbd..4479005 100644 --- a/.isu/issues.json +++ b/.isu/issues.json @@ -1,5 +1,5 @@ { - "next_id": 334, + "next_id": 339, "issues": [ { "id": 1, @@ -3718,7 +3718,7 @@ ], "assigned": [], "author": "piefev", - "state": "open", + "state": "closed", "created_at": "2026-05-28T23:19:14Z" }, { @@ -4071,6 +4071,71 @@ "author": "piefev", "state": "open", "created_at": "2026-06-04T17:06:11Z" + }, + { + "id": 334, + "repo": "we", + "title": "Snapshot bing.com hydration bundles for offline parity (policy b, parent #280)", + "body": "Parent: #280. Policy decision recorded in #306: the Phase-23 parity bar is (b) — complete offline snapshots that hydrate to match the live Chromium golden.\n\nbing.com renders blank offline because `#hp_app` is an empty mount point in the committed snapshot; the nav/search/news widgets are injected by the RMS/Peregrine framework (see #304). The current render also shows a different daily background than the golden (see #305).\n\nWork: capture every assets.msn.com Peregrine hydration bundle + image bing needs (per #304 step 5), commit them under crates/e2e/real-web/snapshots/bing.com/, seed them via `cache_put` in crates/e2e/scenarios/real-web/bing.com.we, and confirm `we` hydrates `#hp_app` offline. Keep the `# xfail` marker until the scenario matches the Chromium golden. Repro: cargo run -p we-e2e -- --scenario crates/e2e/scenarios/real-web/bing.com.we --out-dir crates/e2e/artifacts", + "labels": [ + "real-web,sev-broken" + ], + "assigned": [], + "author": "piefev", + "state": "open", + "created_at": "2026-06-04T17:15:12Z" + }, + { + "id": 335, + "repo": "we", + "title": "Snapshot opera.com hydration bundles for offline parity (policy b, parent #281)", + "body": "Parent: #281. Policy decision recorded in #306: parity bar is (b) — complete offline snapshots that hydrate to match the live Chromium golden.\n\nopera.com is delivered as a client-side-rendered SPA; the committed snapshot is the static shell and renders blank/partial offline. Work: capture opera.com's JS hydration bundle graph + images, commit under crates/e2e/real-web/snapshots/opera.com/, seed via `cache_put` in opera.com.we, confirm `we` hydrates offline. Keep `# xfail` until the render matches the Chromium golden. Repro: cargo run -p we-e2e -- --scenario crates/e2e/scenarios/real-web/opera.com.we --out-dir crates/e2e/artifacts", + "labels": [ + "real-web,sev-broken" + ], + "assigned": [], + "author": "piefev", + "state": "open", + "created_at": "2026-06-04T17:15:25Z" + }, + { + "id": 336, + "repo": "we", + "title": "Snapshot palmplaystore.com hydration bundles for offline parity (policy b, parent #282)", + "body": "Parent: #282. Policy decision recorded in #306: parity bar is (b).\n\npalmplaystore.com is a Nuxt-rendered SPA; the committed snapshot is the static shell and renders blank/partial offline. Work: capture the Nuxt client bundle graph + __NUXT__ payload + images, commit under crates/e2e/real-web/snapshots/palmplaystore.com/, seed via `cache_put`, confirm `we` hydrates offline. Keep `# xfail` until the render matches the Chromium golden. Repro: cargo run -p we-e2e -- --scenario crates/e2e/scenarios/real-web/palmplaystore.com.we --out-dir crates/e2e/artifacts", + "labels": [ + "real-web,sev-broken" + ], + "assigned": [], + "author": "piefev", + "state": "open", + "created_at": "2026-06-04T17:15:25Z" + }, + { + "id": 337, + "repo": "we", + "title": "Snapshot roblox.com hydration bundles for offline parity (policy b, parent #283)", + "body": "Parent: #283. Policy decision recorded in #306: parity bar is (b).\n\nroblox.com renders a fully white page offline: the entire UI lives in `
` which is empty in the snapshot; the footer/cookie-banner are also empty React mount points. Work: capture the js.rbxcdn.com React bundle graph + images, commit under crates/e2e/real-web/snapshots/roblox.com/, seed via `cache_put`, confirm `we` hydrates the sign-up landing page offline. Keep `# xfail` until the render matches the Chromium golden. Repro: cargo run -p we-e2e -- --scenario crates/e2e/scenarios/real-web/roblox.com.we --out-dir crates/e2e/artifacts", + "labels": [ + "real-web,sev-broken" + ], + "assigned": [], + "author": "piefev", + "state": "open", + "created_at": "2026-06-04T17:15:25Z" + }, + { + "id": 338, + "repo": "we", + "title": "Snapshot tiktok.com hydration bundles for offline parity (policy b, parent #284)", + "body": "Parent: #284. Policy decision recorded in #306: parity bar is (b).\n\ntiktok.com (/@nasa profile) is a client-side-rendered SPA; the committed snapshot is the static shell and renders blank/partial offline. Work: capture TikTok's hydration bundle graph + SIGI/universal-data payload + images, commit under crates/e2e/real-web/snapshots/tiktok.com/, seed via `cache_put`, confirm `we` hydrates the profile offline. Keep `# xfail` until the render matches the Chromium golden. Repro: cargo run -p we-e2e -- --scenario crates/e2e/scenarios/real-web/tiktok.com.we --out-dir crates/e2e/artifacts", + "labels": [ + "real-web,sev-broken" + ], + "assigned": [], + "author": "piefev", + "state": "open", + "created_at": "2026-06-04T17:15:25Z" } ] } diff --git a/PLAN.md b/PLAN.md index a6093d6..35c655f 100644 --- a/PLAN.md +++ b/PLAN.md @@ -465,7 +465,7 @@ If `pick.py` happens to return a domain that already has a committed scenario, r - One scenario per site, named after the domain. Each scenario does at minimum: `goto_as `, `cache_put` for subresources, `screenshot`, `dump_dom`, `dump_console`, plus targeted `assert_dom_contains` / `assert_console_contains` for the page's key content. - Snapshots are committed so the scenario suite is offline-runnable and deterministic. Re-snapshot only when intentionally bumping a baseline, in its own commit. -- **Golden screenshot policy.** The committed golden for each scenario is a Chromium-captured reference at the same viewport: `..chromium.expected.png`. The bar is "we renders this page indistinguishably from Chromium," not "we renders this page the way it did last week." A scenario asserts against the Chromium golden via `assert_screenshot_matches`; the diff threshold is intentionally tight. A scenario whose `we` render does not yet match Chromium is marked `# xfail: see isu issue ` and the gap is tracked as its own defect issue. The legacy self-referential `..expected.png` files are deprecated and scheduled for migration; new scenarios use the Chromium golden from day one. Generate the golden with `python3 tests/popular-sites/compare.py --write-golden --browser chromium `. +- **Golden screenshot policy.** The committed golden for each scenario is a Chromium-captured reference at the same viewport: `..chromium.expected.png`. The bar is "we renders this page indistinguishably from Chromium," not "we renders this page the way it did last week." A scenario asserts against the Chromium golden via `assert_screenshot_matches`; the diff threshold is intentionally tight. A scenario whose `we` render does not yet match Chromium is marked `# xfail: see isu issue ` and the gap is tracked as its own defect issue. The legacy self-referential `..expected.png` files are deprecated and scheduled for migration; new scenarios use the Chromium golden from day one. Generate the golden with `python3 tests/popular-sites/compare.py --write-golden --browser chromium `. For client-side-rendered (SPA) sites the golden is captured live and fully hydrated, so the committed offline snapshot must be completed with every hydration bundle + asset (seeded via `cache_put`) until `we` hydrates it offline to match — this is **policy (b), decided in #306** (the rejected alternative (a) was to recapture goldens from the static offline snapshot, which compares blank-vs-blank and lowers the bar). SPA snapshots that do not yet hydrate stay `# xfail` with per-site hydration issues (see #306). - **Interactivity policy.** Each committed scenario must include at least one *interactivity* assertion — a `click`, `type`, focus change, or equivalent user-driven action — that demonstrates the page does what a user would expect it to do, not just that it paints the right pixels on first load. The DSL commands required to express this are tracked under their own `real-web` issues; until they land, scenarios are explicitly marked as not yet meeting the interactivity bar. - Each fix lands in its own merge to `main`, references the `isu` issue, and includes (a) the scenario asserting the bug is gone and (b) any narrower unit test that catches the regression at a lower layer. diff --git a/crates/e2e/real-web/README.md b/crates/e2e/real-web/README.md index 1dca6fa..763409b 100644 --- a/crates/e2e/real-web/README.md +++ b/crates/e2e/real-web/README.md @@ -33,6 +33,22 @@ output. Every new scenario asserts against a Chromium-captured reference PNG; the older `..expected.png` files are deprecated and scheduled for migration on a per-scenario basis (tracked in `isu`). +### SPA goldens: policy (b), decided in #306 + +For client-side-rendered (SPA) sites the committed snapshot is the static +shell the server returns before JS runs, while the Chromium golden was +captured live and fully hydrated — so an offline `we` render of an empty +mount point (e.g. roblox's empty `#react-landing-container`, bing's empty +`#hp_app`) can never match the hydrated golden. #306 weighed two options: +(a) recapture goldens from the offline snapshot (apples-to-apples but +blank-vs-blank, lowering the bar), or (b) complete each snapshot with every +hydration bundle + asset so the offline run hydrates to match the live +golden. **The maintainer chose (b).** The Chromium golden stays the bar; a +SPA whose offline snapshot does not yet hydrate is marked `# xfail` and its +hydration-bundle snapshotting is tracked as its own `real-web` issue +(bing #334, opera #335, palmplaystore #336, roblox #337, tiktok #338, +parents #280–284). + ### Capturing or refreshing a Chromium golden ```sh