From f474a992fce222d0f53670005f0bdb9d2b85aa87 Mon Sep 17 00:00:00 2001 From: Owais Jamil Date: Sun, 23 Aug 2026 03:16:06 -0500 Subject: [PATCH] docs: profiling plans --- ROADMAP.md | 20 +++++++++++----- TODO.md | 69 +++++++++++++++++++++++++++++++++++++----------------- 2 files changed, 61 insertions(+), 28 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 23841a6..9141aea 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -62,12 +62,20 @@ export, and repurpose, not substitutes for missing editor behavior. ### Performance profiling -Build a repeatable fixture corpus and measure document, CRDT, renderer, -vector-editing, SVG, CLI, IPC, and MCP workloads on recorded reference -hardware. Set regression budgets from those results. - -Add indexes, caches, incremental materialization, or alternate rendering only -when a representative benchmark identifies the cost they address. +Performance characterization is part of the initial web release path. Replace +July 2026 prototype results with repeatable measurements across shared fixture +sizes and representative document structures. + +Use Criterion for native algorithms, `samply` for confirmed Rust hotspots, +`hyperfine` and coarse `tracing` spans for complete CLI, IPC, and MCP paths, +and Playwright with Chrome DevTools Protocol for actual browser frames and +memory. Keep the no-op Canvas harness only as a renderer traversal benchmark; +it does not measure rasterization, compositing, text, GPU work, or browser GC. + +Set regression budgets from the refreshed native, process, and browser results. +Add indexes, caches, incremental materialization, alternate rendering, or +specialized allocation tooling only when a representative measurement +identifies the cost it addresses. ## Later diff --git a/TODO.md b/TODO.md index 957094f..eba3c81 100644 --- a/TODO.md +++ b/TODO.md @@ -55,29 +55,54 @@ and rendered output where visual fidelity matters. ## Performance profiling -### Corpus and method - -- [ ] Build executable small, medium, large, and 10,000-shape fixtures -- [ ] Add vector-heavy, deeply nested, imported-SVG, and connection-heavy - fixtures -- [ ] Record reference hardware and benchmark methodology - -### Measure - -- [ ] Profile open, save, Automerge load, and materialization -- [ ] Profile validation, commit, undo/redo, merge, queries, and memory -- [ ] Profile Canvas frames, culling, hit testing, nested transforms, and path - rendering -- [ ] Profile selection overlays and vector-edit previews -- [ ] Profile SVG parse/import/export and common CLI operations -- [ ] Profile local IPC and MCP startup, queries, and mutations - -### Optimize from evidence - -- [ ] Record baselines, dominant costs, and regression budgets +### Corpus and harnesses + +- [ ] Build shared 100, 1,000, 5,000, and 10,000-shape fixtures for flat, + vector-heavy, deeply nested, imported-SVG, connection-heavy, and + semantic/binding-heavy documents +- [ ] Replace the stale Node capture with a traversal-only benchmark against + the current editor renderer; keep its no-op canvas scope explicit +- [ ] Add Criterion benchmarks for document load/save/materialization, + transactions, queries, SVG, layout, and renderer algorithms +- [ ] Add a profiling Cargo profile with release optimizations and debug symbols +- [ ] Record reference hardware, tool versions, fixture seeds, warmups, and + sampling methodology + +### Native and process measurements + +- [ ] Benchmark validation, commit, undo/redo, merge, query, layout, and SVG + operations across fixture sizes +- [ ] Use `samply` to attribute native CPU hotspots identified by benchmarks +- [ ] Use `hyperfine` to measure complete CLI inspect, query, validate, render, + and mutate commands plus MCP startup +- [ ] Add coarse `tracing` spans for document, IPC, and MCP operations so + end-to-end latency can be attributed without instrumenting hot geometry + loops + +### Browser measurements + +- [ ] Add Playwright and CDP workloads for load, pan, zoom, box selection, + single- and multi-object drag, vector editing, connected-shape movement, + and nested selection +- [ ] Capture real Chrome frame, paint, raster, compositor, long-task, GC, and + memory data instead of treating the no-op canvas benchmark as browser + rendering performance +- [ ] Measure JS-to-WASM document operations and projection/store updates with + browser performance marks +- [ ] Test heap retention after opening a 10,000-shape board, sustained editing, + and replacing the active document +- [ ] Save compact summaries and diagnostic traces for representative browser + workloads without enabling full Playwright tracing during measurement + +### Baselines and optimization + +- [ ] Refresh the July 2026 baseline and set regression budgets from current + native, process, and browser measurements +- [ ] Record dominant costs and use Criterion deltas, Chrome traces, and + `samply` profiles to investigate regressions - [ ] Evaluate spatial indexes, path/render caches, incremental materialization, - or alternate renderers only when a measured bottleneck supports the - change + alternate renderers, or allocation profilers only when a measured + bottleneck supports the change ## Backlog -- 2.51.2