diff --git a/specs/clayterm-virtualizer-spec.md b/specs/clayterm-virtualizer-spec.md new file mode 100644 index 0000000..3c2e88c --- /dev/null +++ b/specs/clayterm-virtualizer-spec.md @@ -0,0 +1,233 @@ +# `@clayterm/virtualizer` Current-State Specification + +**Status:** Current-state specification for the implementation on `feat/virtualizer-v1`. + +This document was imported from the draft virtualizer spec in `~/Downloads` and updated to match the current implementation in this repository. It is intended to describe the implemented package surface and behavior as it exists in-tree today. Where the implementation is known to be more provisional or implementation-shaped than the original draft contract, that is called out explicitly. + +## 1. Scope + +`@clayterm/virtualizer` is a TypeScript package that provides viewport virtualization over large terminal text output. It owns: + +- ring-buffer storage of logical lines +- per-line cached display width +- per-line wrap-point caching at the current column width +- anchor-based scrolling +- viewport resolution +- approximate scrollbar-estimation fields + +It does not own: + +- ANSI style parsing beyond CSI/OSC skipping for width and wrapping +- rendering, layout, or terminal output +- input normalization +- input parsing +- search, selection, filtering, or disk-backed scrollback + +## 2. Package Boundary + +The package lives in [`virtualizer/`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer) and exports: + +- [`Virtualizer`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/virtualizer.ts) +- [`VirtualizerOptions`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/types.ts) +- [`ViewportEntry`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/types.ts) +- [`ResolvedViewport`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/types.ts) + +The package has no runtime import of `@clayterm/clayterm`. Width measurement is injected by the caller. + +The intended width provider from the renderer package is `createDisplayWidth()` from root [`width.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/width.ts), which is an async factory returning a synchronous `(text: string) => number` function. + +## 3. Public API + +The current implemented API is: + +```ts +class Virtualizer { + constructor(options: VirtualizerOptions); + + readonly lineCount: number; + readonly baseIndex: number; + readonly columns: number; + readonly rows: number; + readonly totalEstimatedVisualRows: number; + readonly currentEstimatedVisualRow: number; + readonly isAtBottom: boolean; + readonly anchorLineIndex: number; + readonly anchorSubRow: number; + + appendLine(text: string): number; + resize(columns: number, rows: number): void; + scrollBy(deltaVisualRows: number): void; + scrollToFraction(fraction: number): void; + resolveViewport(): ResolvedViewport; + + getLineDisplayWidth(lineIndex: number): number | undefined; +} + +interface VirtualizerOptions { + measureWidth: (text: string) => number; + maxLines?: number; + columns: number; + rows: number; +} + +interface ViewportEntry { + lineIndex: number; + text: string; + wrapPoints: number[]; + totalSubRows: number; + firstSubRow: number; + visibleSubRows: number; +} + +interface ResolvedViewport { + entries: ViewportEntry[]; + totalEstimatedVisualRows: number; + currentEstimatedVisualRow: number; + isAtBottom: boolean; +} +``` + +Notes: + +- `appendLine()` returns the assigned monotonic `lineIndex`. +- `resolveViewport()` reads `columns` and `rows` from internal state and takes no parameters. +- `getLineDisplayWidth()` is a current implemented API for exposing cached per-line display width. It returns `undefined` for evicted or never-assigned indices. + +## 4. Width Model And ANSI Handling + +The current implementation expects `measureWidth(text)` to be: + +- synchronous +- ANSI-agnostic +- additive across concatenation +- based on per-codepoint `wcwidth` semantics + +The virtualizer owns ANSI CSI/OSC skipping internally. + +Recognized sequences: + +- CSI: `ESC [` ... final byte in `0x40..0x7E` +- OSC: `ESC ]` ... terminated by BEL or `ESC \` + +Current behavior: + +- recognized CSI/OSC bytes are skipped for both cached display width and wrap computation +- `measureWidth` is called only on visible codepoints +- unrecognized escape families are not skipped +- no ANSI style state is carried between lines + +Implementation note: + +- unterminated CSI/OSC sequences are currently consumed to the end of the string by the scanner in [`virtualizer/ansi-scanner.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/ansi-scanner.ts) + +## 5. Data Model + +The virtualizer maintains: + +- a ring buffer of `{ text, displayWidth, lineIndex }` +- `baseIndex` +- a monotonic next-line counter +- current `columns` and `rows` +- anchor state: `(anchorLineIndex, anchorSubRow, isAtBottom)` +- `totalEstimatedVisualRows` +- `currentEstimatedVisualRow` +- a wrap cache `Map` + +Current storage implementation lives in [`virtualizer/ring-buffer.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/ring-buffer.ts). + +Current empty-buffer behavior: + +- `lineCount === 0` +- `totalEstimatedVisualRows === 0` +- `currentEstimatedVisualRow === 0` +- `isAtBottom === true` +- `resolveViewport().entries` is empty +- `scrollBy()` and `scrollToFraction()` are no-ops + +## 6. Operations + +### 6.1 `appendLine(text)` + +Current implementation flow: + +1. Compute `displayWidth` with ANSI skipping. +2. If the buffer is at capacity, evict the oldest line before insertion. +3. Delete the evicted line’s wrap-cache entry. +4. Decrement `totalEstimatedVisualRows` by the evicted line’s estimated rows. +5. Adjust anchor and current estimate if eviction occurred before the anchor. +6. Store the new line with the next monotonic `lineIndex`. +7. Increment `totalEstimatedVisualRows` by `max(1, ceil(displayWidth / columns))`. +8. If `isAtBottom`, advance anchor to the new line at sub-row 0. + +### 6.2 `scrollBy(deltaVisualRows)` + +Current implementation: + +- walks exact sub-rows using cached or freshly computed wrap points +- clamps at top and bottom +- clears `isAtBottom` when scrolling up from bottom +- sets `isAtBottom` when clamped to the last exact sub-row of the last line +- recomputes `currentEstimatedVisualRow` from estimated line prefixes plus the anchor sub-row + +### 6.3 `scrollToFraction(fraction)` + +Current implementation: + +- maps the fraction into an estimated target row +- walks from `baseIndex` using estimated rows per line +- lands on an estimated sub-row within the selected line +- recomputes `currentEstimatedVisualRow` after updating the anchor + +### 6.4 `resize(columns, rows)` + +Current implementation: + +- clears the wrap cache on column change +- recomputes `totalEstimatedVisualRows` by summing `ceil(displayWidth / columns)` +- updates `columns` +- recomputes `currentEstimatedVisualRow` +- updates `rows` + +Current implementation detail: + +- anchor sub-row clamping is currently based on the estimated row count for the anchor line at the new width, not an exact re-wrap of that line + +### 6.5 `resolveViewport()` + +Current implementation: + +1. Walks forward from the anchor, filling as many rows as possible. +2. If the forward walk does not fill the viewport, it backfills upward: + - first by expanding the anchor line upward when `anchorSubRow > 0` + - then by including preceding lines +3. Returns ordered `ViewportEntry` records plus estimation metadata. + +Each `ViewportEntry` contains the original `text`, all wrap points for that line, and a visible sub-row window. + +## 7. Output Invariants The Implementation Intends To Satisfy + +The current test suite checks the following output invariants through public API behavior: + +- `entries` ordered by ascending `lineIndex` +- `text` preserved exactly +- `wrapPoints` strictly increasing +- no wrap point inside surrogate pairs +- no wrap point inside recognized CSI/OSC +- `totalSubRows === wrapPoints.length + 1` +- `firstSubRow` and `visibleSubRows` stay within bounds +- total visible sub-rows do not exceed the viewport row budget +- every visible slice fits within `columns` +- output can be reconstructed from `text + wrapPoints` + +## 8. Current Implementation Notes And Known Gaps + +These notes document the current branch shape rather than defining desired future contract. + +- The renderer-side width provider is exposed as `createDisplayWidth(): Promise<(text: string) => number>`, not a directly callable synchronous top-level export. +- `getLineDisplayWidth()` is currently public to support direct verification of cached display width. +- The wrap golden fixture for `abc文d` is currently exercised at `columns = 4` in [`virtualizer/test/wrap-golden.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/wrap-golden.test.ts), reflecting the corrected boundary case used by the branch’s tests rather than the older draft wording from Downloads. +- The implementation and tests should be treated as the source of truth for current behavior where they diverge from the older draft. + +## 9. Source Alignment + +This file replaces the earlier draft-only location in `~/Downloads` with an in-repo copy under [`specs/`](/Users/tarasmankovski/Repositories/frontside/clayterm/specs). It is intended to track the implemented branch and should be updated together with public API or behavioral changes in [`virtualizer/`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer). diff --git a/specs/clayterm-virtualizer-test-plan.md b/specs/clayterm-virtualizer-test-plan.md new file mode 100644 index 0000000..4d6bc5a --- /dev/null +++ b/specs/clayterm-virtualizer-test-plan.md @@ -0,0 +1,216 @@ +# `@clayterm/virtualizer` Current-State Test Plan + +**Status:** Current-state test plan for the implementation in this repository. + +This document was imported from the draft virtualizer test plan in `~/Downloads` and updated to reflect the test suites, filenames, and assertion strategy that exist on `feat/virtualizer-v1`. + +## 1. Scope + +This plan covers: + +- root renderer width-provider tests for `createDisplayWidth()` +- virtualizer conformance-style tests in [`virtualizer/test/`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test) +- golden fixtures +- real-world ANSI fixtures +- property-style randomized tests +- informational benchmark gates + +Current suite count: + +- root renderer width tests: 7 +- virtualizer tests: 113 + +These counts reflect the current branch contents. + +## 2. Test Inventory By File + +Root package: + +- [`test/width.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/test/width.test.ts): renderer width-provider tests for `createDisplayWidth()` + +Virtualizer package: + +- [`virtualizer/test/invariants.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/invariants.test.ts) +- [`virtualizer/test/width.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/width.test.ts) +- [`virtualizer/test/ansi.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/ansi.test.ts) +- [`virtualizer/test/append.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/append.test.ts) +- [`virtualizer/test/empty.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/empty.test.ts) +- [`virtualizer/test/viewport.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/viewport.test.ts) +- [`virtualizer/test/wrap-golden.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/wrap-golden.test.ts) +- [`virtualizer/test/ansi-golden.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/ansi-golden.test.ts) +- [`virtualizer/test/real-world-ansi.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/real-world-ansi.test.ts) +- [`virtualizer/test/scroll.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/scroll.test.ts) +- [`virtualizer/test/fraction.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/fraction.test.ts) +- [`virtualizer/test/eviction.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/eviction.test.ts) +- [`virtualizer/test/resize.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/resize.test.ts) +- [`virtualizer/test/exactness.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/exactness.test.ts) +- [`virtualizer/test/property.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/property.test.ts) +- [`virtualizer/test/bench.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/bench.test.ts) + +## 3. Assertion Strategy + +The current branch uses three broad test styles: + +- deterministic conformance-style tests +- reference-behavior tests for exact formula/provider expectations +- informational benchmarks + +Current observability surfaces used by tests: + +- `appendLine()` return value +- read-only `Virtualizer` state getters +- `resolveViewport()` output +- `getLineDisplayWidth(lineIndex)` +- instrumented `measureWidth` mocks + +The addition of `getLineDisplayWidth()` is important for current coverage because several width assertions now verify cached width directly rather than inferring it from row counts. + +## 4. Traceability To Current Suites + +### 4.1 Renderer width-provider + +Covered in [`test/width.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/test/width.test.ts): + +- `R.WIDTH.ascii` +- `R.WIDTH.cjk` +- `R.WIDTH.combining` +- `R.WIDTH.zwj-emoji` +- `R.WIDTH.additivity` +- `R.WIDTH.empty-string` +- `R.WIDTH.zero-width` + +These validate `createDisplayWidth()` as the current reference provider. + +### 4.2 Core identity and append behavior + +Covered in: + +- [`virtualizer/test/invariants.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/invariants.test.ts) +- [`virtualizer/test/append.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/append.test.ts) +- [`virtualizer/test/empty.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/empty.test.ts) + +Current assertions include: + +- monotonic line indices +- identity survival across eviction +- identity non-reuse +- empty-buffer semantics +- bottom-follow behavior +- append behavior while scrolled up +- wrap-cache stability across append + +### 4.3 Width and ANSI handling + +Covered in: + +- [`virtualizer/test/width.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/width.test.ts) +- [`virtualizer/test/ansi.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/ansi.test.ts) +- [`virtualizer/test/ansi-golden.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/ansi-golden.test.ts) +- [`virtualizer/test/real-world-ansi.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/real-world-ansi.test.ts) + +Current assertions include: + +- `measureWidth` never sees recognized ANSI bytes +- cached display width excludes recognized CSI/OSC +- unrecognized escape forms are not skipped +- wrap points never land inside recognized sequences +- no ANSI state leaks across lines +- captured ANSI fixtures behave consistently at multiple widths + +### 4.4 Viewport invariants and wrapping fixtures + +Covered in: + +- [`virtualizer/test/viewport.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/viewport.test.ts) +- [`virtualizer/test/wrap-golden.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/wrap-golden.test.ts) + +Current assertions include: + +- ordering and text preservation +- wrap-point monotonicity and bounds +- surrogate-pair safety +- visible row budget +- slice width within columns +- self-contained reconstruction + +Current fixture note: + +- the branch’s `G.WRAP.cjk-boundary` fixture uses `abc文d` at `columns = 4`, matching the corrected test expectation in code + +### 4.5 Scroll, fraction, eviction, and resize + +Covered in: + +- [`virtualizer/test/scroll.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/scroll.test.ts) +- [`virtualizer/test/fraction.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/fraction.test.ts) +- [`virtualizer/test/eviction.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/eviction.test.ts) +- [`virtualizer/test/resize.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/resize.test.ts) + +Current assertions include: + +- top and bottom clamping +- `isAtBottom` transitions +- fractional jumps to top and bottom +- eviction anchor recovery +- viewport stability when eviction removes older lines +- cache invalidation across resize +- `displayWidth` stability across resize via `getLineDisplayWidth()` + +## 5. Property And Benchmark Coverage + +Property-style randomized coverage lives in [`virtualizer/test/property.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/property.test.ts): + +- identity monotonicity and non-reuse +- eviction stability +- append independence from scroll position +- viewport invariants after random operations +- estimation-field constraints +- resize invariant preservation + +Informational benchmark coverage lives in [`virtualizer/test/bench.test.ts`](/Users/tarasmankovski/Repositories/frontside/clayterm/virtualizer/test/bench.test.ts): + +- Gate 1: width-provider overhead +- Gate 2: extended ANSI corpus +- Gate 3: estimate vs exact wrap-count match rate +- Gate 4: `scrollToFraction` performance +- Gate 5: skip-optimization comparison +- Gate 6: ANSI-heavy viewport resolution ratio +- Gate 7: resize performance + +These are informational measurements, not conformance gates. + +## 6. Commands + +Run root width-provider tests: + +```sh +deno test --allow-read test/width.test.ts +``` + +Run virtualizer tests: + +```sh +cd virtualizer && deno test +``` + +## 7. Current-State Notes + +This plan is intentionally aligned to the branch implementation, not the older Downloads draft verbatim. + +Notable updates from the older draft: + +- width assertions now use `getLineDisplayWidth()` directly where the branch exposes it +- the renderer width-provider surface is `createDisplayWidth()` rather than a direct synchronous export +- the current suite count is lower than the draft’s projected count because the implementation consolidates multiple properties into fewer files and parameterized tests +- the `wrap-golden` boundary fixture has been corrected to the current checked-in test case + +## 8. Maintenance + +This file should be updated when: + +- the `Virtualizer` public API changes +- test files are renamed or reorganized +- current assertion strategy changes +- additional real-world fixtures or benchmark gates are added + +The in-repo `specs/` directory is now the canonical location for these virtualizer docs.