diff --git a/.agents/memory/ACTIVE/PLAN.md b/.agents/memory/ACTIVE/PLAN.md
index db0ce79..81fabeb 100644
--- a/.agents/memory/ACTIVE/PLAN.md
+++ b/.agents/memory/ACTIVE/PLAN.md
@@ -31,6 +31,22 @@ but no streaming or unist compatibility.
- HTML rendering (direct compilation is a convenience, not a goal)
- MediaWiki behavioral quirk-matching (deferred to "mediawiki" profile)
+## Scope discipline
+
+- Use wikitext as the proving ground for parser primitives before expanding
+ into a broader profile-driven document engine.
+- Treat the parser as one simple workflow step in the larger future system:
+ its job is to produce correct primitives that downstream transforms,
+ renderers, editors, and session-based tools can consume.
+- The longer-term direction includes other markup and rich-text profiles,
+ structured CMS blocks, local-first collaboration, offline or local sync,
+ and LLM-oriented workflows.
+- Unified ecosystem support remains desirable during that transition, but as
+ optional adapters over unist-compatible exports rather than as the core
+ runtime architecture.
+- Do not let that broader direction dilute current work on the wikitext parser
+ itself.
+
## Approach
See `docs/architecture.md` for the full pipeline design. In summary:
diff --git a/.agents/memory/ACTIVE/PROGRESS.md b/.agents/memory/ACTIVE/PROGRESS.md
index 57e1fe1..fbb869a 100644
--- a/.agents/memory/ACTIVE/PROGRESS.md
+++ b/.agents/memory/ACTIVE/PROGRESS.md
@@ -12,9 +12,15 @@
- Test imports use deno-lint-ignore comments with inline jsr:/npm: specifiers (no deno.json import map)
- Benchmarks GC-annotated for allocation-heavy tokenizer paths
- State snapshot recording deferred to Phase 7 (TODO comment in block_parser.ts)
-- Current task: T06 (Inline parser implementation, Phase 4) — not started
+- Current task: Phase 4 complete; Phase 5 orchestration not started
+- Longer-term direction: broader profile-driven document engine is recorded,
+ but current focus remains validating parser primitives through the wikitext
+ parser first
+- Ecosystem direction: unified compatibility stays desirable, but through
+ optional adapters at the edge while the native runtime model remains the
+ long-term target
- Blockers: none
-- Total tests: 448 across 8 test files, 0 failures, 0 compile errors
+- Total tests: 479, 0 failures, 0 compile errors
## Completed
@@ -116,6 +122,23 @@
custom TextSource impl, property-based round-trips
- `deno task test` passes (448 total tests)
+- [x] T06: Implement inline parser (Phase 4)
+ - `inline_parser.ts`: offset-driven inline enrichment generator
+ - Covers apostrophe emphasis, wikilinks, image/category namespace dispatch,
+ bracketed and bare external links, templates, parser functions,
+ triple-brace arguments, comments, behavior switches, signatures,
+ HTML entities, `
`, ``, `[`, and generic HTML tags
+ - Adjacent block-parser text spans are merged before enrichment so inline
+ constructs can span token-sized text events
+ - Position recovery uses line-start tables instead of per-character `Point`
+ allocation in the hot path
+ - `inline_parser_test.ts`: focused examples, behavior-switch edge cases, and
+ property-based invariants
+ - `mod.ts`: re-exports `inlineEvents()`
+ - `mod_bench.ts`: now benchmarks token-only, block-events, and full inline
+ enrichment paths on representative inputs
+ - `deno task test` passes (479 total tests)
+
## Notes for the next agent
- Conventions:
@@ -149,7 +172,6 @@
- State snapshot recording is deferred to Phase 7 (TODO comment in place)
- blockEvents() accepts (source: TextSource, tokens: Iterable)
- What's NOT implemented yet:
- - inline_parser.ts
- parse.ts, tree_builder.ts, stringify.ts, filter.ts
- session.ts
- Verification commands:
diff --git a/.agents/memory/ACTIVE/RISKS.md b/.agents/memory/ACTIVE/RISKS.md
index cc842e2..0b55d59 100644
--- a/.agents/memory/ACTIVE/RISKS.md
+++ b/.agents/memory/ACTIVE/RISKS.md
@@ -41,6 +41,10 @@
`{{SomeTemplate}}` (template). Default: parse all non-`#`-prefixed as
Template. Risk: profiles may need a word list, adding configuration
burden.
+- **Ecosystem adapter pressure**: Reusing unified ecosystem plugins is useful,
+ but letting unified-style assumptions leak into the hot path could distort
+ the event-stream-first runtime and incremental design. Mitigation: keep
+ unified support at the adapter boundary over unist-compatible exports.
- **Heading close marker token mismatch (discovered, resolved)**: The
tokenizer emits `EQUALS` (not `HEADING_MARKER_CLOSE`) for trailing `==`
in headings. The block parser originally checked for `HEADING_MARKER_CLOSE`
diff --git a/.agents/memory/ACTIVE/TASKS.md b/.agents/memory/ACTIVE/TASKS.md
index 91c288d..1fb228e 100644
--- a/.agents/memory/ACTIVE/TASKS.md
+++ b/.agents/memory/ACTIVE/TASKS.md
@@ -90,22 +90,22 @@ Rules:
- [x] `deno task test` passes (448 total tests)
- [x] Test imports use deno-lint-ignore comments (no deno.json import map)
-- [ ] T06: Implement inline parser (Phase 4)
+- [x] T06: Implement inline parser (Phase 4)
- Why: Inline markup enrichment is the next layer above block parsing
- Done when:
- - [ ] `inline_parser.ts` exports inline event enrichment generator
- - [ ] Bold/italic: apostrophe run disambiguation (2=italic, 3=bold, 5=bold+italic)
- - [ ] Wikilinks: `[[target|display]]` with namespace dispatch
- - [ ] External links: `[url text]` and bare URLs
- - [ ] Templates: `{{name|args}}` with named/positional arguments
- - [ ] Template arguments: `{{{param}}}` as Argument nodes
- - [ ] Parser functions: `{{#if:...|...}}` classified by `#` prefix
- - [ ] HTML tags: inline `][`, ``, etc.
- - [ ] Event well-formedness: every enter has matching exit
- - [ ] Never-throw: any input produces valid events
- - [ ] Property-based fuzz tests with fast-check
- - [ ] `deno task test` passes
- - [ ] `mod.ts` re-exports inline parser
+ - [x] `inline_parser.ts` exports inline event enrichment generator
+ - [x] Bold/italic: apostrophe run disambiguation (2=italic, 3=bold, 5=bold+italic)
+ - [x] Wikilinks: `[[target|display]]` with namespace dispatch
+ - [x] External links: `[url text]` and bare URLs
+ - [x] Templates: `{{name|args}}` with named/positional arguments
+ - [x] Template arguments: `{{{param}}}` as Argument nodes
+ - [x] Parser functions: `{{#if:...|...}}` classified by `#` prefix
+ - [x] HTML tags: inline `][`, ``, etc.
+ - [x] Event well-formedness: every enter has matching exit
+ - [x] Never-throw: any input produces valid events
+ - [x] Property-based fuzz tests with fast-check
+ - [x] `deno task test` passes
+ - [x] `mod.ts` re-exports inline parser
## Parking lot
diff --git a/.agents/memory/CONVENTIONS.md b/.agents/memory/CONVENTIONS.md
index 76b83ca..54ccbbf 100644
--- a/.agents/memory/CONVENTIONS.md
+++ b/.agents/memory/CONVENTIONS.md
@@ -21,6 +21,10 @@ survives context resets.
- Update `ACTIVE/PROGRESS.md` after meaningful progress.
- Mark tasks done only when acceptance checks pass.
- Promote architectural decisions to ADRs in `DECISIONS/`.
+- Keep the active scope on the wikitext parser even when broader future
+ platform ideas are discussed.
+- Treat unified ecosystem support as an optional adapter boundary, not the
+ core runtime model.
- Run `deno task test`, `deno task bench`, and `deno doc --lint mod.ts` before
marking any API-touching task complete.
- The parser never throws: enforce the never-throw invariant in every change.
diff --git a/.agents/memory/GLOSSARY.md b/.agents/memory/GLOSSARY.md
index 6e2e058..443cce2 100644
--- a/.agents/memory/GLOSSARY.md
+++ b/.agents/memory/GLOSSARY.md
@@ -43,6 +43,19 @@
`variants: WikistNode[][]`). Represents structurally divergent
interpretations of the same source range (jj-inspired). Not produced by
the core parser; intended for collab/merge tooling.
+- **Profile**: A named syntax and behavior configuration layered on the core
+ parser primitives. A profile can tune classification, parsing rules,
+ recovery behavior, and later editing or rendering semantics for a markup
+ family.
+- **Native runtime**: The project's own execution model built around tokens,
+ events, trees, sessions, streaming, and profile-driven behavior. This is
+ distinct from unified and remains the intended long-term center of gravity.
+- **Unified adapter**: Optional compatibility layer that exposes wikist or
+ related trees through unified parser/compiler or bridge plugins so existing
+ ecosystem plugins can be reused without making unified the core runtime.
+- **Workflow step**: A reminder that the parser is one stage in a larger
+ document pipeline. It should stay focused on producing correct primitives
+ that later consumers, editors, sync layers, and transforms build on.
## Wikitext constructs
diff --git a/.agents/memory/INDEX.md b/.agents/memory/INDEX.md
index 583b786..690f2df 100644
--- a/.agents/memory/INDEX.md
+++ b/.agents/memory/INDEX.md
@@ -2,7 +2,7 @@
## Core references
-- [PROJECT](PROJECT.md)
+- [PROJECT](PROJECT.md): current parser scope, stable architecture, longer-term platform direction
- [CONVENTIONS](CONVENTIONS.md)
- [GLOSSARY](GLOSSARY.md)
diff --git a/.agents/memory/PROJECT.md b/.agents/memory/PROJECT.md
index bc79e0b..7e5e21a 100644
--- a/.agents/memory/PROJECT.md
+++ b/.agents/memory/PROJECT.md
@@ -14,6 +14,22 @@ interchange format.
- Flat file layout at root; `mod.ts` re-exports all public APIs
- Source parser only: no template expansion or HTML rendering
+## Longer-term direction
+
+- Near-term focus stays on finishing the wikitext parser and validating the
+ tokenizer, event stream, tree, and future session primitives there first.
+- The parser is intentionally only one workflow step in the larger system:
+ source text enters the parser, and downstream consumers operate on tokens,
+ events, trees, and later session state.
+- Longer-term, those primitives may expand into a profile-driven structured
+ document engine for other markup and rich-text families, CMS-style blocks,
+ local-first collaboration, offline or local sync, and LLM-friendly flows.
+- Until a native runtime ecosystem exists for those broader goals, the project
+ should still be able to take advantage of unified ecosystem plugins through
+ optional adapters built on unist-compatible exports.
+- Treat that as future platform direction, not a reason to broaden current
+ parser scope prematurely.
+
## Key modules
Implemented:
@@ -24,9 +40,11 @@ Implemented:
- `tokenizer.ts`: charCodeAt generator-based scanner over TextSource
- `block_parser.ts`: block-level event generator (headings, paragraphs, lists,
definition lists, tables, thematic breaks, preformatted blocks)
+- `inline_parser.ts`: inline event enrichment (emphasis, links, templates,
+ arguments, comments, behavior switches, signatures, HTML entities,
+ `]
`, ``, `[`, and generic HTML tags)
Not yet implemented:
-- `inline_parser.ts`: inline event enrichment
- `parse.ts`: orchestration (tokenizer -> block -> inline -> tree)
- `tree_builder.ts`: `buildTree(events) -> WikistRoot`
- `stringify.ts`: AST -> wikitext (round-trip)
@@ -40,6 +58,7 @@ Available now:
- `TokenType`, `Token`, `isToken()` (token.ts)
- `tokenize()` (tokenizer.ts)
- `blockEvents()` (block_parser.ts)
+- `inlineEvents()` (inline_parser.ts)
- `WikitextEvent`, `EnterEvent`, `ExitEvent`, ... + constructors + guards (events.ts)
- `WikistNode`, `WikistRoot`, 37 node types + type guards + builders (ast.ts)
diff --git a/.agents/memory/README.md b/.agents/memory/README.md
index a79799b..f0e4d27 100644
--- a/.agents/memory/README.md
+++ b/.agents/memory/README.md
@@ -16,6 +16,11 @@ live under `SESSIONS` and are gitignored.
Use `ACTIVE` as the control panel for ongoing work, `DECISIONS` for long-lived
architectural choices, and `CHECKLISTS` for repeatable quality gates.
+Keep current parser milestones and the larger platform vision separate. The
+parser remains the active proving ground; broader future direction belongs in
+`PROJECT`, `DECISIONS`, and tightly scoped notes in `ACTIVE`, not in inflated
+task lists.
+
## Edge cases
If `TASKS` grows too large, split into multiple files or an epic folder. If a
]