From ec68f9d157dab0dc56fc753e2668c89be0ea88a3 Mon Sep 17 00:00:00 2001 From: Orual Date: Tue, 24 Feb 2026 00:28:10 -0500 Subject: [PATCH] docs: update project context for dfgraph feature --- CLAUDE.md | 27 ++++++++++++++++++++---- asm/CLAUDE.md | 4 ++-- dfgraph/CLAUDE.md | 52 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 77 insertions(+), 6 deletions(-) create mode 100644 dfgraph/CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md index bfaf9cb..0f17410 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -72,6 +72,13 @@ jj bookmark set emu -r @- - `asm/allocate.py` — IRAM offset and context slot allocation - `asm/codegen.py` — Code generation (direct mode + token stream mode) - `asm/serialize.py` — IRGraph to dfasm source serializer +- `dfgraph/` — Interactive dataflow graph renderer (see `dfgraph/CLAUDE.md`) + - `dfgraph/__main__.py` — CLI: `python -m dfgraph path/to/file.dfasm [--port 8420]` + - `dfgraph/pipeline.py` — Progressive pipeline runner (parse -> lower -> resolve -> place -> allocate with error accumulation) + - `dfgraph/categories.py` — Opcode-to-category mapping via isinstance dispatch on ALUOp hierarchy + - `dfgraph/graph_json.py` — IRGraph-to-JSON conversion for frontend consumption + - `dfgraph/server.py` — FastAPI backend with WebSocket push and file watcher (watchdog, 300ms debounce) + - `dfgraph/frontend/` — TypeScript frontend: Cytoscape.js graph with ELK layout, SVG/PNG export - `tests/` — pytest + hypothesis test suite - `tests/conftest.py` — Hypothesis strategies for token/op generation - `docs/` — Design documents, implementation plans, test plans @@ -81,6 +88,9 @@ jj bookmark set emu -r @- - Python 3.12 - SimPy 4.1 (discrete event simulation) - Lark (Earley parser for dfasm grammar) +- FastAPI + uvicorn (dfgraph web server) +- watchdog (file system monitoring for dfgraph live reload) +- Cytoscape.js + cytoscape-elk (frontend graph rendering and layout) - pytest + hypothesis (property-based testing) - Nix flake for dev environment @@ -185,7 +195,7 @@ SimPy process with I-structure (single-assignment) semantics. ### Module Dependency Graph -`cm_inst.py` defines ISA enums and instruction types (no dependencies). `tokens.py` imports from `cm_inst.py` and defines the token hierarchy. `sm_mod.py` is independent. The `emu/` package imports from root-level modules but root-level modules never import from `emu/`. The `asm/` package imports from both root-level modules and `emu/types.py` (for PEConfig/SMConfig), but neither root-level modules nor `emu/` import from `asm/`. +`cm_inst.py` defines ISA enums and instruction types (no dependencies). `tokens.py` imports from `cm_inst.py` and defines the token hierarchy. `sm_mod.py` is independent. The `emu/` package imports from root-level modules but root-level modules never import from `emu/`. The `asm/` package imports from both root-level modules and `emu/types.py` (for PEConfig/SMConfig), but neither root-level modules nor `emu/` import from `asm/`. The `dfgraph/` package imports from `cm_inst`, `asm/` (ir, lower, resolve, place, allocate, errors, opcodes), and internally between its own modules. Neither root-level modules, `emu/`, nor `asm/` import from `dfgraph/`. ``` cm_inst.py <-- tokens.py <-- emu/types.py @@ -200,8 +210,17 @@ asm/ir.py <-- asm/opcodes.py asm/codegen.py | | | v v v asm/lower.py asm/resolve.py asm/allocate.py - | - asm/place.py + | | + | asm/place.py + | | + +--- dfgraph/pipeline.py ----------+ + | + dfgraph/categories.py dfgraph/graph_json.py + (cm_inst) (asm/ir, asm/opcodes) + | + dfgraph/server.py + | + dfgraph/frontend/ ``` - + diff --git a/asm/CLAUDE.md b/asm/CLAUDE.md index 11bf212..9a561e8 100644 --- a/asm/CLAUDE.md +++ b/asm/CLAUDE.md @@ -23,7 +23,7 @@ Translates dfasm graph assembly source into emulator-ready configurations. Bridg ## Dependencies - **Uses**: `cm_inst` (Port, MemOp, CfgOp, ALUOp, ALUInst, SMInst, Addr), `tokens` (MonadToken, SMToken, CfgToken, LoadInstToken, RouteSetToken), `sm_mod` (Presence), `emu/types` (PEConfig, SMConfig), `lark` (parser) -- **Used by**: Test suite, user programs +- **Used by**: Test suite, user programs, `dfgraph/` (pipeline, graph_json use ir, lower, resolve, place, allocate, errors, opcodes) - **Boundary**: `emu/` and root-level modules must NEVER import from `asm/` ## Key Decisions @@ -52,4 +52,4 @@ Translates dfasm graph assembly source into emulator-ready configurations. Bridg - `MemOp.WRITE` arity depends on const: monadic when const is set (cell_addr from const), dyadic when const is None (cell_addr from left operand) - `RoutingOp.FREE_CTX` (ALU context deallocation) and `MemOp.FREE` (SM free) are disambiguated by mnemonic: assembler uses `free_ctx` for ALU and `free` for SM - + diff --git a/dfgraph/CLAUDE.md b/dfgraph/CLAUDE.md new file mode 100644 index 0000000..2e1535b --- /dev/null +++ b/dfgraph/CLAUDE.md @@ -0,0 +1,52 @@ +# Dataflow Graph Renderer (dfgraph/) + +Last verified: 2026-02-24 + +## Purpose + +Interactive visualisation tool for dfasm dataflow programs. Renders the assembler's IR as a live-updating graph in the browser, enabling developers to see node placement, edge routing, and pipeline errors as they edit source files. + +## Contracts + +- **Exposes**: `python -m dfgraph path/to/file.dfasm [--port 8420]` launches a web server with WebSocket-driven graph updates +- **Guarantees**: Progressive pipeline captures the deepest successful IRGraph even when later passes fail, so partial graphs are always shown. Parse errors are fatal (no graph); resolve/place/allocate errors accumulate in `IRGraph.errors` and are displayed alongside the partial graph. File changes trigger reassembly within 300ms debounce window. +- **Expects**: Valid path to a `.dfasm` file. The `dfasm.lark` grammar file at project root. Node modules installed in `dfgraph/frontend/` for the TypeScript build. + +## Dependencies + +- **Uses**: `cm_inst` (ArithOp, LogicOp, RoutingOp, MemOp, CfgOp, Addr), `asm/` (ir, lower, resolve, place, allocate, errors, opcodes), `lark` (parser), `fastapi`/`uvicorn` (server), `watchdog` (file watcher), `cytoscape`/`cytoscape-elk` (frontend) +- **Used by**: Developer tooling only (not imported by any other package) +- **Boundary**: `cm_inst`, `asm/`, `emu/`, and root-level modules must NEVER import from `dfgraph/` + +## Key Decisions + +- **isinstance dispatch for categories** (`categories.py`): ALUOp subclasses (ArithOp, LogicOp, RoutingOp) share IntEnum values across types, so dict/set lookup would collide. isinstance checks on the type hierarchy avoid this. +- **LogicOp split**: EQ/LT/LTE/GT/GTE map to COMPARISON; AND/OR/XOR/NOT map to LOGIC. This gives visually distinct colours for boolean-producing vs bitwise ops. +- **CONST and FREE_CTX as CONFIG**: RoutingOp.CONST and FREE_CTX are categorised as CONFIG (not ROUTING) since they are infrastructure ops, not data-movement ops. +- **ELK layout engine**: Uses Eclipse Layout Kernel (via cytoscape-elk) for layered DAG layout with orthogonal edge routing. Replaced dagre for better edge routing. +- **Progressive pipeline**: Unlike `asm._run_pipeline()` which raises on error, `run_progressive()` runs each pass independently so the frontend can show partial graphs with error annotations. + +## Invariants + +- `dfgraph/` never imports from `emu/` -- it only needs the assembler IR, not the simulator +- `categorise()` covers all ALUOp/MemOp/CfgOp values; raises `ValueError` on unknown types +- `graph_to_json()` always returns a dict with `type: "graph_update"` -- even on parse failure (with empty node/edge lists) +- WebSocket protocol: server sends `GraphUpdate` JSON on connect and on every file change; client sends nothing meaningful (receive loop is just for keepalive) + +## Key Files + +- `pipeline.py` -- `PipelineStage` enum, `PipelineResult` dataclass, `run_progressive()` function +- `categories.py` -- `OpcodeCategory` enum, `CATEGORY_COLOURS` dict, `categorise()` function +- `graph_json.py` -- `graph_to_json(PipelineResult) -> dict` serialisation +- `server.py` -- `create_app(source_path) -> FastAPI`, `ConnectionManager`, `DebouncedFileHandler` +- `frontend/src/types.ts` -- TypeScript interfaces matching `graph_to_json` output shape +- `frontend/src/main.ts` -- Cytoscape graph rendering, logical/physical view toggle +- `frontend/src/layout.ts` -- ELK layout configurations for both views + +## Gotchas + +- The Lark parser is lazily initialised and cached globally in `pipeline.py` (`_parser` module var) -- safe for single-process use but not for parallel test runners that fork +- `DebouncedFileHandler` runs its callback on a `threading.Timer` thread, so `_on_file_change` uses `asyncio.run_coroutine_threadsafe` to bridge into the async event loop +- Frontend `dist/` directory must contain the compiled TypeScript bundle; the server mounts it as static files at `/dist` + + -- 2.51.2