diff --git a/cm_inst.py b/cm_inst.py index 434a203..ebf37a5 100644 --- a/cm_inst.py +++ b/cm_inst.py @@ -88,6 +88,8 @@ class TokenKind(Enum): class FrameOp(IntEnum): ALLOC = 0 FREE = 1 + ALLOC_SHARED = 2 + FREE_LANE = 3 @dataclass(frozen=True) diff --git a/docs/implementation-plans/2026-03-07-frame-lanes/phase_01.md b/docs/implementation-plans/2026-03-07-frame-lanes/phase_01.md new file mode 100644 index 0000000..7049141 --- /dev/null +++ b/docs/implementation-plans/2026-03-07-frame-lanes/phase_01.md @@ -0,0 +1,271 @@ +# Frame Matching Lanes Implementation Plan + +**Goal:** Extend the PE's frame-based matching to support multiple simultaneous pending operands per instruction within a single activation via matching lanes. + +**Architecture:** Multiple `activation_id` values share one physical frame (constants/destinations) while maintaining independent matching state per lane. Tag store maps `act_id → (frame_id, lane)`. Match data, presence, and port storage gain a lane dimension. + +**Tech Stack:** Python 3.12, SimPy 4.1, pytest + hypothesis + +**Scope:** 6 phases from original design (phases 1-6) + +**Codebase verified:** 2026-03-07 + +--- + +## Acceptance Criteria Coverage + +This phase implements and tests: + +### frame-lanes.AC1: Tag Store Tuple API +- **frame-lanes.AC1.1 Success:** `tag_store` maps `act_id → (frame_id, lane)` where `lane` is an `int` in range `[0, lane_count)`. +- **frame-lanes.AC1.2 Success:** `PEConfig.initial_tag_store` type is `dict[int, tuple[int, int]]`. PE constructor initialises tag_store from it. +- **frame-lanes.AC1.3 Success:** `PEConfig.lane_count` field exists with default 4. Controls third dimension of match arrays. +- **frame-lanes.AC1.4 Success:** All existing tests pass with updated tuple API. + +--- + + + + +### Task 1: Update FrameOp enum with ALLOC_SHARED and FREE_LANE + +**Verifies:** None (enum additions only, no behaviour change) + +**Files:** +- Modify: `cm_inst.py:88-90` + +**Implementation:** + +Add two new enum members to `FrameOp`. Existing values stay unchanged: + +```python +class FrameOp(IntEnum): + ALLOC = 0 + FREE = 1 + ALLOC_SHARED = 2 + FREE_LANE = 3 +``` + +**Testing:** + +No tests needed — IntEnum membership is compiler-verifiable. Existing tests that use `FrameOp.ALLOC` and `FrameOp.FREE` remain unaffected. + +**Verification:** +Run: `python -m pytest tests/ -v -x` +Expected: All 1277+ existing tests pass unchanged. + +**Commit:** `jj commit -m "feat: add ALLOC_SHARED and FREE_LANE to FrameOp enum"` + + + +### Task 2: Add lane_count to PEConfig + +**Verifies:** frame-lanes.AC1.3 + +**Files:** +- Modify: `emu/types.py:17-27` + +**Implementation:** + +Add `lane_count` field to `PEConfig` with default 4. Place it after `matchable_offsets` (line 22) to group dimensional config together: + +```python +@dataclass(frozen=True) +class PEConfig: + pe_id: int = 0 + iram: dict[int, Instruction] | None = None + frame_count: int = 8 + frame_slots: int = 64 + matchable_offsets: int = 8 + lane_count: int = 4 + initial_frames: Optional[dict[int, list[FrameSlotValue]]] = None + initial_tag_store: Optional[dict[int, int]] = None + allowed_pe_routes: Optional[set[int]] = None + allowed_sm_routes: Optional[set[int]] = None + on_event: EventCallback | None = None +``` + +Note: `initial_tag_store` type stays `dict[int, int]` for now — the next task (Task 3) changes it to tuples. + +**Testing:** + +No dedicated tests — `lane_count` has a default value so all existing PEConfig constructions remain valid. The field's effect is tested when match arrays gain the lane dimension (Phase 2). + +**Verification:** +Run: `python -m pytest tests/ -v -x` +Expected: All existing tests pass unchanged. + +**Commit:** `jj commit -m "feat: add lane_count field to PEConfig with default 4"` + + + + + + + +### Task 3: Update PEConfig.initial_tag_store to tuple type + +**Verifies:** frame-lanes.AC1.2 + +**Files:** +- Modify: `emu/types.py:24` — change type annotation + +**Implementation:** + +Change the `initial_tag_store` type from `dict[int, int]` to `dict[int, tuple[int, int]]`: + +```python +initial_tag_store: Optional[dict[int, tuple[int, int]]] = None +``` + +Each entry is now `act_id → (frame_id, lane)`. + +**Testing:** + +No dedicated tests — this is a type change. Downstream call sites are updated in Tasks 4 and 5. + +**Verification:** + +This change alone will break tests that construct `PEConfig` with `initial_tag_store={0: 0}` etc. Do NOT run tests yet — proceed to Task 4 immediately. + +**Commit:** Do not commit yet — combine with Task 4. + + + +### Task 4: Update PE constructor and internals for tuple tag_store + +**Verifies:** frame-lanes.AC1.1 + +**Files:** +- Modify: `emu/pe.py:72` — tag_store initialization +- Modify: `emu/pe.py:88-90` — free_frames removal from tag_store values +- Modify: `emu/pe.py:169-176` — CMToken act_id lookup +- Modify: `emu/pe.py:260-261` — FREE_FRAME opcode handler +- Modify: `emu/pe.py:289` — ALLOC frame control handler +- Modify: `emu/pe.py:304-305` — FREE frame control handler +- Modify: `emu/pe.py:321-322` — PELocalWriteToken handler + +**Implementation:** + +The internal `tag_store` type changes from `dict[int, int]` to `dict[int, tuple[int, int]]`. Every access point must be updated. + +**Line 72 — Initialization:** +```python +# Tag store: act_id → (frame_id, lane) +self.tag_store: dict[int, tuple[int, int]] = dict(config.initial_tag_store or {}) +``` + +**Lines 88-90 — Free frames removal:** +The values are now tuples `(frame_id, lane)`. Extract `frame_id`: +```python +for frame_id, _lane in self.tag_store.values(): + if frame_id in self.free_frames: + self.free_frames.remove(frame_id) +``` + +**Lines 169-176 — CMToken pipeline (act_id lookup):** +Where the code currently does `frame_id = self.tag_store[token.act_id]`, change to: +```python +frame_id, lane = self.tag_store[token.act_id] +``` +The `lane` value is not used yet in Phase 1 — matching still uses the 2D presence/port arrays. Phase 2 adds the lane dimension to match storage. + +**Lines 260-261 — FREE_FRAME opcode:** +Where the code does `freed_frame = self.tag_store.pop(token.act_id)`, change to: +```python +freed_frame, _lane = self.tag_store.pop(token.act_id) +``` + +**Line 289 — ALLOC handler:** +Where the code stores `self.tag_store[token.act_id] = frame_id`, change to: +```python +self.tag_store[token.act_id] = (frame_id, 0) +``` +New allocations always get lane 0. + +**Lines 304-305 — FREE handler:** +Where the code does `frame_id = self.tag_store.pop(token.act_id)`, change to: +```python +frame_id, _lane = self.tag_store.pop(token.act_id) +``` + +**Lines 321-322 — PELocalWriteToken handler:** +Where the code checks `token.act_id in self.tag_store` and then does `frame_id = self.tag_store[token.act_id]`, change the lookup to: +```python +frame_id, _lane = self.tag_store[token.act_id] +``` + +**Testing:** + +No new tests in this task — AC1.1 is verified by the existing test suite passing with the new tuple type (Task 5 updates those tests). + +**Verification:** + +Do NOT run tests yet — existing tests still pass `dict[int, int]` values to `initial_tag_store`. Proceed to Task 5 immediately. + +**Commit:** Do not commit yet — combine with Task 5. + + + +### Task 5: Update all test files and downstream code for tuple tag_store API + +**Verifies:** frame-lanes.AC1.2, frame-lanes.AC1.4 + +**Files:** +- Modify: `tests/test_pe_frames.py` — ~21 `pe.tag_store[N]` value access sites need tuple unpacking (e.g., `frame_id = pe.tag_store[0]` → `frame_id, _lane = pe.tag_store[0]` or `frame_id = pe.tag_store[0][0]`). Also fix `pe.tag_store[0] in range(pe.frame_count)` at line 99 to `pe.tag_store[0][0] in range(pe.frame_count)`. Note: this file does NOT use `initial_tag_store` — changes are to value access patterns only. +- Modify: `tests/test_pe_events.py` — 9 `initial_tag_store` call sites: all `{0: 0}` → `{0: (0, 0)}` +- Modify: `tests/test_network_routing.py` — 2 tests with `initial_tag_store` construction and `pe.tag_store` value assertions +- Modify: `tests/test_snapshot.py` — tag_store capture assertions and PESnapshot type +- Modify: `tests/test_pe.py` — 18 `initial_tag_store` call sites: 17 `{0: 0}` → `{0: (0, 0)}`, one `{1: 0}` → `{1: (0, 0)}`. Note: `pe.presence` indexing changes are deferred to Phase 2 Task 2 (this task changes ONLY `initial_tag_store` values in this file). +- Modify: `tests/test_monitor_graph_json.py` — PESnapshot constructions with tag_store field +- Modify: `monitor/snapshot.py:25` — PESnapshot.tag_store type annotation +- Modify: `monitor/snapshot.py:81` — capture() tag_store copy +- Modify: `asm/codegen.py:371-422` — initial_tag_store generation +- Modify: `tests/conftest.py` — frame_control_token strategy (if it constructs tag_store) + +**Implementation:** + +This is a mechanical find-and-replace across the codebase. Every place that constructs `initial_tag_store` must change from `{act_id: frame_id}` to `{act_id: (frame_id, lane)}` where lane is 0 for all existing code. + +**Pattern for test files:** + +Every `initial_tag_store={0: 0}` becomes `initial_tag_store={0: (0, 0)}`. +Every `initial_tag_store={1: 0}` becomes `initial_tag_store={1: (0, 0)}`. +Every `initial_tag_store={0: 2, 1: 3}` becomes `initial_tag_store={0: (2, 0), 1: (3, 0)}`. + +**Pattern for assertions on tag_store values:** + +Where tests assert `pe.tag_store[0] == 2`, change to `pe.tag_store[0] == (2, 0)`. +Where tests assert `pe.tag_store[0]` (existence check), no change needed. + +**monitor/snapshot.py line 25:** +```python +tag_store: dict[int, tuple[int, int]] +``` + +**monitor/snapshot.py line 81:** +No code change needed — `dict(pe.tag_store)` already copies tuples correctly. + +**asm/codegen.py lines 371-422:** + +Where `initial_tag_store[act_id] = frame_id` is set, change to: +```python +initial_tag_store[act_id] = (frame_id, 0) +``` + +This applies at approximately lines 383 and 412. + +**Testing:** + +This task verifies AC1.4 — all existing tests must pass with the updated tuple API. No new test functions are needed; the existing suite IS the verification. + +**Verification:** +Run: `python -m pytest tests/ -v -x` +Expected: All existing tests pass. Zero failures. + +**Commit:** `jj commit -m "feat: update tag_store to tuple API (act_id → frame_id, lane)"` + +This single commit covers Tasks 3, 4, and 5 together since they form an atomic change — the type, the internals, and all call sites must change together. + + + diff --git a/docs/implementation-plans/2026-03-07-frame-lanes/phase_02.md b/docs/implementation-plans/2026-03-07-frame-lanes/phase_02.md new file mode 100644 index 0000000..015e088 --- /dev/null +++ b/docs/implementation-plans/2026-03-07-frame-lanes/phase_02.md @@ -0,0 +1,268 @@ +# Frame Matching Lanes Implementation Plan + +**Goal:** Extend the PE's frame-based matching to support multiple simultaneous pending operands per instruction within a single activation via matching lanes. + +**Architecture:** Multiple `activation_id` values share one physical frame (constants/destinations) while maintaining independent matching state per lane. Tag store maps `act_id → (frame_id, lane)`. Match data, presence, and port storage gain a lane dimension. + +**Tech Stack:** Python 3.12, SimPy 4.1, pytest + hypothesis + +**Scope:** 6 phases from original design (phases 1-6) + +**Codebase verified:** 2026-03-07 + +--- + +## Acceptance Criteria Coverage + +This phase implements and tests: + +### frame-lanes.AC2: Separate Match Data Storage +- **frame-lanes.AC2.1 Success:** Match operand data lives in `match_data[frame_id][offset][lane]`, separate from `frames[frame_id][slot]`. +- **frame-lanes.AC2.2 Success:** `presence[frame_id][offset][lane]` is a 3D bool array. `port_store[frame_id][offset][lane]` likewise. +- **frame-lanes.AC2.3 Success:** `_match_frame()` uses `(frame_id, match_slot, lane)` to read/write match data, presence, and port. +- **frame-lanes.AC2.4 Success:** `frames[frame_id][slot]` remains shared across all lanes. Constants and destinations are NOT per-lane. + +Also partially satisfies (structural type changes only, full testing in Phase 5): +### frame-lanes.AC6: Monitor and Snapshot Updates (partial) +- **frame-lanes.AC6.1:** `PESnapshot.tag_store` type updated to `dict[int, tuple[int, int]]`. +- **frame-lanes.AC6.2:** `PESnapshot` gains `match_data`, `lane_count` fields. + +--- + + + + +### Task 1: Add match_data 3D array and convert presence/port_store to 3D + +**Verifies:** frame-lanes.AC2.1, frame-lanes.AC2.2 + +**Files:** +- Modify: `emu/pe.py:66-84` — PE constructor storage initialization +- Modify: `emu/pe.py:290-296` — ALLOC handler reset logic + +**Implementation:** + +Add a new `match_data` 3D array and extend `presence` and `port_store` from 2D to 3D by adding the lane dimension. After Phase 1, `config.lane_count` is available. + +**Constructor changes (replace lines 74-84):** + +Replace the current 2D presence and port_store initialization with 3D versions, and add a match_data array: + +```python +# Match data: [frame_id][match_slot][lane] - operand values waiting for partner +self.match_data: list[list[list[Optional[int]]]] = [ + [ + [None for _ in range(config.lane_count)] + for _ in range(config.matchable_offsets) + ] + for _ in range(config.frame_count) +] + +# Presence bits: [frame_id][match_slot][lane] - True if operand waiting for partner +self.presence: list[list[list[bool]]] = [ + [ + [False for _ in range(config.lane_count)] + for _ in range(config.matchable_offsets) + ] + for _ in range(config.frame_count) +] + +# Port store: [frame_id][match_slot][lane] - port of waiting operand +self.port_store: list[list[list[Optional[Port]]]] = [ + [ + [None for _ in range(config.lane_count)] + for _ in range(config.matchable_offsets) + ] + for _ in range(config.frame_count) +] + +self.lane_count = config.lane_count +``` + +**ALLOC handler reset (lines 293-296):** + +Update the presence/port_store reset loop to iterate all lanes, and also clear match_data: + +```python +for i in range(self.matchable_offsets): + for ln in range(self.lane_count): + self.match_data[frame_id][i][ln] = None + self.presence[frame_id][i][ln] = False + self.port_store[frame_id][i][ln] = None +``` + +**Testing:** + +No dedicated tests for storage structure — AC2.1 and AC2.2 are verified by the existing test suite continuing to pass after Task 2 updates `_match_frame()`. The 3D structure is exercised through matching behaviour. + +**Verification:** + +Do NOT run tests yet — `_match_frame()` still uses 2D indexing. Proceed to Task 2 immediately. + +**Commit:** Do not commit yet — combine with Task 2. + + + +### Task 2: Update _match_frame() to use lane dimension + +**Verifies:** frame-lanes.AC2.3, frame-lanes.AC2.4 + +**Files:** +- Modify: `emu/pe.py:343-383` — `_match_frame()` method +- Modify: `emu/pe.py:169-176` — CMToken pipeline where `_match_frame()` is called + +**Implementation:** + +Update `_match_frame()` to accept and use the `lane` parameter. After Phase 1, the CMToken pipeline already unpacks `frame_id, lane = self.tag_store[token.act_id]` — now pass `lane` through. + +**Update the call site (in the CMToken processing pipeline):** + +Where `_match_frame()` is currently called with `(token, inst, frame_id)`, add `lane`: +```python +result = self._match_frame(token, inst, frame_id, lane) +``` + +**Updated `_match_frame()` signature and body:** + +```python +def _match_frame( + self, + token: DyadToken, + inst: Instruction, + frame_id: int, + lane: int, +) -> Optional[tuple[int, int]]: + """Frame-based dyadic matching with lane support. + + Derives match slot from low bits of token.offset: + match_slot = token.offset % matchable_offsets + + Match data, presence, and port are per-lane. + Frame constants/destinations remain shared. + """ + match_slot = token.offset % self.matchable_offsets + + if self.presence[frame_id][match_slot][lane]: + # Partner already waiting — pair them + partner_data = self.match_data[frame_id][match_slot][lane] + partner_port = self.port_store[frame_id][match_slot][lane] + self.presence[frame_id][match_slot][lane] = False + self.match_data[frame_id][match_slot][lane] = None + + # Use port metadata to determine left/right ordering + if partner_port == Port.L: + left, right = partner_data, token.data + else: + left, right = token.data, partner_data + + self._on_event(Matched( + time=self.env.now, component=self._component, + left=left, right=right, act_id=token.act_id, + offset=token.offset, frame_id=frame_id, + )) + return left, right + else: + # Store and wait for partner + self.match_data[frame_id][match_slot][lane] = token.data + self.port_store[frame_id][match_slot][lane] = token.port + self.presence[frame_id][match_slot][lane] = True + return None +``` + +Key changes from current code: +- All `self.frames[frame_id][match_slot]` reads/writes for match data → `self.match_data[frame_id][match_slot][lane]` +- All `self.presence[frame_id][match_slot]` → `self.presence[frame_id][match_slot][lane]` +- All `self.port_store[frame_id][match_slot]` → `self.port_store[frame_id][match_slot][lane]` +- `self.frames` is NOT touched — constants and destinations remain shared (AC2.4) + +**Testing:** + +AC2.3 and AC2.4 are verified by the existing test suite passing. All existing tests use lane 0 (set by Phase 1's tuple tag_store), so matching behaviour is identical. + +Two tests directly check `pe.presence`: +- `tests/test_pe.py:160` — `assert pe.presence[frame_id][0] is True` → change to `pe.presence[frame_id][0][0]` +- `tests/test_pe.py:624` — `assert pe.presence[frame_id][...] is False` → change to `pe.presence[frame_id][...][0]` + +**Verification:** +Run: `python -m pytest tests/ -v -x` +Expected: All existing tests pass. + +**Commit:** `jj commit -m "feat: separate match_data from frames, add lane dimension to presence/port_store"` + +This single commit covers Tasks 1 and 2 since they form an atomic change. + + + +### Task 3: Update snapshot capture for 3D match storage + +**Verifies:** None (snapshot updates for AC6 are in Phase 5, but this keeps snapshot working) + +**Files:** +- Modify: `monitor/snapshot.py:18-30` — PESnapshot dataclass +- Modify: `monitor/snapshot.py:82-89` — capture() presence/port_store conversion +- Modify: `tests/test_snapshot.py` — snapshot assertion updates + +**Implementation:** + +Update PESnapshot to reflect the new 3D storage shapes and add match_data field. + +**PESnapshot dataclass updates:** + +```python +@dataclass(frozen=True) +class PESnapshot: + pe_id: int + iram: dict[int, Instruction] + frames: tuple[tuple[FrameSlotValue, ...], ...] + tag_store: dict[int, tuple[int, int]] + presence: tuple[tuple[tuple[bool, ...], ...], ...] + port_store: tuple[tuple[tuple[Port | None, ...], ...], ...] + match_data: tuple[tuple[tuple[int | None, ...], ...], ...] + free_frames: tuple[int, ...] + lane_count: int + input_queue: tuple[Token, ...] + output_log: tuple[Token, ...] +``` + +**capture() updates:** + +Replace the 2D presence/port_store capture with 3D, and add match_data capture: + +```python +presence = tuple( + tuple( + tuple(lane_val for lane_val in offset_lanes) + for offset_lanes in frame_presence + ) + for frame_presence in pe.presence +) +port_store = tuple( + tuple( + tuple(lane_val for lane_val in offset_lanes) + for offset_lanes in frame_ports + ) + for frame_ports in pe.port_store +) +match_data = tuple( + tuple( + tuple(lane_val for lane_val in offset_lanes) + for offset_lanes in frame_match + ) + for frame_match in pe.match_data +) +``` + +Pass `match_data=match_data` and `lane_count=pe.lane_count` to the PESnapshot constructor. + +**Testing:** + +Update any snapshot tests that assert on `presence` or `port_store` shape to expect 3D tuples. Update tests that construct PESnapshot directly to include `match_data` and `lane_count` fields. + +**Verification:** +Run: `python -m pytest tests/ -v -x` +Expected: All tests pass. + +**Commit:** `jj commit -m "feat: update PESnapshot for 3D match storage and match_data field"` + + + diff --git a/docs/implementation-plans/2026-03-07-frame-lanes/phase_03.md b/docs/implementation-plans/2026-03-07-frame-lanes/phase_03.md new file mode 100644 index 0000000..bf43ead --- /dev/null +++ b/docs/implementation-plans/2026-03-07-frame-lanes/phase_03.md @@ -0,0 +1,363 @@ +# Frame Matching Lanes Implementation Plan + +**Goal:** Extend the PE's frame-based matching to support multiple simultaneous pending operands per instruction within a single activation via matching lanes. + +**Architecture:** Multiple `activation_id` values share one physical frame (constants/destinations) while maintaining independent matching state per lane. Tag store maps `act_id → (frame_id, lane)`. Match data, presence, and port storage gain a lane dimension. + +**Tech Stack:** Python 3.12, SimPy 4.1, pytest + hypothesis + +**Scope:** 6 phases from original design (phases 1-6) + +**Codebase verified:** 2026-03-07 + +--- + +## Acceptance Criteria Coverage + +This phase implements and tests: + +### frame-lanes.AC3: FrameOp Extensions +- **frame-lanes.AC3.1 Success:** `FrameOp.ALLOC_SHARED` added. When received, PE looks up `parent_act_id` (from payload), finds parent's `frame_id`, assigns next free lane from that frame's lane pool, records `tag_store[act_id] = (frame_id, lane)`. Clears only that lane's presence/port bits. +- **frame-lanes.AC3.2 Success:** `FrameOp.FREE_LANE` added. Removes tag_store entry, clears that lane's presence/port/match_data across all matchable offsets. Does NOT return frame to free list. +- **frame-lanes.AC3.3 Success:** `FrameOp.FREE` (existing) becomes smart: removes tag_store entry, clears lane data. If no other tag_store entries reference the same frame_id, returns frame to free list and clears frame slots. If other entries exist, behaves like FREE_LANE. +- **frame-lanes.AC3.4 Success:** `FrameOp.ALLOC` (existing) unchanged — allocates fresh frame, assigns lane 0. +- **frame-lanes.AC3.5 Success:** `FrameAllocated` event gains `lane: int` field. `FrameFreed` event gains `lane: int` and `frame_freed: bool` fields. +- **frame-lanes.AC3.6 Success:** When all lanes for a frame are occupied and ALLOC_SHARED is received, PE emits `TokenRejected` with reason "no free lanes" and drops the token. + +### frame-lanes.AC8: Test Coverage (partial) +- **frame-lanes.AC8.1 Success:** Test: two act_ids sharing a frame via ALLOC_SHARED have independent matching — L operand for act_id 0 does not interfere with L operand for act_id 1 at the same offset. +- **frame-lanes.AC8.2 Success:** Test: ALLOC_SHARED with all lanes occupied emits TokenRejected. +- **frame-lanes.AC8.3 Success:** Test: FREE on a shared frame frees only the lane; other lanes' data is preserved. FREE on last lane frees the frame. + +--- + + + + +### Task 1: Add lane_free tracking and update FrameAllocated/FrameFreed events + +**Verifies:** frame-lanes.AC3.5 + +**Files:** +- Modify: `emu/events.py:84-97` — add lane fields to FrameAllocated and FrameFreed +- Modify: `emu/pe.py` — add lane_free data structure, update all event emissions +- Modify: `tests/test_pe_events.py` — update assertions for new event fields +- Modify: `tests/test_pe_frames.py` — update assertions for new event fields + +**Implementation:** + +**emu/events.py — Update event dataclasses:** + +```python +@dataclass(frozen=True) +class FrameAllocated: + time: float + component: str + act_id: int + frame_id: int + lane: int + +@dataclass(frozen=True) +class FrameFreed: + time: float + component: str + act_id: int + frame_id: int + lane: int + frame_freed: bool +``` + +**emu/pe.py — Add lane_free tracking in constructor:** + +After the `free_frames` initialization (line 87), add: + +```python +# Lane tracking: which lanes are free per frame +self.lane_free: dict[int, set[int]] = {} +``` + +`lane_free` is populated lazily — when a frame is allocated via ALLOC, its lanes are set up. + +**emu/pe.py — Update existing ALLOC handler event emission:** + +After Phase 1, ALLOC stores `(frame_id, 0)` in tag_store. Update to also set up lane tracking and emit `lane=0`: + +```python +if token.op == FrameOp.ALLOC: + if self.free_frames: + frame_id = self.free_frames.pop() + self.tag_store[token.act_id] = (frame_id, 0) + # Set up lane tracking: lane 0 is taken, rest are free + self.lane_free[frame_id] = set(range(1, self.lane_count)) + # Initialize frame slots to None + for i in range(self.frame_slots): + self.frames[frame_id][i] = None + # Reset all lanes' match state + for i in range(self.matchable_offsets): + for ln in range(self.lane_count): + self.match_data[frame_id][i][ln] = None + self.presence[frame_id][i][ln] = False + self.port_store[frame_id][i][ln] = None + self._on_event(FrameAllocated( + time=self.env.now, component=self._component, + act_id=token.act_id, frame_id=frame_id, lane=0, + )) + else: + logger.warning(f"PE {self.pe_id}: no free frames available") +``` + +**emu/pe.py — Update existing FREE handler for smart behaviour:** + +Note: This replaces the Phase 1 FREE handler wholesale. The Phase 1 version only added tuple unpacking (`frame_id, _lane = self.tag_store.pop(...)`). This version adds lane data clearing, frame-in-use checking, and conditional frame return. + +```python +elif token.op == FrameOp.FREE: + if token.act_id in self.tag_store: + frame_id, lane = self.tag_store.pop(token.act_id) + # Clear this lane's match state + for i in range(self.matchable_offsets): + self.match_data[frame_id][i][lane] = None + self.presence[frame_id][i][lane] = False + self.port_store[frame_id][i][lane] = None + # Check if any other activations use this frame + frame_in_use = any(fid == frame_id for fid, _ in self.tag_store.values()) + if frame_in_use: + # Return lane to pool, keep frame + self.lane_free[frame_id].add(lane) + self._on_event(FrameFreed( + time=self.env.now, component=self._component, + act_id=token.act_id, frame_id=frame_id, + lane=lane, frame_freed=False, + )) + else: + # Last lane — return frame to free list + self.free_frames.append(frame_id) + if frame_id in self.lane_free: + del self.lane_free[frame_id] + # Clear frame slots + for i in range(self.frame_slots): + self.frames[frame_id][i] = None + self._on_event(FrameFreed( + time=self.env.now, component=self._component, + act_id=token.act_id, frame_id=frame_id, + lane=lane, frame_freed=True, + )) +``` + +**emu/pe.py — Update FREE_FRAME opcode handler (lines 259-266):** + +Same smart free logic applies here: + +```python +if token.act_id in self.tag_store: + freed_frame, lane = self.tag_store.pop(token.act_id) + # Clear this lane's match state + for i in range(self.matchable_offsets): + self.match_data[freed_frame][i][lane] = None + self.presence[freed_frame][i][lane] = False + self.port_store[freed_frame][i][lane] = None + # Check if any other activations use this frame + frame_in_use = any(fid == freed_frame for fid, _ in self.tag_store.values()) + if frame_in_use: + self.lane_free[freed_frame].add(lane) + self._on_event(FrameFreed( + time=self.env.now, component=self._component, + act_id=token.act_id, frame_id=freed_frame, + lane=lane, frame_freed=False, + )) + else: + self.free_frames.append(freed_frame) + if freed_frame in self.lane_free: + del self.lane_free[freed_frame] + for i in range(self.frame_slots): + self.frames[freed_frame][i] = None + self._on_event(FrameFreed( + time=self.env.now, component=self._component, + act_id=token.act_id, frame_id=freed_frame, + lane=lane, frame_freed=True, + )) +``` + +**Also update constructor initialisation for pre-loaded tag_store entries:** + +After Phase 1, the constructor removes allocated frames from `free_frames`. Also initialise `lane_free` for those frames: + +```python +for act_id, (frame_id, lane) in self.tag_store.items(): + if frame_id in self.free_frames: + self.free_frames.remove(frame_id) + if frame_id not in self.lane_free: + # First time seeing this frame — set up lane tracking + all_lanes = set(range(self.lane_count)) + self.lane_free[frame_id] = all_lanes - {lane} + else: + self.lane_free[frame_id].discard(lane) +``` + +**Test updates:** + +All existing tests that assert on FrameAllocated or FrameFreed events need updated assertions to include the new fields. For existing single-activation tests: +- `FrameAllocated` assertions add `lane=0` +- `FrameFreed` assertions add `lane=0, frame_freed=True` + +Specific test files affected: +- `tests/test_pe_frames.py` lines 505-507 (test_alloc_remote), 550-552 (test_free_frame_opcode), 122-125 (test_free_frame_control_token) +- `tests/test_pe_events.py` — any tests asserting on FrameAllocated/FrameFreed event fields + +**Verification:** +Run: `python -m pytest tests/ -v -x` +Expected: All existing tests pass with updated event assertions. + +**Commit:** `jj commit -m "feat: add lane tracking and update FrameAllocated/FrameFreed events with lane fields"` + + + +### Task 2: Implement ALLOC_SHARED and FREE_LANE handlers + +**Verifies:** frame-lanes.AC3.1, frame-lanes.AC3.2, frame-lanes.AC3.3, frame-lanes.AC3.4, frame-lanes.AC3.6 + +**Files:** +- Modify: `emu/pe.py` — add ALLOC_SHARED and FREE_LANE cases to `_handle_frame_control()` + +**Implementation:** + +Add two new cases to `_handle_frame_control()` after the existing ALLOC and FREE handlers: + +```python +elif token.op == FrameOp.ALLOC_SHARED: + # Shared allocation: find parent's frame, assign next free lane + parent_act_id = token.payload + if parent_act_id not in self.tag_store: + self._on_event(TokenRejected( + time=self.env.now, component=self._component, + token=token, reason=f"parent act_id {parent_act_id} not in tag store", + )) + return + parent_frame_id, _ = self.tag_store[parent_act_id] + free_lanes = self.lane_free.get(parent_frame_id, set()) + if not free_lanes: + self._on_event(TokenRejected( + time=self.env.now, component=self._component, + token=token, reason="no free lanes", + )) + return + lane = min(free_lanes) # Deterministic: pick lowest free lane + free_lanes.remove(lane) + self.tag_store[token.act_id] = (parent_frame_id, lane) + # Clear only this lane's match state + for i in range(self.matchable_offsets): + self.match_data[parent_frame_id][i][lane] = None + self.presence[parent_frame_id][i][lane] = False + self.port_store[parent_frame_id][i][lane] = None + self._on_event(FrameAllocated( + time=self.env.now, component=self._component, + act_id=token.act_id, frame_id=parent_frame_id, lane=lane, + )) + +elif token.op == FrameOp.FREE_LANE: + # Free lane only — never returns frame to free list + if token.act_id in self.tag_store: + frame_id, lane = self.tag_store.pop(token.act_id) + for i in range(self.matchable_offsets): + self.match_data[frame_id][i][lane] = None + self.presence[frame_id][i][lane] = False + self.port_store[frame_id][i][lane] = None + self.lane_free[frame_id].add(lane) + self._on_event(FrameFreed( + time=self.env.now, component=self._component, + act_id=token.act_id, frame_id=frame_id, + lane=lane, frame_freed=False, + )) +``` + +**Testing:** + +No new tests in this task — AC3.1-AC3.6 are tested in Task 3. + +**Verification:** +Run: `python -m pytest tests/ -v -x` +Expected: All existing tests pass (new handlers only activate for new FrameOp values). + +**Commit:** `jj commit -m "feat: implement ALLOC_SHARED and FREE_LANE frame control handlers"` + + + + + + + +### Task 3: Tests for ALLOC_SHARED, lane exhaustion, and FREE_LANE + +**Verifies:** frame-lanes.AC3.1, frame-lanes.AC3.2, frame-lanes.AC3.6, frame-lanes.AC8.1, frame-lanes.AC8.2 + +**Files:** +- Create: `tests/test_pe_lanes.py` + +**Implementation:** + +Create a new test file dedicated to lane functionality. Follow the existing test patterns from `tests/test_pe_frames.py`: +- `simpy.Environment()` setup +- `PEConfig` with `on_event=events.append` +- `ProcessingElement(env, pe_id, config)` construction +- Token injection via `pe.input_store.put(token)` in a SimPy process +- Event collection via the events list + +**Testing:** + +Tests must verify these specific AC cases: + +- **frame-lanes.AC3.1 (ALLOC_SHARED):** Send `FrameControlToken(op=FrameOp.ALLOC, act_id=0, payload=0)` to allocate a frame. Then send `FrameControlToken(op=FrameOp.ALLOC_SHARED, act_id=1, payload=0)` where payload is the parent act_id. Verify `tag_store[1]` has the same `frame_id` as `tag_store[0]` but a different lane. Verify `FrameAllocated` event has correct lane. + +- **frame-lanes.AC3.2 (FREE_LANE):** After ALLOC_SHARED, send `FrameControlToken(op=FrameOp.FREE_LANE, act_id=1, payload=0)`. Verify `tag_store` no longer has act_id 1. Verify act_id 0 is still present. Verify frame is NOT in `free_frames`. Verify `FrameFreed` event has `frame_freed=False`. + +- **frame-lanes.AC3.6 (lane exhaustion):** Allocate a frame, then ALLOC_SHARED until all `lane_count` lanes are occupied. Send one more ALLOC_SHARED. Verify `TokenRejected` event with reason "no free lanes". + +- **frame-lanes.AC8.1 (independent matching):** Set up two act_ids sharing a frame (ALLOC + ALLOC_SHARED). Load an instruction with IRAM. Send L operand via DyadToken for act_id 0 and L operand via DyadToken for act_id 1 at the same offset. Verify both presence bits are set independently — neither token triggers a match (both are waiting for their R partner). Then send R for act_id 0 — verify only act_id 0 matches and fires, act_id 1's L is still pending. + +- **frame-lanes.AC8.2 (exhaustion):** Same as AC3.6 but via the test coverage AC numbering — allocate all lanes, attempt one more, verify TokenRejected. + +**Verification:** +Run: `python -m pytest tests/test_pe_lanes.py -v` +Expected: All new tests pass. + +Run: `python -m pytest tests/ -v -x` +Expected: All tests pass (new and existing). + +**Commit:** `jj commit -m "test: add tests for ALLOC_SHARED, FREE_LANE, and lane exhaustion"` + + + +### Task 4: Tests for smart FREE behaviour + +**Verifies:** frame-lanes.AC3.3, frame-lanes.AC3.4, frame-lanes.AC8.3 + +**Files:** +- Modify: `tests/test_pe_lanes.py` — add smart FREE test class + +**Implementation:** + +Add tests to the lane test file created in Task 3. + +**Testing:** + +Tests must verify these specific AC cases: + +- **frame-lanes.AC3.3 (smart FREE on shared frame):** Allocate a frame (act_id=0, lane 0). ALLOC_SHARED (act_id=1, lane 1). Send matching operands to act_id=1 so presence bits are set. FREE act_id=0. Verify: act_id=0 removed from tag_store, act_id=1 still present, frame NOT in free_frames, act_id=1's pending match data is preserved (presence bit still True for lane 1). Verify `FrameFreed` event has `frame_freed=False`. + +- **frame-lanes.AC3.3 (smart FREE on last lane):** Same setup. FREE act_id=0, then FREE act_id=1. After second FREE: frame IS returned to free_frames, `lane_free` entry for that frame is cleaned up. Verify `FrameFreed` event has `frame_freed=True`. + +- **frame-lanes.AC3.4 (ALLOC unchanged):** Verify that regular ALLOC still works — allocates fresh frame, assigns lane 0, no parent required. This is a regression check. + +- **frame-lanes.AC8.3 (data preservation):** Set up shared frame with two act_ids. Store a DyadToken L operand on act_id=1's lane. FREE act_id=0. Verify act_id=1's match_data and presence are untouched — the pending operand is still there. + +**Verification:** +Run: `python -m pytest tests/test_pe_lanes.py -v` +Expected: All tests pass. + +Run: `python -m pytest tests/ -v -x` +Expected: All tests pass. + +**Commit:** `jj commit -m "test: add tests for smart FREE behaviour and data preservation across lanes"` + + + diff --git a/docs/implementation-plans/2026-03-07-frame-lanes/phase_04.md b/docs/implementation-plans/2026-03-07-frame-lanes/phase_04.md new file mode 100644 index 0000000..1f2d02f --- /dev/null +++ b/docs/implementation-plans/2026-03-07-frame-lanes/phase_04.md @@ -0,0 +1,191 @@ +# Frame Matching Lanes Implementation Plan + +**Goal:** Extend the PE's frame-based matching to support multiple simultaneous pending operands per instruction within a single activation via matching lanes. + +**Architecture:** Multiple `activation_id` values share one physical frame (constants/destinations) while maintaining independent matching state per lane. Tag store maps `act_id → (frame_id, lane)`. Match data, presence, and port storage gain a lane dimension. + +**Tech Stack:** Python 3.12, SimPy 4.1, pytest + hypothesis + +**Scope:** 6 phases from original design (phases 1-6) + +**Codebase verified:** 2026-03-07 + +--- + +## Acceptance Criteria Coverage + +This phase implements and tests: + +### frame-lanes.AC4: ALLOC_REMOTE Data-Driven +- **frame-lanes.AC4.1 Success:** ALLOC_REMOTE reads `fref+2` from frame. If value is non-zero, emits `FrameControlToken` with `op=ALLOC_SHARED` and `payload=parent_act_id`. If zero, emits `op=ALLOC` as before. +- **frame-lanes.AC4.2 Success:** No new opcodes. Behaviour is entirely data-driven from frame constants. + +### frame-lanes.AC5: FREE_FRAME Instruction +- **frame-lanes.AC5.1 Success:** `FREE_FRAME` opcode uses the smart FREE behaviour from AC3.3. Frees the executing token's activation lane; returns frame to free list only if last lane. + +### frame-lanes.AC8: Test Coverage (partial) +- **frame-lanes.AC8.4 Success:** Test: ALLOC_REMOTE emits ALLOC_SHARED when `fref+2` is non-zero. +- **frame-lanes.AC8.5 Success:** Test: ALLOC_REMOTE emits ALLOC when `fref+2` is zero (backwards compatible). + +--- + + + + +### Task 1: Update ALLOC_REMOTE to read fref+2 for data-driven shared allocation + +**Verifies:** frame-lanes.AC4.1, frame-lanes.AC4.2 + +**Files:** +- Modify: `emu/pe.py:231-248` — ALLOC_REMOTE handler in `_process_token()` + +**Implementation:** + +The current ALLOC_REMOTE handler reads `fref+0` (target PE) and `fref+1` (target act_id) from frame constants, then emits a `FrameControlToken(op=FrameOp.ALLOC)`. Extend it to also read `fref+2` (parent act_id for shared allocation). + +**Updated handler:** + +```python +elif inst.opcode == RoutingOp.ALLOC_REMOTE: + # PE-level: read target PE, act_id, and optional parent act_id from frame constants + # fref+0: target PE + # fref+1: target act_id + # fref+2: parent act_id (0 = fresh ALLOC, non-zero = ALLOC_SHARED) + target_pe = self.frames[frame_id][inst.fref] if inst.fref < len(self.frames[frame_id]) else 0 + target_act = self.frames[frame_id][inst.fref + 1] if inst.fref + 1 < len(self.frames[frame_id]) else 0 + parent_act = self.frames[frame_id][inst.fref + 2] if inst.fref + 2 < len(self.frames[frame_id]) else 0 + + if parent_act: + alloc_op = FrameOp.ALLOC_SHARED + payload = parent_act + else: + alloc_op = FrameOp.ALLOC + payload = 0 + + fct = FrameControlToken( + target=target_pe, + act_id=target_act, + op=alloc_op, + payload=payload, + ) + self._on_event(Executed( + time=self.env.now, component=self._component, + op=inst.opcode, result=0, bool_out=False, + )) + yield self.env.timeout(1) # EXECUTE cycle + yield self.env.timeout(1) # EMIT cycle + self.env.process(self._deliver(self.route_table[target_pe], fct)) +``` + +Key changes from current code: +- Added `parent_act` read from `fref+2` +- Conditional: if `parent_act` is non-zero, use `ALLOC_SHARED` with `payload=parent_act`; otherwise, use `ALLOC` with `payload=0` (backwards compatible) +- No new opcodes (AC4.2) +- Note: Frame slots at `fref+0`, `fref+1`, `fref+2` must be `int` values (not `FrameDest`). The codegen guarantees this for properly assembled programs. No runtime type check added, consistent with the existing ALLOC_REMOTE pattern at `fref+0` and `fref+1`. + +**Testing:** + +No new tests in this task — AC4.1 and AC4.2 are tested in Task 3. + +**Verification:** +Run: `python -m pytest tests/ -v -x` +Expected: All existing tests pass. Existing ALLOC_REMOTE tests set up frames with only `fref+0` and `fref+1` populated — `fref+2` defaults to `None` which is falsy, so existing tests get `op=ALLOC` as before. + +**Commit:** `jj commit -m "feat: ALLOC_REMOTE reads fref+2 for data-driven ALLOC_SHARED"` + + + +### Task 2: Verify FREE_FRAME uses smart FREE behaviour + +**Verifies:** frame-lanes.AC5.1 + +**Files:** +- Verify: `emu/pe.py:249-267` — FREE_FRAME handler + +**Implementation:** + +After Phase 3, the FREE_FRAME opcode handler in `_process_token()` already uses the smart FREE behaviour (tag_store.pop with tuple unpacking, lane data clearing, frame-in-use check). This task verifies that the Phase 3 changes correctly cover the FREE_FRAME path. + +If Phase 3 was implemented correctly, the FREE_FRAME handler at lines 260-266 should already: +1. Unpack `frame_id, lane = self.tag_store.pop(token.act_id)` +2. Clear the lane's match_data/presence/port_store +3. Check if other activations reference the same frame_id +4. Return frame to free_frames only if last lane +5. Emit FrameFreed with `lane` and `frame_freed` fields + +If the Phase 3 implementation only updated `_handle_frame_control()` FREE and forgot the FREE_FRAME opcode path, this task is where you fix it. Both paths must have identical smart FREE logic. + +**Testing:** + +No new dedicated tests — AC5.1 is a subset of AC3.3 applied to a different code path. The existing `test_free_frame_opcode` test in `tests/test_pe_frames.py` verifies the basic path; the lane-aware behaviour is tested via the AC8.3 tests in Phase 3. + +**Verification:** +Run: `python -m pytest tests/test_pe_frames.py -v -k "free_frame"` +Expected: All FREE_FRAME tests pass. + +**Commit:** No commit needed if Phase 3 already handled this path. If a fix is needed: `jj commit -m "fix: ensure FREE_FRAME opcode uses smart FREE behaviour"` + + + + + + + +### Task 3: Tests for data-driven ALLOC_REMOTE + +**Verifies:** frame-lanes.AC8.4, frame-lanes.AC8.5 + +**Files:** +- Modify: `tests/test_pe_lanes.py` — add ALLOC_REMOTE data-driven tests + +**Implementation:** + +Add tests to the lane test file. + +**Testing:** + +Tests must verify these specific AC cases: + +- **frame-lanes.AC8.4 (ALLOC_REMOTE emits ALLOC_SHARED):** Set up PE0 with an allocated frame. Write frame slots: `fref+0 = 1` (target PE 1), `fref+1 = 5` (target act_id), `fref+2 = 3` (parent act_id, non-zero). Load ALLOC_REMOTE instruction. Send a MonadToken to trigger it. Set up PE1 with a route table entry so we can capture the emitted token. Verify the FrameControlToken sent to PE1 has `op=FrameOp.ALLOC_SHARED` and `payload=3`. + + For capturing the emitted FrameControlToken: use `output_store = simpy.Store(env)` and set `pe.route_table[1] = output_store`, then check `output_store.items[0]`. + +- **frame-lanes.AC8.5 (ALLOC_REMOTE emits ALLOC when fref+2 is zero):** Same setup but `fref+2 = 0` (or slot is None). Verify the FrameControlToken has `op=FrameOp.ALLOC` and `payload=0`. This is the backwards-compatible path. + +**Verification:** +Run: `python -m pytest tests/test_pe_lanes.py -v -k "alloc_remote"` +Expected: All new tests pass. + +Run: `python -m pytest tests/ -v -x` +Expected: All tests pass. + +**Commit:** `jj commit -m "test: add tests for data-driven ALLOC_REMOTE (ALLOC_SHARED vs ALLOC)"` + + + +### Task 4: Test FREE_FRAME with shared frame (smart FREE via opcode) + +**Verifies:** frame-lanes.AC5.1 + +**Files:** +- Modify: `tests/test_pe_lanes.py` — add FREE_FRAME smart free test + +**Implementation:** + +Add a test that exercises the FREE_FRAME opcode path specifically (not the FrameControlToken FREE path, which is already tested in Phase 3). + +**Testing:** + +- **frame-lanes.AC5.1 (FREE_FRAME smart free):** Set up PE with a shared frame (two act_ids on the same frame via initial_tag_store with different lanes). Load FREE_FRAME instruction at the offset used by act_id=0. Send a MonadToken to act_id=0 to trigger FREE_FRAME execution. Verify: act_id=0's lane is freed, act_id=1 is still in tag_store with the same frame, frame is NOT in free_frames, FrameFreed event has `frame_freed=False`. Then trigger FREE_FRAME for act_id=1. Verify: frame IS returned to free_frames, FrameFreed event has `frame_freed=True`. + +**Verification:** +Run: `python -m pytest tests/test_pe_lanes.py -v -k "free_frame"` +Expected: All tests pass. + +Run: `python -m pytest tests/ -v -x` +Expected: All tests pass. + +**Commit:** `jj commit -m "test: add test for FREE_FRAME opcode with smart free on shared frame"` + + + diff --git a/docs/implementation-plans/2026-03-07-frame-lanes/phase_05.md b/docs/implementation-plans/2026-03-07-frame-lanes/phase_05.md new file mode 100644 index 0000000..c2a3cc2 --- /dev/null +++ b/docs/implementation-plans/2026-03-07-frame-lanes/phase_05.md @@ -0,0 +1,195 @@ +# Frame Matching Lanes Implementation Plan + +**Goal:** Extend the PE's frame-based matching to support multiple simultaneous pending operands per instruction within a single activation via matching lanes. + +**Architecture:** Multiple `activation_id` values share one physical frame (constants/destinations) while maintaining independent matching state per lane. Tag store maps `act_id → (frame_id, lane)`. Match data, presence, and port storage gain a lane dimension. + +**Tech Stack:** Python 3.12, SimPy 4.1, pytest + hypothesis + +**Scope:** 6 phases from original design (phases 1-6) + +**Codebase verified:** 2026-03-07 + +--- + +## Acceptance Criteria Coverage + +This phase implements and tests: + +### frame-lanes.AC6: Monitor and Snapshot Updates +- **frame-lanes.AC6.1 Success:** `PESnapshot.tag_store` type becomes `dict[int, tuple[int, int]]`. +- **frame-lanes.AC6.2 Success:** `PESnapshot` gains `match_data`, `lane_count` fields reflecting the separated match storage. +- **frame-lanes.AC6.3 Success:** Monitor REPL `pe` command displays lane info in tag_store output. +- **frame-lanes.AC6.4 Success:** Monitor graph JSON serialises lane info correctly. + +### frame-lanes.AC7: Codegen Updates +- **frame-lanes.AC7.1 Success:** `codegen.py` generates `initial_tag_store` with `(frame_id, lane)` tuples. Existing single-activation code uses lane 0. +- **frame-lanes.AC7.2 Success:** No codegen changes needed for ALLOC_SHARED (manual construction only for now). + +--- + +Note: AC6.1 and AC6.2 (PESnapshot type changes) are structurally completed in Phase 2 Task 3 as part of the match storage separation. This phase covers the remaining monitor/codegen pieces that depend on those type changes. + + + + +### Task 1: Update Monitor REPL formatting for lane info + +**Verifies:** frame-lanes.AC6.3 + +**Files:** +- Modify: `monitor/formatting.py:194-202` — `format_pe_state()` tag_store display + +**Implementation:** + +The current tag_store display at `monitor/formatting.py:194-202` formats entries as `{key: value}`. After Phase 1, tag_store values are `(frame_id, lane)` tuples. Update the formatting to show lane info clearly. + +**Current code (lines 194-202):** +```python +if pe_snapshot.tag_store: + tag_str = ", ".join( + f"{colour(str(k), 'white')}: {colour(str(v), 'white')}" + for k, v in sorted(pe_snapshot.tag_store.items()) + ) + lines.append(f" Tag store: {{{tag_str}}}") +else: + lines.append(" Tag store: (empty)") +``` + +**Updated code:** +```python +if pe_snapshot.tag_store: + tag_str = ", ".join( + f"{colour(str(k), 'white')}: frame {colour(str(fid), 'white')} lane {colour(str(lane), 'white')}" + for k, (fid, lane) in sorted(pe_snapshot.tag_store.items()) + ) + lines.append(f" Tag store: {{{tag_str}}}") +else: + lines.append(" Tag store: (empty)") +``` + +This changes the display from `{0: 0}` to `{0: frame 0 lane 0}`, making lane assignments visible at a glance. + +**Testing:** + +The REPL tests at `tests/test_repl.py:456-473` verify that `do_pe()` produces output but don't assert on specific tag_store formatting content. The formatting change is verified by manual inspection. Existing tests remain valid because they only check `len(out) > 0`. + +**Verification:** +Run: `python -m pytest tests/test_repl.py -v` +Expected: All tests pass. + +**Commit:** `jj commit -m "feat: display lane info in monitor REPL pe command"` + + + +### Task 2: Update Monitor graph JSON for lane serialisation + +**Verifies:** frame-lanes.AC6.4 + +**Files:** +- Modify: `monitor/graph_json.py:104-127` — `_serialise_pe_state()` function +- Modify: `tests/test_monitor_graph_json.py` — update PESnapshot constructions + +**Implementation:** + +The current `_serialise_pe_state()` at `monitor/graph_json.py:124` passes `tag_store` directly to JSON: +```python +"tag_store": pe_snap.tag_store, +``` + +After Phase 1, tag_store values are `(frame_id, lane)` tuples. Use explicit dict format for self-documenting JSON that the TypeScript frontend can easily type: + +```python +"tag_store": { + str(act_id): {"frame_id": fid, "lane": lane} + for act_id, (fid, lane) in pe_snap.tag_store.items() +}, +"lane_count": pe_snap.lane_count, +``` + +This produces JSON like `{"0": {"frame_id": 2, "lane": 0}}` instead of `{"0": [2, 0]}`, which is more explicit and easier to type in TypeScript. + +**Test updates:** + +Update `tests/test_monitor_graph_json.py` PESnapshot constructions to include the new fields (`match_data`, `lane_count`). Tests that construct `PESnapshot` directly with `tag_store={}` will work as-is (empty dict). Tests that use non-empty tag_store must change values from `int` to `tuple[int, int]`. + +**Verification:** +Run: `python -m pytest tests/test_monitor_graph_json.py -v` +Expected: All tests pass. + +**Commit:** `jj commit -m "feat: serialise lane info in monitor graph JSON"` + + + + + + + +### Task 3: Update codegen to emit tuple initial_tag_store + +**Verifies:** frame-lanes.AC7.1 + +**Files:** +- Modify: `asm/codegen.py:383` — first initial_tag_store assignment +- Modify: `asm/codegen.py:412` — second initial_tag_store assignment + +**Implementation:** + +The codegen at `asm/codegen.py` builds `initial_tag_store` as `dict[int, int]` mapping `act_id → frame_id`. Change both assignment sites to produce `dict[int, tuple[int, int]]` mapping `act_id → (frame_id, lane)` with lane 0 for all existing single-activation code. + +**Line 383 (empty layout path):** +```python +initial_tag_store[act_id] = (frame_id, 0) +``` + +**Line 412 (populated layout path):** +```python +initial_tag_store[act_id] = (frame_id, 0) +``` + +No other changes needed. The type annotation for the local variable can be updated: +```python +initial_tag_store: dict[int, tuple[int, int]] = {} +``` + +**Testing:** + +Existing codegen tests at `tests/test_codegen_frames.py` don't explicitly assert on `initial_tag_store` contents — they test IRAM, setup_tokens, and seed_tokens. The type change is validated by the overall test suite passing (PEConfig now expects tuple values from Phase 1). + +**Verification:** +Run: `python -m pytest tests/test_codegen_frames.py -v` +Expected: All tests pass. + +Run: `python -m pytest tests/ -v -x` +Expected: All tests pass. + +**Commit:** `jj commit -m "feat: codegen emits initial_tag_store with (frame_id, lane) tuples"` + + + +### Task 4: Verify no codegen changes needed for ALLOC_SHARED + +**Verifies:** frame-lanes.AC7.2 + +**Files:** +- No files to modify + +**Implementation:** + +AC7.2 states: "No codegen changes needed for ALLOC_SHARED (manual construction only for now)." This is a verification task — confirm that the codegen does not attempt to generate ALLOC_SHARED tokens or any lane-related control flow. ALLOC_SHARED is invoked only via manual test construction or hand-crafted assembly. + +**Verification:** + +Grep codegen for any reference to ALLOC_SHARED or FREE_LANE: +```bash +grep -r "ALLOC_SHARED\|FREE_LANE" asm/ +``` +Expected: No results. The codegen is unaware of these new FrameOp values. + +Run: `python -m pytest tests/ -v -x` +Expected: All tests pass. + +**Commit:** No commit needed — this is a verification-only task. + + + diff --git a/docs/implementation-plans/2026-03-07-frame-lanes/phase_06.md b/docs/implementation-plans/2026-03-07-frame-lanes/phase_06.md new file mode 100644 index 0000000..77dec37 --- /dev/null +++ b/docs/implementation-plans/2026-03-07-frame-lanes/phase_06.md @@ -0,0 +1,121 @@ +# Frame Matching Lanes Implementation Plan + +**Goal:** Extend the PE's frame-based matching to support multiple simultaneous pending operands per instruction within a single activation via matching lanes. + +**Architecture:** Multiple `activation_id` values share one physical frame (constants/destinations) while maintaining independent matching state per lane. Tag store maps `act_id → (frame_id, lane)`. Match data, presence, and port storage gain a lane dimension. + +**Tech Stack:** Python 3.12, SimPy 4.1, pytest + hypothesis + +**Scope:** 6 phases from original design (phases 1-6) + +**Codebase verified:** 2026-03-07 + +--- + +## Acceptance Criteria Coverage + +This phase implements and tests: + +### frame-lanes.AC8: Test Coverage (final) +- **frame-lanes.AC8.6 Success:** Test: full loop pipelining scenario — two iterations of a dyadic instruction running concurrently on different lanes, both producing correct results. + +--- + + +### Task 1: Full loop pipelining integration test + +**Verifies:** frame-lanes.AC8.6 + +**Files:** +- Modify: `tests/test_pe_lanes.py` — add integration test class + +**Implementation:** + +Add an integration test that simulates the complete loop pipelining lifecycle from the design plan's Architecture section. This test exercises every Phase 1-5 feature together. + +**Testing:** + +The test must verify frame-lanes.AC8.6 by simulating this lifecycle: + +``` +1. ALLOC(act_id=0) → frame, lane 0 +2. Setup: write constants/dests to frame +3. Iteration 1: inject L and R DyadTokens for act_id=0 +4. ALLOC_SHARED(act_id=1, parent=0) → same frame, lane 1 +5. Iteration 2: inject L and R DyadTokens for act_id=1 +6. Both iterations match independently, both produce correct results +7. FREE(act_id=0) → lane 0 freed, frame stays +8. FREE(act_id=1) → last lane, frame returned to free list +``` + +**Test structure:** + +Follow the established test patterns from `tests/test_pe_frames.py`: +- Use `simpy.Environment()` and `PEConfig(on_event=events.append)` +- Direct PE construction (no `build_topology` needed for single-PE test) +- Use `inject_and_run()` helper for sequential token injection +- Use a `simpy.Store` as `pe.route_table[target]` to capture output tokens + +**Detailed scenario:** + +1. **PE setup:** Create PE with `frame_count=4`, `matchable_offsets=4`, no pre-loaded frames or tag_store. Install a dyadic ADD instruction at IRAM offset 0 with `OutputStyle.INHERIT`, `dest_count=1`, `fref=8`. + +2. **Allocate frame for iteration 1:** + - Inject `FrameControlToken(target=0, act_id=0, op=FrameOp.ALLOC, payload=0)` + - Verify `FrameAllocated` event with `lane=0` + - Get `frame_id` from `pe.tag_store[0]` + +3. **Write destination to frame:** + - Write a `FrameDest(target_pe=1, offset=0, act_id=0, port=Port.L, token_kind=TokenKind.MONADIC)` to `pe.frames[frame_id][8]` + - Set up `pe.route_table[1] = simpy.Store(env)` to capture output + +4. **Allocate shared frame for iteration 2:** + - Inject `FrameControlToken(target=0, act_id=1, op=FrameOp.ALLOC_SHARED, payload=0)` (payload=0 is parent act_id) + - Verify `FrameAllocated` event with `lane=1` and same `frame_id` + - Verify `pe.tag_store[1][0] == pe.tag_store[0][0]` (same frame_id) + +5. **Inject iteration 1 operands (act_id=0):** + - Inject `DyadToken(target=0, offset=0, act_id=0, data=100, port=Port.L)` + - Inject `DyadToken(target=0, offset=0, act_id=0, data=200, port=Port.R)` + - Verify `Matched` event for `act_id=0` with `left=100, right=200` + - Verify output token emitted with `data=300` (100+200) + +6. **Inject iteration 2 operands (act_id=1):** + - Inject `DyadToken(target=0, offset=0, act_id=1, data=1000, port=Port.L)` + - Inject `DyadToken(target=0, offset=0, act_id=1, data=2000, port=Port.R)` + - Verify `Matched` event for `act_id=1` with `left=1000, right=2000` + - Verify output token emitted with `data=3000` (1000+2000) + +7. **Interleaved verification:** + - Confirm that iteration 1's L operand (injected first) did NOT interfere with iteration 2's matching — they're on different lanes + - Both `Matched` events should have independent operand values + +8. **Free iteration 1 (not last lane):** + - Inject `FrameControlToken(target=0, act_id=0, op=FrameOp.FREE, payload=0)` + - Verify `FrameFreed` with `frame_freed=False` + - Verify `0 not in pe.tag_store` + - Verify `1 in pe.tag_store` (iteration 2 still active) + - Verify `frame_id not in pe.free_frames` (frame stays allocated) + +9. **Free iteration 2 (last lane):** + - Inject `FrameControlToken(target=0, act_id=1, op=FrameOp.FREE, payload=0)` + - Verify `FrameFreed` with `frame_freed=True` + - Verify `1 not in pe.tag_store` + - Verify `frame_id in pe.free_frames` (frame returned to pool) + +**Key assertions for AC8.6:** +- Both iterations produce mathematically correct results (100+200=300, 1000+2000=3000) +- Both iterations ran on the SAME frame (shared constants/destinations) +- Both iterations used DIFFERENT lanes (lane 0 and lane 1) +- Freeing one iteration preserved the other's state +- Freeing the last iteration returned the frame + +**Verification:** +Run: `python -m pytest tests/test_pe_lanes.py -v -k "loop_pipelining"` +Expected: Test passes. + +Run: `python -m pytest tests/ -v -x` +Expected: All tests pass. + +**Commit:** `jj commit -m "test: add full loop pipelining integration test (AC8.6)"` + diff --git a/docs/implementation-plans/2026-03-07-frame-lanes/test-requirements.md b/docs/implementation-plans/2026-03-07-frame-lanes/test-requirements.md new file mode 100644 index 0000000..d9c3d6a --- /dev/null +++ b/docs/implementation-plans/2026-03-07-frame-lanes/test-requirements.md @@ -0,0 +1,40 @@ +# Test Requirements: Frame Matching Lanes + +## Automated Test Coverage + +| AC ID | Criterion | Test Type | Expected Test File | Implementation Phase | +|-------|-----------|-----------|-------------------|---------------------| +| frame-lanes.AC1.1 | `tag_store` maps `act_id -> (frame_id, lane)` where `lane` is an `int` in range `[0, lane_count)` | unit | `tests/test_pe_frames.py` (existing tests adapted to tuple API) | Phase 1 | +| frame-lanes.AC1.2 | `PEConfig.initial_tag_store` type is `dict[int, tuple[int, int]]`. PE constructor initialises tag_store from it. | unit | `tests/test_pe_frames.py`, `tests/test_pe_events.py`, `tests/test_pe.py` (all existing tests updated to pass tuple values) | Phase 1 | +| frame-lanes.AC1.3 | `PEConfig.lane_count` field exists with default 4. Controls third dimension of match arrays. | unit | `tests/test_pe_lanes.py` (verified structurally when match arrays gain lane dimension in Phase 2) | Phase 1 | +| frame-lanes.AC1.4 | All existing tests pass with updated tuple API. | integration | `tests/` (full test suite regression run) | Phase 1 | +| frame-lanes.AC2.1 | Match operand data lives in `match_data[frame_id][offset][lane]`, separate from `frames[frame_id][slot]`. | unit | `tests/test_pe_frames.py`, `tests/test_pe.py` (existing matching tests exercise 3D match_data via lane 0) | Phase 2 | +| frame-lanes.AC2.2 | `presence[frame_id][offset][lane]` is a 3D bool array. `port_store[frame_id][offset][lane]` likewise. | unit | `tests/test_pe.py` (existing presence assertions updated from 2D to 3D indexing with `[0]` lane suffix) | Phase 2 | +| frame-lanes.AC2.3 | `_match_frame()` uses `(frame_id, match_slot, lane)` to read/write match data, presence, and port. | unit | `tests/test_pe_frames.py`, `tests/test_pe.py` (existing matching tests pass through `_match_frame` with lane parameter) | Phase 2 | +| frame-lanes.AC2.4 | `frames[frame_id][slot]` remains shared across all lanes. Constants and destinations are NOT per-lane. | unit | `tests/test_pe_lanes.py` (verified in Phase 3 AC8.1 test: two act_ids sharing a frame read the same frame slot constants) | Phase 3 | +| frame-lanes.AC3.1 | `FrameOp.ALLOC_SHARED`: PE looks up `parent_act_id` from payload, finds parent's `frame_id`, assigns next free lane, records `tag_store[act_id] = (frame_id, lane)`. Clears only that lane's presence/port bits. | unit | `tests/test_pe_lanes.py` | Phase 3 | +| frame-lanes.AC3.2 | `FrameOp.FREE_LANE`: Removes tag_store entry, clears that lane's presence/port/match_data. Does NOT return frame to free list. | unit | `tests/test_pe_lanes.py` | Phase 3 | +| frame-lanes.AC3.3 | `FrameOp.FREE` becomes smart: removes tag_store entry, clears lane data. Returns frame to free list only if no other tag_store entries reference the same frame_id. | unit | `tests/test_pe_lanes.py` | Phase 3 | +| frame-lanes.AC3.4 | `FrameOp.ALLOC` unchanged: allocates fresh frame, assigns lane 0. | unit | `tests/test_pe_lanes.py` (regression check), `tests/test_pe_frames.py` (existing ALLOC tests) | Phase 3 | +| frame-lanes.AC3.5 | `FrameAllocated` event gains `lane: int` field. `FrameFreed` event gains `lane: int` and `frame_freed: bool` fields. | unit | `tests/test_pe_events.py`, `tests/test_pe_frames.py` (existing event assertions updated with new fields) | Phase 3 | +| frame-lanes.AC3.6 | When all lanes for a frame are occupied and ALLOC_SHARED is received, PE emits `TokenRejected` with reason "no free lanes". | unit | `tests/test_pe_lanes.py` | Phase 3 | +| frame-lanes.AC4.1 | ALLOC_REMOTE reads `fref+2` from frame. If non-zero, emits `FrameControlToken` with `op=ALLOC_SHARED` and `payload=parent_act_id`. If zero, emits `op=ALLOC`. | unit | `tests/test_pe_lanes.py` | Phase 4 | +| frame-lanes.AC4.2 | No new opcodes. Behaviour is entirely data-driven from frame constants. | unit | Verified by absence: `grep -r "ALLOC_SHARED\|FREE_LANE" asm/` returns no results. No dedicated test file. | Phase 5 | +| frame-lanes.AC5.1 | `FREE_FRAME` opcode uses the smart FREE behaviour from AC3.3. Frees the executing token's activation lane; returns frame to free list only if last lane. | unit | `tests/test_pe_lanes.py` | Phase 4 | +| frame-lanes.AC6.1 | `PESnapshot.tag_store` type becomes `dict[int, tuple[int, int]]`. | unit | `tests/test_snapshot.py`, `tests/test_monitor_graph_json.py` (PESnapshot constructions updated) | Phase 2 | +| frame-lanes.AC6.2 | `PESnapshot` gains `match_data`, `lane_count` fields reflecting the separated match storage. | unit | `tests/test_snapshot.py` (snapshot capture assertions updated for new fields) | Phase 2 | +| frame-lanes.AC6.4 | Monitor graph JSON serialises lane info correctly. | unit | `tests/test_monitor_graph_json.py` (assertions on serialised tag_store JSON structure with frame_id/lane keys) | Phase 5 | +| frame-lanes.AC7.1 | `codegen.py` generates `initial_tag_store` with `(frame_id, lane)` tuples. Existing single-activation code uses lane 0. | unit | `tests/test_codegen_frames.py` (existing codegen tests pass with tuple-valued initial_tag_store) | Phase 5 | +| frame-lanes.AC7.2 | No codegen changes needed for ALLOC_SHARED (manual construction only for now). | unit | Verified by absence: `grep -r "ALLOC_SHARED\|FREE_LANE" asm/` returns no results. No dedicated test file. | Phase 5 | +| frame-lanes.AC8.1 | Two act_ids sharing a frame via ALLOC_SHARED have independent matching: L operand for act_id 0 does not interfere with L operand for act_id 1 at the same offset. | unit | `tests/test_pe_lanes.py` | Phase 3 | +| frame-lanes.AC8.2 | ALLOC_SHARED with all lanes occupied emits TokenRejected. | unit | `tests/test_pe_lanes.py` | Phase 3 | +| frame-lanes.AC8.3 | FREE on a shared frame frees only the lane; other lanes' data is preserved. FREE on last lane frees the frame. | unit | `tests/test_pe_lanes.py` | Phase 3 | +| frame-lanes.AC8.4 | ALLOC_REMOTE emits ALLOC_SHARED when `fref+2` is non-zero. | unit | `tests/test_pe_lanes.py` | Phase 4 | +| frame-lanes.AC8.5 | ALLOC_REMOTE emits ALLOC when `fref+2` is zero (backwards compatible). | unit | `tests/test_pe_lanes.py` | Phase 4 | +| frame-lanes.AC8.6 | Full loop pipelining scenario: two iterations of a dyadic instruction running concurrently on different lanes, both producing correct results. | e2e | `tests/test_pe_lanes.py` | Phase 6 | + +## Criteria Requiring Human Verification + +| AC ID | Criterion | Justification | Verification Approach | +|-------|-----------|---------------|----------------------| +| frame-lanes.AC6.3 | Monitor REPL `pe` command displays lane info in tag_store output. | The existing REPL tests at `tests/test_repl.py` only assert that `do_pe()` produces non-empty output (`len(out) > 0`); they do not assert on specific formatting content. The formatting change from `{0: 0}` to `{0: frame 0 lane 0}` is a display concern that is most reliably verified by visual inspection. While a string-matching test could be added, the REPL formatting is intentionally loosely tested to allow cosmetic changes without test churn. | Run `python -m monitor` with a loaded program, execute the `pe 0` command, and confirm tag_store entries display as `act_id: frame F lane L` format. Verify that multi-lane scenarios (after ALLOC_SHARED) show distinct lane numbers per act_id. |