From abc1a7cfe255a15bd23c9a858b2cf0841e4517d9 Mon Sep 17 00:00:00 2001 From: David Hagerty Date: Mon, 9 Feb 2026 11:58:50 -0500 Subject: [PATCH] docs: add human test plan for V2 Phase 1 Co-Authored-By: Claude Opus 4.6 --- docs/test-plans/2026-02-07-v2-phase1.md | 143 ++++++++++++++++++++++++ 1 file changed, 143 insertions(+) create mode 100644 docs/test-plans/2026-02-07-v2-phase1.md diff --git a/docs/test-plans/2026-02-07-v2-phase1.md b/docs/test-plans/2026-02-07-v2-phase1.md new file mode 100644 index 0000000..3249c55 --- /dev/null +++ b/docs/test-plans/2026-02-07-v2-phase1.md @@ -0,0 +1,143 @@ +# Human Test Plan: V2 Phase 1 + +**Generated:** 2026-02-09 +**Implementation plan:** `docs/implementation-plans/2026-02-07-v2-phase1/` +**Test requirements:** `docs/implementation-plans/2026-02-07-v2-phase1/test-requirements.md` + +## Automated Test Summary + +**Total acceptance criteria:** 53 +**Automated coverage:** 44 criteria (83%) across 16 test files +**Human verification:** 7 criteria (13%) +**Compile-time verification:** 1 criterion (2%) +**Hybrid:** 1 criterion + +All 283 automated tests pass (`cargo test`). + +--- + +## Human Verification Checklist + +### Prerequisites + +1. Build the project: `cargo build` +2. Set up a test project: + ```bash + cargo run -- project add test-proj /tmp/test-proj + ``` +3. Create test data (goal + tasks + decisions) by running: + ```bash + cargo run -- run --project test-proj "Create a hello world program" + ``` + (Requires `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` environment variable) + +--- + +### P1d.AC1.1: TUI removal (compile-time) + +**Criterion:** No references to `ratatui`, `crossterm`, or `tui` remain in source. + +- [ ] Run `cargo check` -- should succeed +- [ ] Search for removed dependencies: + ```bash + grep -r "ratatui\|crossterm\|tui" src/ --include="*.rs" + ``` + Expected: No matches + +--- + +### P1c.AC5.1: Session listing CLI + +**Criterion:** `rustagent sessions` lists sessions with IDs, start/end times, and goal references. + +- [ ] Run `cargo run -- sessions --help` -- verify subcommands listed +- [ ] Run `cargo run -- sessions --goal ` (use goal from prerequisite) +- [ ] Verify output lists session(s) with: + - Session ID + - Start time + - End time (or "ongoing") + - Goal reference + +--- + +### P1c.AC5.2: Session latest CLI + +**Criterion:** `rustagent sessions latest` displays handoff notes with all 4 sections. + +- [ ] Run `cargo run -- sessions latest --goal ` +- [ ] Verify output contains handoff notes with sections: + - `## Done` (lists completed tasks) + - `## Remaining` (lists pending/in-progress tasks) + - `## Blocked` (lists blocked tasks or shows none) + - `## Decisions Made` (lists decisions with chosen options) + +--- + +### P1c.AC5.3: ADR export CLI + +**Criterion:** `rustagent decisions export` writes ADR markdown files. + +- [ ] Run `cargo run -- decisions export --project test-proj --output /tmp/adrs` +- [ ] Verify ADR files appear in `/tmp/adrs/` +- [ ] Inspect file contents for: + - `# ADR-NNN:` title header + - `## Status:` section + - `## Context:` section + - `## Options Considered:` with labels + - `## Outcome:` section (if applicable) + +--- + +### P1c.AC5.4: Graph interchange CLI + +**Criterion:** `rustagent graph export/import/diff` commands work end-to-end. + +- [ ] Run `cargo run -- graph export --goal ` -- verify TOML output +- [ ] Save output to file: `cargo run -- graph export --goal > /tmp/graph.toml` +- [ ] Run `cargo run -- graph import /tmp/graph.toml` -- verify import summary +- [ ] Modify a node title in `/tmp/graph.toml` +- [ ] Run `cargo run -- graph diff /tmp/graph.toml` -- verify diff shows the change +- [ ] Run `cargo run -- graph --help` -- verify subcommands listed + +--- + +### P1d.AC6.1: Single-agent run CLI + +**Criterion:** `rustagent run` creates goal, session, executes agent loop, records outcome. + +- [ ] Set `ANTHROPIC_API_KEY` environment variable +- [ ] Run: + ```bash + RUST_LOG=rustagent=debug cargo run -- run --project test-proj "Create a hello world program" + ``` +- [ ] Verify in logs/output: + - Goal node created in DB + - Session created + - Agent executes tool calls (visible in debug logs) + - Outcome recorded (Completed, Blocked, or TokenBudgetExhausted) +- [ ] Run `cargo run -- sessions latest --goal ` -- verify handoff notes generated + +--- + +### P1d.AC6.2: Profile selection CLI + +**Criterion:** `--profile` flag selects agent profile; invalid profile gives clear error. + +- [ ] Run with explicit profile: + ```bash + cargo run -- run --project test-proj --profile reviewer "Review the codebase" + ``` + Verify the agent uses the reviewer profile's system prompt (visible in debug logs with `RUST_LOG=rustagent=debug`) +- [ ] Run with invalid profile: + ```bash + cargo run -- run --project test-proj --profile nonexistent "Do something" + ``` + Verify a clear error message (not a panic or stack trace) + +--- + +## Notes + +- All human verification items are CLI integration tests that require either live LLM API keys or are thin presentation layers over already-tested business logic. +- The underlying business logic for every human verification item is covered by automated tests (see test-requirements.md for mappings). +- If any human verification step fails, check the corresponding automated test first to isolate whether the issue is in the business logic or the CLI wiring. -- 2.51.2