Reading a run's diagnostic report #

Ultimate Tool can explain one immutable run without re-running it, touching hardware, or reading the current project manifest. The report answers what outcome was recorded, whether the recorded evidence verifies, what the run expected versus observed, which expectations even apply, and where an instrumented crash resolved. Detail rows are paged, so a report never dumps an unbounded symbol or event list.

The CLI, HTTP service, MCP adapter, and embedded dashboard all read the same versioned core operation; none of them implements its own diagnostics. This page covers the CLI. See HTTP API and MCP for the other surfaces.

Get a run ID #

Reports are addressed by immutable run ID. List the newest runs first:

target/debug/ultimate-tool runs list --manifest c64dev.toml

Read the report #

target/debug/ultimate-tool runs report RUN_ID --project-root examples/first-run
target/debug/ultimate-tool runs report RUN_ID --manifest examples/first-run/c64dev.toml

--project-root ROOT reads run history directly from a project directory, so a report stays readable after the manifest is edited or deleted. --manifest instead derives the project root from that manifest. The two are mutually exclusive. Neither reads the manifest's current contents: a report describes what the run recorded, not what the project says now.

The report shows:

• status (finalized or provisional), outcome, target_kind, execution_plane, test, and the start and finish times;
• failure and first_error when the run recorded one;
• expectations: whether the selected test's expectation snapshot is available, and the recorded snapshot when present;
• results: expected-versus-observed terminal and screen verdicts, heartbeat disposition, and a crash summary;
• artifact identity, evidence integrity counts, and nullable budget and comparison summaries;
• counts: the row count of every detail section, so a client can size its paging without loading rows.

Use the global --format json for the machine-readable envelope (CLI envelope schema 1). The inner diagnostic report is schema 1 on every surface; the HTTP representation names it diagnostic_report_schema_version.

Page the detail sections #

Detail rows are read one bounded page at a time:

target/debug/ultimate-tool runs report-items RUN_ID --section checkpoints --project-root examples/first-run
target/debug/ultimate-tool runs report-items RUN_ID --section assertions --limit 25 --cursor CURSOR

--section is required and is one of:

--limit defaults to 50 and accepts 1 through 100. A page smaller than total prints a next cursor: line; pass that opaque token back as --cursor. The cursor is bound to its run, section, and the SHA-256 of the verified source evidence, so cross-run, cross-section, stale, and malformed cursors are rejected rather than silently returning the wrong page.

The checkpoints section reports the runner's workflow checkpoints (name/passed/detail/timestamp). It is not the frozen memory-checkpoint ledger.

Fidelity and boundaries #

• Read-only and historical. Reports and detail pages only read an existing run record and its already-recorded evidence. They never build, never contact a target, and issue no hardware requests. The outer envelopes differ by surface—CLI schema 1, service schema 13, MCP schema 12—while the inner report is schema 1 everywhere.
• Finalized runs fail closed. A finished run verifies every registered evidence file first. If a registered file no longer matches its recorded digest, the report fails with the diagnostics error instead of presenting a success-colored fallback.
• Provisional runs. A run that has not finished is provisional: only its in-record observations are available and finalized, checksum-verified detail is refused. The service may additionally report whether it still considers the run active.
• Expectation availability. Expectations are recorded, not_recorded_legacy (a finished C64 test run predating the recorded snapshot), not_applicable (a raw, source-linked, or native operation that runs no manifest test), or unavailable (a modern test run missing a matching snapshot).
• Verdicts. A result verdict is matched, mismatched, or unavailable. unavailable means the comparison cannot be made from the recorded evidence; it is neither a pass nor a failure.
• Crash summary. The report resolves the primary crash match and a bounded inline source chain (at most eight frames) only from the run's verified same-run build analysis; an ambiguous or uncovered site is withheld rather than guessed. An ordinary, uninstrumented panic reports its origin as unavailable. The captured crash-entry registers are bounded crash-state evidence for the configured crash, not a computed crash site.
• Budget and comparison appear only when the run recorded those summaries.
• Frozen memory-checkpoint summary. A finalized run with recorded capture metadata exposes a typed checkpoints summary from its verified same-run evidence. Only a full eligible proof is verified; recorded invalid or narrowly recognized legacy captures are unavailable, never valid matches. Absent captures remain null, and malformed or corrupted evidence fails closed. The paged checkpoints section above still contains workflow rows, not RAM bytes. Read the bounded memory rows through the checkpoint comparison operations.

In the dashboard #

The embedded dashboard offers the same report as a question-driven view. It loads one structured report for the selected run and fetches a section's rows only when a reader asks for that section, one manual request per page. The bus timeline is a separate opt-in control, offered only when the run actually recorded a 6510 debug capture; a non-capture run issues no timeline request. The dashboard script performs no diagnostics itself, and all of these requests are GET-only and historical.

serve still requires a valid manifest to start and binds loopback only, even though reading a report never touches the current manifest.