diff --git a/AGENTS.md b/AGENTS.md index e11e5ac..067d431 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,377 +1,160 @@ # AGENTS.md -Architecture and rules for agents editing this project. - -## Project Shape - -This is a web app for npm supply-chain audits. Preserve the documented audit -behavior and invariants unless the user explicitly asks otherwise. - -Audits run server-side and stream to the browser. When the user runs an audit -the browser POSTs a validated request to a Netlify **edge function**, which runs -the audit and streams progress and the final result back over SSE (see the SSE -Contract below). The browser is a thin client: it submits the request, renders -the streamed `log` lines in the terminal, and renders the streamed `result` in -the tables. It does not orchestrate discovery, trust, downloads, or maintainer -checks itself. - -This is deliberate. Because the server computes the report, the report is -authoritative by construction — the browser never submits trust data — and there -is no CORS-proxy/SSRF surface to maintain. Do not move audit computation back -into the browser, and do not add a browser-submitted report-write path. - -The server-side audit graph under `src/lib/*` runs in both the Deno edge runtime -and the Node scheduled-function runtime. Keep code reachable from `runAudit` or -`runUserPublishes` free of runtime-specific APIs: hashing uses Web Crypto -(`crypto.subtle`), not `node:crypto`, and audit code must not use browser-only -globals. - -The server-side entry points are: - -- `netlify/edge-functions/audit-stream.ts` — `POST /api/audit-stream`. Runs - `runAudit` for the `trust`/`manual`/`external` kinds, streams `log`/`result`, - saves the completed report server-side, and streams a final `done` with the - saved report id. This is the trust boundary. The run is a resumable durable job - (keyed by a client `jobId`; see the SSE Contract) so it survives the platform's - ~60s connection recycle — the first request runs the audit, reconnects tail it. -- `netlify/edge-functions/user-publishes-stream.ts` — - `POST /api/user-publishes-stream`. Runs `runUserPublishes` and streams - `log`/`result`. It does not persist anything. -- `netlify/functions/trust-reruns-background.ts` — an hourly scheduled background - function. It reruns the all-package `trust`/package-trust report for opted-in - org sets so the public timeline can grow without a browser session. It must not - run, derive, store, or query `manual` or `external` report data. -- `netlify/functions/audit-jobs-cleanup-background.ts` — an hourly scheduled - background function that prunes `audit_jobs` rows older than 2h (the resumable - audit's transient progress store; the durable report lives in `reports`). -- `netlify/functions/reports.ts` — a serverless function for reading stored - reports and the public trust timeline, and for creating daily tracking - schedules, through Netlify Database. It does not compute audits and has no - report-write endpoint. - -## npm Access - -npm is fetched directly from the server. Because audits run in edge/background -functions, not the browser, there is no cross-origin restriction and no need for -a proxy. There is no host-generic proxy and no request-controlled upstream host -— i.e. no SSRF surface to defend. - -`src/lib/npmClient.ts` (`npmGet`/`npmGetJson`) fetches the upstream hosts -directly, with the retry/backoff/`FailureLog` semantics under Invariants: - -- `registry.npmjs.org` — packuments and per-version manifests. -- `api.npmjs.org` — weekly download counts. -- `npm.antfu.dev` (fast-npm-meta) — batched discovery metadata. - -When adding a new npm upstream, add the URL helper in `npmClient.ts` and fetch -it directly; do not add a proxy layer. - -## Edge Bundling (Deno) - -The edge functions run on Deno, and Netlify's npm-in-edge bundling is still beta: -it fails to load some clean-ESM third-party packages the audit graph pulls in -(currently `valibot` and `packumeta`). Those are mapped to esm.sh in -`import_map.json`, wired via `deno_import_map` in `netlify.toml`, so the edge -bundler resolves Deno-compatible builds at deploy time. The Vite browser build -and vitest ignore the import map and keep resolving the same bare specifiers from -`node_modules`, so nothing changes for the client. - -- Relative imports in any file reachable from an edge function MUST carry an - explicit `.ts` extension. Deno resolves the literal specifier — it does not add - extensions or map `.js` -> `.ts` — so extensionless or `.js` relative imports - bundle fine in Vite/vitest/Node but fail the edge bundler. That is why - `src/lib/*` and the edge-reachable `_shared`/`db` imports use `.ts` - (`tsconfig` enables `allowImportingTsExtensions`). Node-only files (e.g. - `report-schedules.ts` and the serverless functions) keep `.js`; they never enter - an edge bundle. -- Do not colocate tests in `netlify/edge-functions/`. Netlify treats every - `.ts`/`.js` file there as an edge-function entrypoint, including `*.test.ts`, - so Vitest-transformed mocks will break `netlify dev` and deploy bundling. Put - edge-function tests under `netlify/tests/edge-functions/` instead. -- This failure only surfaces at deploy-time edge bundling, NOT in local dev — the - `@netlify/vite-plugin` bundles the edge functions differently and resolves npm - deps and extensionless imports fine. Do not assume a green local run means the - edge bundle is deployable; verify with `pnpm dlx netlify-cli build` (aka - `netlify build`), which runs the real edge bundler. -- Any NEW third-party npm dependency reachable from `runAudit` / - `runUserPublishes` (i.e. bundled into an edge function) may need an - `import_map.json` entry. Keep the pinned esm.sh versions in sync with - `package.json`. -- `@netlify/database` is deliberately NOT mapped: it is Netlify-first-party and - resolves natively in functions and edge functions. Database access uses its - native tagged-SQL client; `db/schema.ts` validates returned rows with Valibot. - -## SSE Contract - -Both audit endpoints stream `text/event-stream`, one JSON payload per `data:` -line, with these events: - -- `log` — a progress line (string), carrying an `id:` (a monotonic sequence - number) so a reconnecting client can resume after the last line it saw. The - browser appends it to the terminal. -- `result` — the full report object. -- `done` — terminal success. For `audit-stream`, `{ id, url }` of the saved - report (or `{ error }` if only the save failed — the result still streamed). - For `user-publishes-stream`, `{}` (nothing is persisted). -- `error` — terminal failure (string): the audit itself failed. - -`audit-stream` is **resumable**. The client-facing connection is recycled by the -platform (~60s), far shorter than a scope-heavy audit, so one connection can't -carry the whole run and no keepalive beats a total cut. Each run is instead a -durable job keyed by a client-generated `jobId` (the `audit_jobs` table and -`netlify/functions/_shared/audit-jobs.ts`): the FIRST request runs the audit, -persisting its progress log — and, on completion, the result and saved report id -— and keeps running even if its own client disconnects. A reconnecting request -(same `jobId`, `from` = last `log` id it saw) does NOT re-run; it replays stored -lines after `from`, tails the job until it finishes, and forwards the result/link -(or error) the run recorded. `streamAudit` drives this loop transparently, so the -terminal keeps scrolling and reports stay complete however long the audit runs. -This is the platform-intended SSE pattern — reconnect + resume, as Netlify's own -EventSource examples rely on — implemented over POST because the request body -(esp. `external` member lists) can exceed URL limits. `user-publishes-stream` is -not resumable and persists nothing. - -Client readers live in `src/lib/sseStream.ts` (`readSseStream`, which reassembles -frames across chunks and parses the `id:`), with `src/lib/auditStream.ts` -(`streamAudit`, which reconnects/resumes) and `src/lib/userPublishStream.ts` -(`streamUserPublishes`) wrapping the two endpoints. Request bodies are validated -server-side with valibot (`AuditRequestSchema`, `UserPublishRequestSchema` in -`src/lib/schemas.ts`); the validated request is the trust boundary. - -## Report Sharing - -Report links are stateful, and the server owns the write. As part of a completed -`/api/audit-stream` run, `audit-stream.ts` calls `saveReportSnapshot` to store -the `AuditResult` plus completed-run context (`orgs`, `scope`, `scopeLabel`, -`capturedAt`) in Netlify Database, then streams the saved id in the `done` event. -The id has the form `--`; the hash is derived from -the payload, so saving the same report on the same day is idempotent. There is -no browser-facing report-write endpoint — the browser never POSTs a report. - -`GET /api/reports/:id` returns the stored row. The UI share action only copies -the already-created report link. - -Daily tracking is created with `POST /api/reports/:id/schedule-daily`. The -endpoint is eligible only when the saved report already has a -`report_trust_history` row, which means it came from an all-scope package trust -report. Schedules are keyed by the same normalized org set as the timeline. - -`src/AppRouter.svelte` handles `/` and `/report/:id`, preserving the shared app -shell during client-side navigation. The report route renders -`src/SharedReport.svelte`, which reuses `components/ResultsView.svelte` for the -read-only snapshot. - -Reads and schedule writes are small JSON, so `reports.ts` stays a serverless -function. Only the audit stream needs an edge function: SSE streaming with no -serverless timeout and no response-size cap on large packument-derived results. - -## Platform-first UI - -Prefer native web-platform features over hand-rolled equivalents — reinventing -them is how avoidable accessibility and behavior bugs creep in. Reach for the -platform first: `popover` / `` for overlays (the trust glossary is -`popover="auto"` + CSS anchor positioning), `navigator.clipboard` for copy, -`crypto.subtle` / `crypto.randomUUID` for hashing and ids, `matchMedia` + -`prefers-color-scheme` for theming, `scrollIntoView` / `:focus-visible` / -`role="status"|"alert"` live regions for focus, scroll, and announcements, and -the URL hash for view state. Target Baseline — _widely available_ by default, -_newly available_ when it earns its keep (as `popover` does) — and enhance -progressively: e.g. CSS anchor positioning behind `@supports` with a fixed -fallback, never a JS polyfill. - -Two deliberate exceptions exist so they don't get "fixed": - -- The audit stream reads a `fetch` `ReadableStream`, not `EventSource` — the - request body (esp. `external` member lists) can exceed URL limits and - `EventSource` is GET-only. See the SSE Contract. -- Human-facing dates and timestamps use the shared `Intl.DateTimeFormat` - helpers in `src/lib/dateFormatting.ts`, so they follow the viewer's locale - and local timezone. Keep raw ISO-8601 values for persistence, API boundaries, - sorting, report ids, exports, and `