diff --git a/TODO.md b/TODO.md new file mode 100644 index 0000000..8ad681e --- /dev/null +++ b/TODO.md @@ -0,0 +1,134 @@ +# TODO.md — Sifter + +Source: PLAN-sifter.md (archived). Do not re-open decisions in §10 of the plan; +they are inputs, not suggestions. + +Status: scaffold only. Build starts ONLY after explicit user approval. + +--- + +## Pre-work (human, before any build) + +- [ ] Copy `src/contexts/AuthContext.js`, the login screen component, and shared + layout/styles from Verifier into `vendor/` +- [ ] Paste Verifier's README section skeleton into `vendor/README-skeleton.md` +- [ ] Create Tangled repo: https://tangled.org/new/repo (name: sifter) + and `git push -u origin main` + +Agent lifts from `vendor/` verbatim; never fetches from the web. + +--- + +## M1 — Scaffold (CRA + auth + deploy) + +- [ ] CRA app (JavaScript, not TS). Ignore CRA deprecation warning; do NOT + migrate to Vite. Jest via `react-scripts test` only. +- [ ] OAuth via `@atproto/oauth-client-browser` in `src/contexts/AuthContext.js` + (lifted from vendor/). Scope: `atproto transition:generic` +- [ ] `clientMetadata` in AuthContext.js AND `public/client-metadata.json`, + matching deployed domain sifter.psingletary.com +- [ ] Login screen + layout/styles lifted from vendor/ (login page explainer: + "most accounts have zero orphans; this tool mostly exists to prove that") +- [ ] Deploys to wisp.place (`wispctl deploy --path ./build --site sifter --spa`) + and Tangled Sites (deploy dir `/build`, SPA fallback) +- [ ] Login works end-to-end + +## M2 — Scan (ship publicly on its own — decided, §10#2) + +- [ ] `src/lib/scan.js` — pure: page `listBlobs` (limit 1000), `describeRepo`, + page `listRecords` (limit 100) per collection, diff held − referenced +- [ ] `src/lib/extract.js` — generic recursive walk (never hardcode paths); + fixtures F1–F5 in `src/__tests__/__fixtures__/` (§3b is the single source + of truth for shapes) +- [ ] `missing` cross-check against `com.atproto.repo.listMissingBlobs`; + disagreement → warning surfaced +- [ ] Per-collection blob-ref accounting (sanity check table) +- [ ] Fail-closed `trustworthy` flag: false if ANY listRecords/listBlobs page + fails, describeRepo fails, pagination ends early, or listMissingBlobs + disagrees → report read-only + banner + retry; deletion disabled +- [ ] Report UI: headline numbers, per-collection table, orphan grid + (thumbnail via getBlob, truncated CID click-to-copy, MIME, size, + checkboxes, select-all, per-item download), `missing` listed separately + with explanation, atproto.at link for independent verification +- [ ] Tests: `extract.test.js` (F1–F5), `scan.test.js` (mocked paging incl. + failure → trustworthy=false; clean mocked account → 0 orphans) +- [ ] Author validates scan against real account + (did:plc:stznz7qsokto2345qtdzogjb, PDS lionsmane.us-east.host.bsky.network), + cross-checks blob count against https://atproto.at +- [ ] Zero-orphan case renders as success state, not empty state +- [ ] Large-repo handling: cursor loops with live progress counts, cap ~4 + collections parallel (scan only), memory = Sets of CIDs + +> **HARD STOP — obtain author sign-off before M3/M4.** + +## M3 — Backup (before any deletion exists) + +- [ ] `src/lib/backup.js` — getBlob fetch each selected blob + `manifest.json` + (CIDs, MIME, sizes, direct URLs) + zip via `jszip` +- [ ] Backup is the path of least resistance; delete gated behind completed + backup OR explicit acknowledgement +- [ ] Per-item download from grid (M2 carries it, verify) + +## M4 — Cleanup, single blob (mechanism proof) + +- [ ] `src/lib/cleanup.js` — serial reference-and-delete per orphan: + createRecord in `com.psingletary.sifter.tempRef` (TID rkey, minimal + value { $type, blob, createdAt }) → deleteRecord → PDS GC +- [ ] Blob metadata recovery via getBlob HEAD-style fetch (Content-Type, + Content-Length; fallback application/octet-stream / 0); cache once +- [ ] Create rejected → retry once with `validate: false`; still failing → + mark `undeletable`, move on. Never use a different collection. +- [ ] Mandatory verification: after each batch re-run listBlobs, per-CID + outcome `deleted` / `still-present` (abort run) / `undeletable`. + Never trust deleteRecord 200 alone. +- [ ] Rate limiting: ~1s pacing default; read `ratelimit-*` headers on every + write, back off as remaining→0, show countdown. VERIFY constants against + current Bluesky docs at build time (plan §4: 5000/hr, 35000/day, + CREATE=3, DELETE=1 → 4 pts/orphan, fallbacks only) +- [ ] localStorage run state keyed by DID; startup sweep of own temp + collection (listRecords + delete strays, report count); idempotent per CID +- [ ] **Live test performed by AUTHOR on one blob, verified via re-scan. Agent + does not run the live test.** + +> **HARD STOP — wait for author's verified live test before M5.** + +## M5 — Batch cleanup + +- [ ] Batch UI: per-CID live log, pause/resume, rate-limit countdown, final + verified summary +- [ ] Only after M4 live test proves the mechanism on the author's PDS +- [ ] Optional defer: `applyWrites` single-transaction create+delete — only if + live test proves blob GC fires; two-call path is the shipped default + +## M6 — Polish + README + +- [ ] README matching Verifier's section order (vendor/README-skeleton.md): + title, Features, What this does, Usage, Development, Build, Deployment, + OAuth Configuration, License (MIT) +- [ ] Honest documentation of the mechanism incl. throwaway records + firehose + visibility; self-hosted PDS note (SQL cross-check); dedup note (no + storage-usage graph movement) +- [ ] Edge cases from §6 all handled (deactivated account, getBlob 404 → + already-absent, mechanism-doesn't-work abort, third-party lexicons, + legacy blob format, non-image icons) +- [ ] No API keys, no backend, no analytics + +--- + +## Acceptance criteria (from plan §8) + +- [ ] (Manual, author) Scan on account with known orphans finds them +- [ ] (Auto) Clean mocked account → 0 orphans, stated plainly +- [ ] (Auto) Forced listRecords failure → trustworthy=false, deletion disabled +- [ ] (Auto) Extractor fixtures F1–F5 pass +- [ ] No orphan deleted without completed backup or explicit acknowledgement +- [ ] Every reported deletion confirmed by follow-up listBlobs +- [ ] No temp records remain in `com.psingletary.sifter.tempRef` after any run, + including interrupted mid-flight +- [ ] Works from static hosting; no backend + +## Out of scope (§9) + +Missing-blob fixing (report only) · server/admin cleanup · deleting user +records · multi-account/delegated cleanup · CAR-file fast path (deferred idea +only, behind validation)