diff --git a/docs/plans/PLAN-BACKUP3-X10-CANONICAL-MAP.md b/docs/plans/PLAN-BACKUP3-X10-CANONICAL-MAP.md new file mode 100644 index 0000000..d332c1e --- /dev/null +++ b/docs/plans/PLAN-BACKUP3-X10-CANONICAL-MAP.md @@ -0,0 +1,142 @@ +# PLAN-BACKUP3-X10-CANONICAL-MAP — Update ole-blu backup3 to the X10 canonical structure + +Status: DRAFT for operator review (no execution yet). 2026-08-20. +Canonical copy: `~/dev/_shared/docs/plans/` (Tangled-synced; repo is PUBLIC — paths only, no secrets). +Related: `docs/BACKUP3-PLAN.md`, studio `~/msd512-audit/PLAN-3B-X10-REORG.md` + +`PLAN-3B-EXECUTION.md` (STEP 5 = this task), `backup-move-scripts` skill. +Owner: Patrick Singletary. + +## 0. Live status (Mac Studio, 2026-08-20 ~07:15 EDT) + +- Crucial 2TB -> X10 backup/mirror: COMPLETE (112,656 files, 840 GiB, 0 errors) at + `/Volumes/Crucial X10/_photos-library-mirror/`. +- X10 full inventory + dedup analysis: COMPLETE (230,326 files, all sha256; 671 GiB recoverable). +- X10 reorg STEP 2 (move+dedup): RUNNING in tmux `x10-step2` (caffeinate) since ~06:21. + ~50.7k / 122k rows done (~41%) at 07:11; ledger `~/msd512-audit/run-x10/move-ledger.tsv`. + Move plan: 111,520 MOVE rows (1,999.5 GiB) + 10,526 DEDUP rows (555.2 GiB), 0 conflicts. +- Remaining gates: GATE-2 (move ledger review) -> STEP 3 (empty legacy dir cleanup) -> + GATE-3 -> STEP 4 (verify + freeze) -> GATE-4. Structure is NOT frozen yet. +- ole-blu backup3: launchd LIVE (60s interval, --apply), correctly no-op'ing ("X10 not + mounted", last run 07:09). Safe until the X10 physically attaches to the Air. +- mac-studio backup3: launchd UNLOADED (reorg R7), NO live `~/.config/backup3/` — inert. + Its future update is PLAN-3B STEP 5; this plan is its template (shared config). + +## 1. Goal + +Update ole-blu's backup3 (script + shared config) to write into the X10's post-reorg +canonical structure, keeping per-machine subdirs so both machines back up collision-free. +Design the mapping in SHARED config so the Mac Studio's later update reuses it as-is. +OneDrive targets unchanged (active/small tier stays `OneDrive//`). + +## 2. Canonical X10 structure (locked by reorg plan) + +``` +Media/{Movies,TV,Twitch,Music-Videos,Clips,Rips,Other-Video}/ +Music/ Pictures/ Documents/ Desktop/ Downloads/ QBT/{torrents,incomplete}/ +z-New folder/ Archive/ photos-restore.photoslibrary/ _ledger/ _CONFLICTS/ +``` + +Reorg mapping that touches backup3 history: `archive/` -> `Archive/` (relative paths +preserved, so historical per-machine dirs land at `Archive//`), +`downloads/` -> `Downloads/` (legacy `downloads/ole blu/` -> `Downloads/ole blu/`). + +## 3. Mapping: current -> canonical (per-machine subdir `` appended) + +| Category | Current X10 target | New X10 target | +|---|---|---| +| Music (~/Music) | archive/Music/ | X10/Music/ (split w/ OneDrive) | +| Pictures | archive/Pictures/ | X10/Pictures/ (split) | +| Documents | archive/Documents/ | X10/Documents/ (split) | +| Desktop | archive/Desktop/ | X10/Desktop/ (split) | +| Movies (~/Movies) | archive/Movies/ (always) | X10/Media/Movies/ (always) [DP-4] | +| Downloads/stacher | archive/stacher/ | X10/Media/Music-Videos/ (stacher = music videos, per reorg) | +| Downloads/Media | archive/media/ | X10/Media/Other-Video/ [DP-2] | +| Downloads/*installer | archive/installers/ | X10/Archive/installers/ [DP-3] | +| Downloads/Music | OneDrive/Music/ | unchanged | +| Downloads/*.pdf | OneDrive/Documents/ | unchanged | +| Downloads/other | archive/downloads/ | X10/Downloads/ (split) | +| Ledger mirror | archive/_ledger/ledger-.csv | X10/_ledger/ledger-.csv | +| Run-log mirror | archive/_ledger/run--*.log | X10/_ledger/ | +| Cleanup trash | X10/.backup3-trash// | unchanged (root hidden dir) | + +## 4. Code changes (`~/dev/_shared/bin/backup3.zsh`) + +1. `run_categories` (2 call sites): x10 target `"$xd/$cat/$MACHINE"` -> `"$xd/$MACHINE"`. + x10_base semantic changes from "root" to "exact canonical parent"; OneDrive side + (`"$od/$cat/$MACHINE"`) unchanged. +2. `route_downloads`: `xd="$ARCHIVE_ROOT"` -> `xd="$X10_ROOT"`; rewrite the five X10 + path strings per Section 3 (stacher, Media, installers, downloads, ledger-adjacent). +3. Ledger + run-log mirror block: `"$ARCHIVE_ROOT/_ledger"` -> `"$X10_ROOT/_ledger"`. +4. `cleanup_pass` prune list: drop `archive`, add `_ledger`, `_CONFLICTS`, + `.backup3-trash`, `photos-restore.photoslibrary` (opaque bundle, never scanned — + matches reorg "LEAVE 108,278"). +5. NEW structure assertion (canonical_ok): after `x10_mounted`, require + `$X10_ROOT/_ledger` AND at least one canonical top-level (Downloads|Media|Pictures| + Documents|Desktop|Music); else log "X10 present but not canonical — no-op" and exit 0. + [DP-5] Backstop for attaching the drive mid-reorg or pre-freeze. +6. `status()`: print canonical-marker state. + +Unchanged: rsync opts, size threshold, ledger schema, launchd plist, OneDrive routing, +UUID mount check. + +## 5. Config changes (shared so the Studio inherits) + +- `~/.config/backup3/backup3.conf` (+ repo `config/backup3/backup3.conf`): add + `CANON_ROOT="$X10_ROOT"`; update comments. `ARCHIVE_ROOT` kept for the legacy/back-compat + path until the Studio migration lands. +- `~/.config/backup3/machines/ole-blu.conf` (+ `mac-studio.conf` + repo stub): CATEGORIES + x10_base -> `$CANON_ROOT/`; `Movies -> $CANON_ROOT/Media/Movies|yes`. + Update the schema comment (x10_base is now the exact parent). + +## 6. Decision points (a/b/c) + +- DP-1 Machine dir naming: (a) keep `ole-blu`/`mac-studio` (hyphen, current MACHINE_ID); + legacy `ole blu` dirs stay as history [recommended]; (b) also one-time rename legacy + `Downloads/ole blu` -> `Downloads/ole-blu` post-reorg + _ledger note; (c) adopt whatever + the reorg left and change MACHINE_ID. +- DP-2 Downloads/Media dest: (a) X10/Media/Other-Video/ [recommended — reorg's + "undetermined video" bucket]; (b) X10/Media/Clips/; (c) X10/Media/Movies/. +- DP-3 installers dest: (a) X10/Archive/installers/ [recommended]; (b) X10/Downloads/ + installers/; (c) X10/Archive/_misc/installers/. +- DP-4 ~/Movies routing (Air: screen recordings + iMovie Library): (a) all -> + X10/Media/Movies/ [recommended, simple]; (b) iMovie Library + screen recordings -> + X10/Media/Other-Video/, rest -> Movies; (c) split by extension into Clips/Other-Video. +- DP-5 Structure assertion: (a) require _ledger + one canonical top-level [recommended]; + (b) require _ledger only; (c) no assertion (trust UUID check). +- DP-6 Activation: (a) implement now, activate after reorg GATE-4 freeze [recommended]; + (b) implement + activate immediately (assertion is the backstop); (c) hold until the + Studio script is updated too, do both in one pass. + +## 7. Phases (gated, same posture as the reorg) + +- P1 Implement: edits to backup3.zsh + confs in `_shared`; commit by filename; plain push + (RESUME.md flags remote force-diverged — NEVER force-push; report if rejected). +- P2 Verify on Air: `zsh -n`; `--status`; sandbox routing harness (one file per routing + rule -> assert WOULD-MOVE targets are canonical, zero dirs created in dry-run, + assertion no-ops on a fake non-canonical X10); `plutil -lint` plist. +- P3 Activate: copy new confs to `~/.config/backup3/` on Air; launchd untouched; first + supervised `--apply` the first time the X10 attaches after GATE-4. +- P4 Verify live: ledger rows land at canonical paths; `X10/_ledger/ledger-ole-blu.csv` + mirrored; spot-check restores. +- P5 Studio model (later, PLAN-3B STEP 5): same shared confs + this mapping; studio + installs live config, re-enables launchd (`launchctl load .../com.psingletary.backup3.plist`). + +## 8. Risks + +- Drive attached to Air mid-reorg: assertion no-ops (DP-5) + user guidance (don't attach + until GATE-4). +- Legacy `ole blu` vs `ole-blu` sibling dirs (DP-1). +- Historical restore paths change (archive/X -> Archive/X): covered by ledger + _ledger + provenance (reorg writes `_ledger/provenance.csv`). +- Cross-machine dedup stays deferred (reorg GATE-0 deferred ~124 GB of cross-category + byte-identical dups; ExFAT has no hardlinks) — per-machine dirs intentionally duplicate. +- `_shared` remote force-diverged: plain push only; if rejected, surface to operator. + +## 9. Verification checklist (after P2/P3) + +1. Dry-run routes each rule to the canonical target (grep WOULD-MOVE lines). +2. No double-nested destinations (`grep -E '//$'` finds nothing). +3. Dry-run created NO directories anywhere (incl. absolute /Volumes targets). +4. Assertion fires (exit 0, logged) against a legacy-layout fake mount. +5. `zsh -n` on script + all confs; `plutil -lint` on plist. +6. Live apply: ledger lines at canonical dests; `_ledger/ledger-ole-blu.csv` updated.