diff --git a/toolchain/macos/NEO-CLEANUP.md b/toolchain/macos/NEO-CLEANUP.md new file mode 100644 index 0000000000..9d337f0d0b --- /dev/null +++ b/toolchain/macos/NEO-CLEANUP.md @@ -0,0 +1,151 @@ +# Cleaning Neo + +Keep **20 GiB free** for swap, updates, and interactive work. Run host commands +on Neo; its SSH login shell is fish, so send multi-line Bash explicitly: + +```bash +ssh -o BatchMode=yes -o ConnectTimeout=20 neo /bin/bash -s <<'REMOTE' +df -h /System/Volumes/Data +"$HOME/.local/bin/cleaner" --apply +df -h /System/Volumes/Data +REMOTE +``` + +Cleaner handles regenerable caches. It does not remove worktrees, project +dependencies, models, agent histories, or Photos libraries. An inaccessible +cache must be reported and skipped so the rest of the cleanup can finish. +Snapshot thinning and remote-backed media pruning remain explicit options; +neither belongs in unattended cleanup. See [SCORE.md](SCORE.md#cleaner). + +## Keeping the space free + +- Keep voice evaluation and intermediate audio on Poorslice; run video/browser + rendering and retain bulk outputs on Panda. Neo can keep the controlling + sessions and source edits. Verify remote outputs before removing local copies. +- Neo's `computer.aesthetic.cleaner` job runs daily at 03:15 with + `ProcessType=Background`, `LowPriorityIO=true`, and `Nice=10`. +- Install the hourly warning with + `bash toolchain/macos/disk-space-watch.sh --install` on Neo. It records free + space in `~/.local/share/slab/disk-space/latest.json`, marks `low-space` below + 20 GiB, and notifies at most once per 24 hours. It never deletes anything. + +The Cleaner installer currently writes a **weekly** schedule. After reinstalling +it on Neo, restore the daily schedule and background priority: + +```bash +python3 - <<'PY' +import pathlib, plistlib +path = pathlib.Path.home() / "Library/LaunchAgents/computer.aesthetic.cleaner.plist" +config = plistlib.loads(path.read_bytes()) +config.update(StartCalendarInterval={"Hour": 3, "Minute": 15}, + ProcessType="Background", LowPriorityIO=True, Nice=10) +path.write_bytes(plistlib.dumps(config)) +PY +launchctl bootout "gui/$(id -u)/computer.aesthetic.cleaner" +launchctl bootstrap "gui/$(id -u)" "$HOME/Library/LaunchAgents/computer.aesthetic.cleaner.plist" +``` + +Run that block in Bash on Neo. If the job is already unloaded, `bootout` can +report it missing; still run `bootstrap`. Inspect the installed plist and +`~/Library/Logs/cleaner.log` after changing the schedule. + +## Reviewing stale checkouts + +Commit age alone does not establish that a checkout is unused. Inspect the +registered worktrees, then each candidate: + +```bash +git -C "$HOME/aesthetic-computer" worktree list --porcelain +# Set candidate to one exact checkout being reviewed. +git -C "$candidate" status --porcelain=v1 --untracked-files=all +git -C "$candidate" ls-files --others --ignored --exclude-standard +git -C "$candidate" log -1 --format='%h %cs %s' +git -C "$candidate" rev-list --count main..HEAD +``` + +Check process command lines **and** open working directories, plus references +in `~/Library/LaunchAgents`, `~/.config/slab`, and `~/.local/bin`. For example, +the old `aesthetic-computer-slab-sync` checkout still supplied the Emacs MCP +service; `ac-jev-workshop` still had process references. Both were retained. + +Before removing an approved candidate: + +1. Record its absolute path, HEAD, branch, and merge status against the current + main branch. Preserve unmerged commits and local edits; do not classify them + as disposable because the checkout is old. +2. Review ignored and untracked files. Preserve unique QA scripts, logs, images, + and build artifacts in a recovery archive. Verify saved regular files by + checksum and saved symlinks by link target; do not follow dependency or vault + symlinks into shared trees. +3. Recheck HEAD, edits, extras, and live users immediately before removal. + Unlink only the reviewed, saved extras, then use + `git -C "$HOME/aesthetic-computer" worktree remove "$candidate"` without + `--force`. Stop if Git refuses. Keep branches and commits. +4. Record restore commands: `git worktree add` at the retained branch or exact + detached commit, followed by extraction of the saved extras into that tree. + Verify the checkout disappeared and its commit still exists. + +Fuser checkouts under `~/Developer/fuser-*` require Jeffrey's explicit approval +even when merged. Do not include worktree removal in Cleaner or a timer. + +## Retiring Neo's local Photos library + +This is an explicitly requested, one-time removal of the **local library +bundle**, not deletion of photos through the Photos app. Photo deletions inside +a synced library can propagate to iCloud. Apple's supported device-only controls +are described in [Turn off iCloud Photos](https://support.apple.com/en-us/102179); +its [library relocation guide](https://support.apple.com/en-us/108345) also +distinguishes retiring a local library from deleting individual photos. + +The terminal recovery on 2026-09-21 used this scope: + +1. Jeffrey confirmed the photos originated on his phone and authorized removal + of Neo's local copy. Missing cloud IDs in SQLite were **not** treated as proof + that photos had failed to sync. +2. Close Photos and quiesce the current user's photo-library writers while + copying/removing the bundle. Restore or restart those services on every exit; + do not leave a daemon suspended or disable its job permanently. +3. Preserve `originals/`, `internal/`, `database/Photos.sqlite`, its `-wal` and + `-shm` sidecars when present, and `database/DataModelVersion.plist` in a private, + compressed archive on Panda. Match the sender and receiver SHA-256 digests + before deleting the local library. +4. Remove the approved `~/Pictures/Photos Library.photoslibrary` bundle as a + whole. Do not edit its SQLite records or selectively delete files inside a + library that will remain in use. Verify it is gone, release old service file + handles, and measure free space again. + +That archive is **limited recovery data**, not a complete usable Photos library +or a backup of all full-resolution iCloud originals. Extract it separately to +recover/import local originals; keep the database and WAL together for metadata +recovery. Do not open the partial bundle as a live library. If a complete local +backup is required, preserve the entire closed library instead. + +Avoid copying every rebuildable preview before an authorized retirement: Neo's +library occupied about 20.3 GiB but held only about 1.7 MiB under `originals/`. +The useful recovery archive was 1.2 GiB. A full preview transfer was abandoned as +unnecessarily slow, and its incomplete copies were removed after verification. +Re-enabling Photos locally can rebuild its storage; prefer Optimize Mac Storage +if a new synced library is wanted. Photos retirement is never scheduled cleanup. + +## Verified recovery, 2026-09-21 + +| Action | Observed result | +| --- | --- | +| Repaired Cleaner | Continued past a denied cache; reclaimed about 1.6 GiB | +| Removed two idle Swift `.build` caches | About 516 MiB reclaimed | +| First five approved, merged worktrees | 16.59 GiB reclaimed | +| Four further approved worktrees | 7.45 GiB reclaimed | +| Retired local Photos library | Final filesystem reading: about 43.5 GiB free | + +The second batch was `ac-aesel-feed-live`, +`ac-worktrees/oskiewar-fighter-generation`, `ac-worktrees/oskiewar-xbox`, and +`ac-worktrees/tape-upload-fix`. Their ten regular extras and symlink targets were +saved before removal. These are historical outcomes, not a standing deletion +list. Filesystem free-space deltas can differ from `du` and change during work. + +Recovery records on Neo are under `~/.local/share/slab/recovery/20260921/`: +`worktree-cleanup-plan.json`, `four-checkouts/receipt.json`, +`four-checkouts/restore.sh`, and `photos/{receipt.json,RESTORE.txt}`. +Panda holds `~/Backups/neo-photos-20260921/Local-originals-and-database.tar.gz` +with its checksum, receipt, and recovery instructions. Keep private archives and +per-host receipts out of Git. diff --git a/toolchain/macos/SCORE.md b/toolchain/macos/SCORE.md index 7c6bbb80f1..e6e359d041 100644 --- a/toolchain/macos/SCORE.md +++ b/toolchain/macos/SCORE.md @@ -164,6 +164,24 @@ Final Cut/Xcode data, Docker, and local model stores. `--remote-backed` is interactive/explicit only: each surface is kept unless its own remote verifier passes, and it is never included in the weekly LaunchAgent. +### Neo headroom + +Use [Neo cleanup](NEO-CLEANUP.md) for the verified worktree and local Photos +procedures, recovery records, and the host's daily schedule. + +Keep at least 20 GiB free on Neo for swap and system updates. Its hourly +`disk-space-watch` checks only filesystem free space and warns at most once +per day; it never deletes data. Install it with +`bash toolchain/macos/disk-space-watch.sh --install`. Its current reading is +`~/.local/share/slab/disk-space/latest.json`. + +Neo's Cleaner runs daily at 03:15 at background priority. Inaccessible caches +are skipped rather than aborting the remaining cleanup. Keep voice evaluation +and its intermediate audio on Poorslice; keep video/browser rendering and its +bulk outputs on Panda. Neo holds the controlling sessions and source edits. +Copy back only the results needed locally. Preserve source, transcripts, +models, and personal media; move generated bulk only after verifying its copy. + ### Safe regenerable buckets Always clear first — fully recover with no judgment call: diff --git a/toolchain/macos/cleaner.sh b/toolchain/macos/cleaner.sh index 87df95ecea..be7361bbb5 100755 --- a/toolchain/macos/cleaner.sh +++ b/toolchain/macos/cleaner.sh @@ -284,7 +284,9 @@ audit() { clean_contents() { [[ -d "$1" ]] || return 0 - find "$1" -mindepth 1 -maxdepth 1 -exec rm -rf -- {} + + if ! find "$1" -mindepth 1 -maxdepth 1 -exec rm -rf -- {} +; then + skip "could not fully clear $1; continuing with other caches" + fi } skip() { diff --git a/toolchain/macos/disk-space-watch.sh b/toolchain/macos/disk-space-watch.sh new file mode 100644 index 0000000000..2c5b0016ad --- /dev/null +++ b/toolchain/macos/disk-space-watch.sh @@ -0,0 +1,58 @@ +#!/bin/bash +# Cheap free-space warning; cleanup remains the separate Cleaner job. +set -euo pipefail + +STATE="$HOME/.local/share/slab/disk-space" +LABEL=computer.aesthetic.disk-space-watch +FLOOR_KB=20971520 # 20 GiB reserved for swap, updates, and interactive work. + +case "${1:---check}" in + --install) + mkdir -p "$HOME/.local/bin" "$HOME/Library/LaunchAgents" "$STATE" + target="$HOME/.local/bin/disk-space-watch" + if [[ "$0" != "$target" ]]; then install -m 0755 "$0" "$target"; fi + plist="$HOME/Library/LaunchAgents/$LABEL.plist" + cat > "$plist" < + + + Label$LABEL + ProgramArguments$target--check + RunAtLoad + StartInterval3600 + ProcessTypeBackground + LowPriorityIO + +EOF + plutil -lint "$plist" + launchctl bootout "gui/$(id -u)/$LABEL" >/dev/null 2>&1 || true + launchctl bootstrap "gui/$(id -u)" "$plist" + echo "Installed hourly disk-space warning (20 GiB floor)." + exit 0 + ;; + --check) ;; + *) echo "usage: disk-space-watch.sh [--check | --install]" >&2; exit 2 ;; +esac + +volume=/System/Volumes/Data +[[ -d "$volume" ]] || volume=/ +available=$(df -Pk "$volume" | awk 'NR == 2 {print $4}') +case "$available" in ''|*[!0-9]*) echo "Cannot read free disk space" >&2; exit 1 ;; esac +mkdir -p "$STATE" +now=$(date +%s) +printf '{"checkedAt":%s,"availableKiB":%s,"floorKiB":%s}\n' \ + "$now" "$available" "$FLOOR_KB" > "$STATE/latest.json" + +if (( available >= FLOOR_KB )); then + # Keep the alert timestamp across recovery so a brief dip cannot notify twice. + rm -f "$STATE/low-space" + exit 0 +fi +: > "$STATE/low-space" +last=$(cat "$STATE/last-alert" 2>/dev/null || echo 0) +case "$last" in ''|*[!0-9]*) last=0 ;; esac +if (( now - last >= 86400 )); then + free_gib=$(awk -v kb="$available" 'BEGIN {printf "%.1f", kb / 1048576}') + /usr/bin/osascript -e "display notification \"${free_gib} GiB free; keep 20 GiB available. Use Cleaner and keep large jobs on compute hosts.\" with title \"Low disk space\"" >/dev/null 2>&1 + printf '%s\n' "$now" > "$STATE/last-alert" +fi