aesthetic.computer / core
Monorepo for Aesthetic.Computer aesthetic.computer
Star
Fork 0
Watch
core silo datomic
2 folders · 6 files
postgres
transactor
backup.cron
bootstrap-silo.fish
deploy.fish
MIGRATION.md
README.md
update-silo-env.fish
README.md

silo/datomic #

Host-side infrastructure for the Datomic Pro instance that backs kidlisp (v1). The Clojure sidecar that talks to this transactor lives at /kidlisp-sidecar/.

Components on silo #

  • Postgres — Datomic storage backend. Datomic treats it as a KV blob store; the logical schema lives inside Datomic, not in Postgres.
  • Datomic transactor — JVM process, single node, auto-restart via systemd.
  • pg_dump backup — nightly, retained 14 days.

Layout #

transactor/
  transactor.properties.template   merged with vault secrets at deploy
  datomic-transactor.service       systemd unit
postgres/
  init.sql                         creates datomic db + role
  backup.fish                      pg_dump wrapper
backup.cron                        cron entry (run nightly)
deploy.fish                        installs/updates transactor + pg config

One-time: add silo env vars #

The silo dashboard needs to know where the sidecar lives and what admin secret to present. The sidecar's ADMIN_SECRET was generated into aesthetic-computer-vault/kidlisp-datomic/sidecar.env.gpg. Copy that value into silo's existing env:

cd /workspaces/aesthetic-computer/aesthetic-computer-vault
fish vault-tool.fish unlock
# ADMIN_SECRET=<hex>   ← read from kidlisp-datomic/sidecar.env
# Append two lines to silo/.env:
#   DATOMIC_SIDECAR_URL=http://127.0.0.1:8891
#   DATOMIC_SIDECAR_ADMIN_SECRET=<hex>
fish vault-tool.fish edit silo/.env
fish vault-tool.fish lock

Silo proxies all /api/datomic/* requests to the sidecar with this header. Sidecar binds to 127.0.0.1 on silo, so the URL is internal-only.

End-to-end bring-up sequence #

Run each step from your local terminal (not via Claude tool calls — gpg + ssh keys need interactive agent access).

# 1. Unlock vault
fish aesthetic-computer-vault/vault-tool.fish unlock

# 2. Append the two sidecar env vars to silo/.env
fish silo/datomic/update-silo-env.fish

# 3. Install JVM, Postgres, create datomic user + dirs on silo
fish silo/datomic/bootstrap-silo.fish

# 4. Download Datomic Pro (Apache 2.0) and ship the jar to silo
#    The bootstrap script prints exact commands if the jar is absent.

# 5. Deploy transactor config + systemd + backup cron
fish silo/datomic/deploy.fish

# 6. Build the Clojure sidecar uberjar and ship it
fish kidlisp-sidecar/deploy.fish

# 7. Redeploy silo to pick up the new env vars
fish silo/deploy.fish

# 8. Backfill Mongo kidlisp → Datomic (from silo, since sidecar is localhost)
ssh -i aesthetic-computer-vault/home/.ssh/id_rsa root@silo.aesthetic.computer \
  "cd /opt/silo && CLIENT_SECRET=\$(grep CLIENT_SECRET /opt/kidlisp-sidecar/.env | cut -d= -f2) \
    node /opt/ac/system/backend/backfill-kidlisp-to-datomic.mjs"

# 9. Verify the dashboard → datomic tab shows healthy transactor,
#    schema installed, entity counts matching Mongo.

# 10. Flip the feature flag to cut over.
#     Edit aesthetic-computer-vault/lith/.env and add:  KIDLISP_DATOMIC=on
#     Then:  cd lith && fish deploy.fish
#     After this, store-kidlisp.mjs routes all traffic to Datomic.

# 11. Lock the vault when done
fish aesthetic-computer-vault/vault-tool.fish lock

Individual runbook steps (reference) #

  • First-time software install on silo: handled by bootstrap-silo.fish (apt install temurin-21-jdk + postgresql, create datomic user, mkdir tree, create PG role + db, verify jar present).
  • Datomic jar acquisition: not scripted — operator downloads datomic-pro-<version>.zip from https://docs.datomic.com/pro/ and scp's to /tmp/ on silo. Bootstrap script prints the exact unzip/symlink commands.
  • Config rollout: deploy.fish renders transactor.properties.template in /dev/shm, uploads to silo, restarts transactor via systemd.
  • Sidecar rollout: kidlisp-sidecar/deploy.fish builds uberjar locally, decrypts sidecar.env.gpg into /dev/shm, scp's both to silo, restarts.
  • Backup cron: installed at /etc/cron.d/datomic-backup by deploy.fish, runs 03:17 UTC daily, 14-day retention in /var/backups/datomic/.

Ongoing ops #

  • Logs: journalctl -u datomic-transactor -f
  • Restart: systemctl restart datomic-transactor
  • Backup now: fish /opt/datomic/backup.fish
  • Restore: stop transactor, psql the latest dump into a fresh db, restart.

Non-goals #

  • No HA transactor in v1. Single node. If/when kidlisp traffic warrants it, Datomic Pro supports standby transactors with automatic failover.
  • Not moving moods/paintings/handles/chat here — see root CLAUDE.md and the v1 plan.