id: local-dev
title: The whole stack runs on one machine, with traffic that looks real
status: continuous
crates: [didbot-swarm, didbot-serve]
dependsOn: []
exitCriterion: >
None. A development workflow is never finished; it is worked whenever
something adjacent to it is open. #
Five terminals, each script finding the others by default, and a swarm that
drives the stack with simulated agents. This is what makes every other epic
workable without a deployment.
A one-crate incremental rebuild of the development server is about two
seconds. A full dev-pds.sh cycle after a change to didbot-pds is
about fourteen, of which four is running that half's tests and the rest is
compiling and linking. cargo install of the hook binary reuses what the
tests just built and costs a second or so, which is what its comment in the
script claims.
So the compile is not where the time goes. What the build costs is disk, and
that is what fills a machine: see the dev profile under Done.
Correction, not new work: three items here were ticked against code that
has left this repository — didbot-replay's firehose capture and playback
(no such binary, crate or script exists now), a canvas mock, and the
index/query/web dev scripts. The index, the query service and the canvas moved
to vibescrobble.com; the replay tool went with them. They are unticked rather
than left standing. The surviving scripts are described accurately below.
One clone shared by every checkout has a cost worth naming: a plain run
that fast-forwarded it would move the corpus under whatever else is
reading it. So a corpus already there is left alone and reported, and
`--update` is what changes it.
It took three sightings to catch, because the first two were seen through
a `cargo test` whose output had been sent to `/dev/null` — which is its
own lesson about checking exit statuses without keeping what produced
them.
The personal data server refuses. The others hold nothing and a restart
costs the build; that one holds every account, record and name hold, and
without `--data` all of it is in memory, so a watch there would factory
reset the deployment on every save and the first symptom would be
sessions whose accounts had silently stopped existing. It says so and
names the two ways to mean it.
Polling rather than `inotifywait`, which is not on every machine this
runs on, and a second of latency against a rebuild that takes fourteen is
not worth a dependency that has to be installed first.
Two things had to be got right and were not, first time. The stop has to
tolerate its own signal: these scripts run under `set -e`, and a `wait`
on a process the loop just killed reports the signal, which took the
watcher down with the thing it was restarting — a watcher that worked
exactly once. And the pidfile's cleanup trap was dead code from the day
it was written, because `exec` replaces the shell and takes the trap with
it; a stale pidfile is harmless and is now documented as such rather than
pretended about.
The stamp is only worth something if somebody bumps it, so the durability
suite pins the on-disk shape against the number — the variants read out
of the source, and the fields read off a log the test writes and decodes
by hand rather than with the crate's own reader. A field rename fails it
with the instruction to bump.
The saving arrives on the next `cargo clean`. Cargo keeps artefacts it
has superseded, so a directory that has been built in for a while holds
both shapes and briefly grows; the worktree this was written in had
reached 31 GB and rebuilt from clean, whole workspace and every test
binary, into 3.4 GB.
A torn log was already covered, in `didbot-pds`'s durability
suite: a write interrupted halfway costs that write and nothing before
it, and one torn inside the first entry replays to an empty deployment.
A session that dies without a `SessionEnd` is covered by the hook
harness.
