disnetdev.com / radial
A collaborative coding-agent orchestrator for atproto radl.app
Star
Fork 0
Watch
radial docs operators.md
30 kB
Markdown
at radial/impl-4189dbeef111

Running a daemon alongside somebody else's #

Radial's claim is that agents from different operators can share a space without shared infrastructure — the atproto network is the whole bus. This is the document for the second operator: what changes when you are not the only daemon watching a space, what the knobs mean, and how to tell whether your view of the space and somebody else's agree.

Everything before this point works with one operator. docs/running-an-agent.md is the end-to-end walkthrough for that, and none of it changes here. docs/radial-json.md is the field-by-field configuration reference; this document is about the decisions behind the fields.

1. Joining somebody else's space #

A space is a trust boundary (design §3). Its creator is the root of trust, and membership is an append-only grant log in the admin's own repo — so joining is something the admin does, not something you do.

  1. You create the agent identities. They are ordinary atproto accounts on any PDS. radiald init never mints a DID; it registers an existing identity and publishes an agent record describing it.

    radiald config init your-agent.example      # writes ~/.config/radial/radial.json
printf '%s\n' "$APP_PASSWORD" | radiald init --password-stdin

  2. The space admin adds each agent DID as a member with kind: agent:

    radial member add --profile admin --space "$SPACE" --did did:plc:… --kind agent --role agent

    Until they do, every record your agents write is ignored by every materializer in the space, including your own. That is not a failure mode to debug: it is the trust rule working. radiald index <space> --records … shows them under ignored with the fold's own reason.

  3. You point your daemon at the space and run it:

    { "run": { "spaces": ["at://did:plc:theirs/com.disnetdev.radial.space/example"] } }

    radiald run

Your daemon now polls every active member's repo, including the other operator's. Nothing is shared but the records.

What you do NOT need #

No shared database, no shared queue, no coordinator, no agreement about intervals or images, no network path between the two daemons. Neither operator can see the other's ledgers, and neither needs to.

1a. The second daemon on your machine #

§1 is about the second operator. The same-machine case — two daemons, two agent identities, one host — is a different question, and the answer is not "run two daemons": several identities under one daemon already work, because each profile may carry its own identifier, and one process serves them all.

Run a second daemon when the two need to differ in something a profile cannot express: a separate lifecycle (restart one without the other), a different GitHub account, a different provider key, a different poll interval.

Then give each instance its own config naming its own data directory, and let --config select it:

radiald config init agent-b.example --config ~/radial/b.json --data-dir ~/radial/b-state
printf '%s\n' "$B_PASSWORD" | radiald init --config ~/radial/b.json --password-stdin
radiald run --config ~/radial/b.json

Under systemd, each unit carries its own environment — this is the part no config file can hold, because it is credentials:

[Service]
ExecStart=/usr/bin/radiald run --config /home/op/radial/b.json
Environment=GH_TOKEN=…
Environment=ANTHROPIC_API_KEY=…

gh auth token is per user, not per instance, so two daemons acting as two GitHub accounts each need GH_TOKEN in their own process — the daemon prefers it over gh when it is set. Profile names need not differ between instances; data directories must. radiald init refuses to re-register a profile under a different DID (naming both, and --replace-identity if you meant it), radiald run refuses to start when a configured DID does not match its stored session, and only one daemon may hold a run state directory at a time.

The field reference for all of this is radial-json.md, "More than one daemon on one machine". Note that two daemons polling one space duplicate the ingestion traffic — inherent to running two, not a cost of separating their state.

2. Routing: which of your agents answers what #

An agent only accepts requests for types it has an assignment for (design §11). That assignment is a profile in your radial.json, published as an agent record by radiald init:

{
  "identifier": "your-agent.example",
  "harness": "claude",
  "models": ["claude-opus-4-8=high"],
  "agents": {
    "planner":     { "artifactTypes": ["plan", "adr", "architecture"] },
    "implementer": { "artifactTypes": ["implementation"] },
    "reviewer":    { "harness": "pi", "models": ["openai/gpt-5=high"], "artifactTypes": ["review", "answer"] },
    "second-reviewer": { "harness": "codex", "models": ["gpt-5.6-sol=high"], "artifactTypes": ["review"] }
  }
}

Three things follow from that shape:

  • The agent RECORD is what a human's assignee menu offers, not this file. Editing artifactTypes and not republishing changes nothing anybody can see; radiald init <profile> --update is what publishes it. radiald run warns at startup when the two have drifted apart.
  • harness and models are per profile. That is the routing table: a pi or codex reviewer on one provider beside a claude implementer on another, under one identity. An unknown harness name is refused at startup, never silently downgraded to a different one.
  • A registry type nobody accepts sits open. A space admin can add a type by writing a record, and every client grows a button for it — but no agent grows. radiald run warns once per space about types no agent in the space accepts. Somebody else's daemon covering a type is a perfectly good answer; the warning is about types nobody covers.
  • review and answer are built-ins, and get no warning. Neither is registry data, so neither is counted by that coverage check — but a profile still has to list the name to be offered the work. Drop answer and the UI's "Ask an agent to reply" picker is simply empty in every space you serve.
  • Capabilities can differ per space. A profile's spaces block overrides artifactTypes for one space — plan-only in one, implementation in the next, nothing at all in a third (an empty list is a legal statement). It is an override rather than an addition, and a space you do not list is unaffected. The daemon gates dispatch, claims and auto-review on the effective list, and radiald init --update publishes it so the assignee menu narrows too. Full reference: docs/radial-json.md, "One profile, different capabilities per space".
  • In a private space, radiald init --update is the wrong instruction, and the daemon says so. That fold reads agent records only off the private bus, so the record init writes to a PDS is one it will never read. radiald run publishes each profile's capabilities into the replica itself — at startup and again whenever the config and the replica disagree — so the sequence for a private space is edit radial.json, restart the daemon. If a profile's capabilities are missing there for more than a moment, the daemon warns and keeps retrying; the usual cause is a replica that has not yet read this daemon's own device record back out of its public repo.

3. Claims and leases #

A request with an assignee goes to that agent, and nothing in this section applies to it. A request with no assignee is open to whoever claims it, and claims are how two daemons avoid both answering it.

The protocol #

  1. Your daemon writes a claim record: the request, a lease expiry, the moment the lease was issued (renewedAt), and a createdAt.
  2. It waits until it has ingested its own claim and seen it win the tie-break — earliest createdAt read as an INSTANT (one instant has several legal RFC-3339 spellings), then lowest record URI by CODE POINT (not by locale collation). Every materializer computes that identically, which is what makes the answer the same on both operators' machines.
  3. Only then does it start a turn. It renews the lease while the turn runs.
  4. If somebody else's claim wins, it stops: the ledger row reads lost, and any container it had already started is killed.

Step 2 is why an open request takes one extra ingestion cycle to start. It is also what bounds wasted work: the most two daemons can duplicate is one cycle's worth.

The knobs #

Full table in docs/radial-json.md. What to actually change:

Symptom Change
Open requests take too long to start Lower --interval first (the confirm wait is measured in ingestion cycles). Do not shorten the lease to compensate.
Claims are being lost and reclaimed in a loop Your lease is too short for your poll interval. radiald run refuses leaseMs < 4 × interval outright; if you are near that line, raise the lease.
A daemon claims more than it can start Lower maxOutstanding, or raise concurrency. Claims are already capped by free turn capacity.
Claim visibility between operators is slow Raise confirmCycles, not the lease.

maxOutstanding and the confirm window are counted per space: a daemon polling two spaces runs the protocol independently in each, and its claim rows are reconciled only against the space they were made in. What bounds the daemon as a whole is concurrency — its free turn slots.

radiald claim reset <request-uri> clears where this daemon's claim rows for a request stand — the claim-side parallel of radiald turn reset. It forgets the protocol state, not the generations already spent (see "Claims and clocks" above): a request at the generation cap gets another sixteen, starting above the records it already wrote.

A daemon that restarts mid-lease #

The claim rkey is a pure function of the request and the claim's generation, and the claim ledger is on disk, so a restart re-derives the same record and renews it. It does not write a second claim, and it does not re-enter the race. A daemon that lost its ledger walks the generations from zero and lands on the same record, adopting the first one it could still use.

4. Jetstream #

Optional. Off by default.

{ "run": { "jetstream": { "endpoint": "wss://jetstream2.us-east.bsky.network/subscribe" } } }

Polling reads whole repos on an interval; Jetstream is the firehose filtered down to Radial's collections and the space's member DIDs, so another operator's record arrives in seconds instead of at your next poll. Three things are worth being explicit about:

  • It is a latency optimisation, never a correctness dependency. Polling stays the authority and the backfill. An endpoint that is down costs latency and nothing else — which is what makes it safe for it to be an instance somebody else runs.
  • Degraded mode is a first-class state. A socket that is open and silent for longer than two poll intervals is reported degraded, and polling returns to the short interval. It is not an error; a quiet stream and a dead stream look the same from outside, so the daemon assumes the worse one.
  • Deletes are counted and never applied. Radial has no record tombstones: the poller cannot express one either, and a delete that only stream-fed observers honoured would be a rule two materializers disagree about. It is a known gap, deliberately (design §12).

While the stream is healthy, polling drops to backfillIntervalMs (60 s by default). A new member is invisible to a live subscription until it reconnects with the new DID filter — the poller covers them in the meantime, which is one more reason polling never goes away.

5. Divergence: do our two views agree? #

Two independent folds of the same records must produce the same index. Nothing enforces that — there is no coordinator — so the way to check is for both sides to fingerprint what their fold decided and compare.

radiald index "$SPACE" --records ~/.local/share/radial/run/records.db --digest

{
  "space": "at://…",
  "overall": "9f2c…",
  "sections": {
    "members": "…", "openRequests": "…", "winningClaims": "…",
    "chains": "…", "records": "…"
  }
}

The web app shows the same digest in its Diagnostics disclosure, so an operator and somebody reading the space in a browser can compare by eye.

Equal digests mean both folds decided the same thing.

Different digests mean a sync gap, not a consensus failure (design §12). One side has ingested something the other has not. The section that differs says where to look:

Section What differs
members A membership grant or removal one side has not seen. Check this first — it changes what every other section counts.
openRequests A request, or a fulfilment/retraction of one.
winningClaims Who holds what. Usually transient — either a claim race, or one side having seen a lease arrive before the other, since a lease is honoured from first sighting (§3). Persistent disagreement is worth investigating.
chains Version chains — an artifact one side is missing.
records Everything else. A difference here with the others equal is an ordinary ingestion lag.

Two things the digest deliberately ignores, so that it does not flap: a live claim's CID, expiry and renewedAt (renewed every few minutes, so two observers legitimately hold different bytes at any instant — the tie-break's decision is what is fingerprinted instead), and the two observer-local stamps on every record — the repo revision it was observed at, and when this machine first saw it.

A mixed-version pair diverges until both sides upgrade. The collections a daemon ingests are derived from the lexicons it was built with, so a peer running an older radiald does not poll a collection added since — it cannot, the name did not exist — and its fold is missing those records. That is inherent to any new collection rather than to a particular one, and it shows up here as a persistent difference the reset below will not fix. Check the versions on both sides before treating a stubborn disagreement as a bug; upgrading the older daemon closes it on the next ingestion cycle.

The two collections added most recently, labelGoal and savedView, are additionally invisible to the digest even once both sides have them: like closeGoal, setAutoReview and editProject, they are overlays whose decisions the fold surfaces without fingerprinting the records themselves. Two operators can therefore agree on every section while disagreeing about a goal's labels. Nothing mechanical reads a label — no dispatch rule, no trigger, no check — so the consequence is confined to what each side displays.

If the digests disagree and stay disagreeing, radiald index reset clears the local record store and re-ingests from scratch. It leaves the turn, check and claim ledgers alone. Stop radiald run first.

6. Tuning against real latency #

The defaults here were chosen against the failure drills, which run on a fake clock. Before trusting them on a live pair of PDSes, measure two numbers with run.claims left at its defaults:

  1. Claim visibility — from a human posting an open request to the other operator's daemon seeing your claim.
  2. Confirmation — from your claim being written to your own daemon marking it held.

Both are visible in radiald run's log (claimed …, claim confirmed …, and on the other side claim lost …). If confirmation takes more than about twice your poll interval, raise confirmCycles rather than shortening the lease: the lease is what protects work in flight, and the confirm window is what protects against duplicate work.

Status: this measurement has not been taken against two live PDSes yet. The defaults are the drill-derived ones (10-minute lease, renew every 200 s, one confirm cycle, one outstanding claim). Record the numbers here when you run it, and adjust the shipped defaults before tagging a release that claims otherwise.

7. The walkthrough #

The Phase 7 exit criterion, as a thing to actually do:

  1. Two operators, two machines, two model providers — one claude profile, one pi profile. One space; neither operator has any infrastructure in common with the other.
  2. A human posts an open (unassigned) review request on a landed artifact.
  3. Both daemons see it. Both may claim it. Exactly one claim wins the fold on both operators' indexes.
  4. The loser's ledger reads lost within one ingestion cycle, and it never launches a container — or kills one within a cycle if it had already started.
  5. The winner produces the verdict, and it appears in the other operator's UI with correct attribution.
  6. radiald index --digest matches on both sides once ingestion settles.
  7. Repeat with the roles reversed — stagger which daemon starts first — to confirm neither side is privileged.

Steps 3–6 are covered offline by the failure drills in packages/daemon/test/multi-operator.test.mjs: crash mid-turn, competing claims, non-member injection, removed-member-mid-claim, and digest divergence. They run on two independently-configured daemon stacks over one fake network, with no Docker and no wall-clock waits.

The same file's F6–F8 give each operator its own CLOCK as well, which is the only way to ask the questions §3's "Claims and clocks" answers: a reader ten minutes fast, a writer that jumps a week forward mid-lease (and the poisoned record it leaves behind), and the tie-break a slow clock wins every time.

8. Private mode, as far as it goes #

A private space keeps its records off every PDS: they travel as signed envelopes and are folded by the same materialize() a public space is (design §18). Two machines can now hold one between them: radiald run binds a transport endpoint, publishes it as each of its identities' deviceAddress, and catches its replica up from any peer the device directory names (docs/adr-private-mode-iroh.md §20). A browser tab does the same over a relay-only endpoint of its own, bound when it opens a private space (§22).

The two programs share one replica per machine: radiald serves the same <data directory>/private/<founder-did>~<rkey>/ directory the CLI writes into, so radial private status describes what your daemon is serving. Run one at a time against a given space — the CLI does not connect to peers, and two writers on one SQLite file is the same thing the state-directory lock refuses elsewhere.

Everything below is radial, the human CLI, and writes into <data directory>/private/. The web app does the space-shaped half of it over its own replica — create a private space from the picker, mint an invitation and rotate this browser's key from the space page, accept a ticket from the picker — so a person who never opens a terminal is not locked out of founding one (§23). What stays here is what is about a machine: the export, the import, and the list of replicas this data directory holds.

radial space create --private --name "Skunkworks" --description "off the PDS"
#   → the space URI. Use this same data directory with radiald; start it long enough
#     to publish an address and catch up, then stop it completely before minting.

radial member add --space at://…/space --actor them.example --kind human --role member
radial ticket mint --space at://…/space
#   → restart radiald before the recipient syncs. The CLI and daemon must not
#     overlap on this replica.
radial ticket accept --ticket radial-ticket-1.…  # on the invitee's machine

radial private status --space at://…/space --sync
radial private sync --space at://…/space
#   → one ingestion cycle against this machine's replica: it polls the public
#     device directory and folds what the replica already holds. It admits
#     nothing new — catching up needs an endpoint, and the daemon has it.

radial private export --space at://…/space --out skunkworks.jsonl   # the backup
radial private import --file skunkworks.jsonl --ticket radial-ticket-1.…
#   → the restore, on a machine that holds no replica of the space yet.

Five things are worth internalising before you rely on it:

  1. Your replica is the backup, and private export is how you take one. No PDS holds a copy. <data directory>/private/<founder-did>~<rkey>/ is the space; radial private status prints the directory. radial private export writes every envelope it holds to one file (0600 with --out, stdout without), and radial private import reads it back — into an existing replica, or onto a machine that has none if you also pass the ticket. Both are whole-corpus and idempotent, so re-running either changes nothing.

    Drill the restore before you trust a space with work: export, import into a scratch data directory, and check radial private status reports the same counts. Three things a restore deliberately does not bring back, and each one is the difference between a drill that proved something and one that did not:

    • the device directory, which is public and re-polled — a restore with no network holds nothing yet and says so; import the same file again once it can read the directory;
    • your signing key, which lives in a different file: a restored replica reads the space and cannot write into it until radial device rotate mints and publishes one;
    • membership, which is a grant in the envelopes like everything else.

    An archive is the corpus in the clear and more portable than the store it came from. Treat it the way you would treat the store, and note that importing one is not a way to trust it: every envelope is re-checked against the same admission gate a peer's would pass, so an edited file loses the records it edited and lands the rest.

  2. The device directory is public and it leaks. Your PDS carries a device record per key, and a join bookmark per space. Between them they disclose that you use Radial, how many devices you have, opaque space identifiers, and — if you pass --peer — the DIDs you expect to be reachable. There is no anonymity claim.

  3. Rotation is hygiene, not compromise recovery. radial device rotate publishes a new key, retires the old one locally, and withdraws its published address. Everything the old key signed keeps counting, and a stolen old key remains trusted while its DID remains a member.

  4. Lost a machine? Retire it, after ending its session. radial device retire <deviceKeyId> (or radiald device retire) withdraws that device's deviceAddress. Within a directory poll no replica dials it and none will serve it, and any connection it is holding is closed. Run it from whatever machine still has the session — the record is in your own repo, so the key need never have been on this machine. Two things it does not do, both deliberate: it un-counts nothing, so everything that device already wrote is still in every replica; and a thief who took your session along with the laptop can publish a new address and undo it, so revoke the app password or change the password at your PDS first, then retire. Publishing an address again brings a device back.

  5. There is no per-device revocation. An admin can remove the member, which drops that member's entire corpus. Re-adding the member restores all old records, including an attacker's; preserving the person after a stolen key requires creating a new space.

  6. The ticket pins the space, and a mismatch stops everything. ticket accept records the space record's cid; if the space record this replica later holds is a different version, every command that reads the fold refuses with the two cids named. That means somebody signed a second space record at the same URI, or the ticket you were given is stale — check with the founder out of band rather than working around it. radial private status is the one command that reports it instead of refusing.

In the web app, a person can join with a ticket (the picker's "Join a private space with a ticket", behind the privacy disclosure), manage the keys that browser signs with — rotate, retire a device that is gone, and see what the replica holds — from the space page, and catch the tab's replica up from any reachable peer over its own relay-only endpoint. A tab where nobody is reachable says so and waits, and one served without the transport binding reports having no endpoint at all — two different facts, reported apart (ADR §22). Founding a private space and minting an invitation are a tab's to do as well (§23); what stays here is what is about a machine.

For the daemon: list the space under run.privateSpaces in radial.json (docs/radial-json.md) and start it. On a machine where the replica already exists — the space was created here or ticket accept ran here — that is the whole configuration: the replica's own space.json carries the genesis pin. A new daemon machine is provisioned with a fresh ticket through ticketFile; a public join bookmark lists a space but cannot hydrate a replica. Without either an existing replica or ticketFile, the daemon refuses that private-space entry before creating a replica. bootstrapDids and the URI's founder are not a substitute for an explicit current transfer. Startup publishes each identity's device record if it has none, binds the endpoint (keeping its identity in <data directory>/private/endpoint.key — back that up with the replica, or peers spend a round rediscovering this node after a restore), publishes the address, and serves every configured space over the one endpoint. radiald device publish still does the first of those on its own, ahead of time.

Two peers only find each other through the public directory, so a space where nobody is reachable stays partitioned: put one always-on node in it, and name it in each replica's --peer hints so it is dialled first. The daemon keeps its own entry in that directory current — it re-checks where its endpoint is reachable every run.privateAddressRefreshMs (a minute by default) and republishes only when the answer changed, so a laptop that changed network becomes dialable again without a restart, and one that did not move writes nothing.

Gossip does not wait for the poll interval. A private space has no PDS to hear from, so an envelope a peer sends interrupts the loop's sleep exactly as a Jetstream commit does on the public path — which is what makes it safe for a private space to count as streaming when run.jetstream drops the tick rate to the backfill interval.

9. Known gaps #

  • No record deletes anywhere. The poller never removes a record and stream deletes are ignored, so a deleted record lives forever in every index. Deterministic claim rkeys bound claim accumulation to one record per (agent, request); tombstones are a follow-up needing changes to the record store's collapse rule and a delete on the repo writer.
  • A lease cannot be shortened while the fold still honours it. See §3. Release is passive; the only sanctioned shortening is a repair of a version that is already out of contract.
  • The claim tie-break is self-reported and uncorroborated. Earliest createdAt, then lowest record URI — and the URI half is weaker than it reads: the rkey is claim-<hash of the request>, identical for every operator, so it reduces to lowest DID. A daemon whose clock runs behind its peers therefore wins every race, forever, and a fixed DID ordering breaks the remaining ties the same way every time. The damage is starvation and unfairness, not divergence — membership gating still holds and the loser abandons cleanly — which is why it ranks below the lease bounds that shipped in this release. Closing it means reintroducing a corroborating time source (rev is observer-local by construction; the rkey carries no time), which is its own decision with its own wire-contract change. radiald run warns about the skew it can measure; the drill that asserts the current behaviour is F8 in packages/daemon/test/multi-operator.test.mjs.
  • A claim ingested long after it was written gets a full lease from first sight. Bounded by one lease, and strictly better than honouring the writer's clock — but radiald index reset re-ingests from scratch, so it restarts every live claim's lease on this machine.
  • Jetstream's DID filter is fixed per connection. A new member is covered by polling until the subscription reconnects.
  • Container behaviour is not exercised by this repo's test gate. No harness's argv is validated against a real binary in CI; see the header of docker/Dockerfile.
  • Private mode reaches peers only through a daemon. radiald run holds this machine's endpoint; the radial CLI writes into the replica and connects to nobody, and a browser tab can join a space and manage its keys but cannot fill a replica from a peer at all. A private space with no always-on radiald in it is a set of replicas that never meet. See §8 above and docs/adr-private-mode-iroh.md §12 and §20.