The write pipeline #

What a write passes through before it reaches the log, in order, and what each stage costs, checks and answers.

The order is not arbitrary. It runs broadest and cheapest first, so a write that cannot possibly succeed is refused before anything expensive happens, and every stage's refusal is distinguishable on the wire — an agent retries differently for each, and one that cannot tell them apart retries the wrong thing forever.

flowchart TD
    R[["write request"]] --> E

    E{{"1 · e-stop<br/>Estop::check_use / check_issue"}}
    E -->|"Revoke, or Pause on issuance"| EH["503 Halted"]
    E -->|clear, or Pause on a record write| L

    L{{"2 · lifecycle<br/>ServerState policy"}}
    L -->|"not accepting writes"| LH["503 ServerNotReady<br/>Retry-After, blocking gates named"]
    L -->|accepting| F

    F{{"3 · per-repo freeze<br/>Provisioner::require_writable"}}
    F -->|"AccountState refuses writes"| FH["403 account frozen<br/>reason: admin | lifetime | policy"]
    F -->|writable| P

    P{{"4 · parsing<br/>record + diff"}}
    P -->|malformed| PH["400 InvalidRecord"]
    P -->|"parsed, diff computed"| Q

    Q{{"5 · per-repo queue<br/>admission order"}}
    Q -->|full| QH["429 / 503 backpressure"]
    Q -->|queued| T

    T{{"6 · policy tree<br/>applicability index"}}
    T -->|"no policy matches"| W
    T -->|"candidates, in order"| V

    V{{"7 · evaluation<br/>Evaluator::evaluate"}}
    V -->|"Reject"| VH["403 policy refused<br/>policy-authored reason"]
    V -->|"Freeze"| VF["403 account frozen<br/>drains this repo's queue"]
    V -->|"unavailable / over its deadline"| VU["403 refused<br/>fail closed, scoped to this policy"]
    V -->|Allow| W

    W(["8 · commit<br/>store lock held here only"]) --> S["sequence assigned<br/>policy version + now recorded"]

What each stage checks #

1 · E-stop — didbot_pds::Estop #

The cheapest check there is: an atomic latch read, no credential, no lookup. It runs first because it is both cheapest and most urgent, and because Halted is the wire answer callers already switch on.

It is not one gate. The mode and the operation class decide together:

	Mode::Pause	Mode::Revoke
record write to an existing repo (check_use)	proceeds	halted
provisioning a new account (check_issue)	halted	halted

Pause blocks issuance and leaves outstanding work alone by definition, which is why a lapsed operator claim throws Pause and not Revoke: an operator going unreachable must not stop the agents already here from writing.

E-stop and the lifecycle are independent facts. A server can be fully claimed and halted, and neither is derivable from the other.

2 · Lifecycle — didbot_pds::ServerState #

Whether this deployment accepts writes at all. Public: no credential is spent to learn it, because "this server is not accepting writes right now" is not a secret and paying for a credential lookup to reach it is work for nothing.

Answers 503 ServerNotReady with Retry-After and the blocking gates named, never a 404 — "this method does not exist" and "this method is not ready" are facts a relay reacts to completely differently.

3 · Per-repo freeze — Provisioner::require_writable #

AccountState's policy for the named repository. Cheap, and needs no authentication for a reason worth stating: the repository identifier is caller-supplied, but the only thing a caller achieves by supplying one is getting itself refused. A check that can only deny is safe on unauthenticated input. Trusting caller-supplied input is dangerous when it grants and harmless when it only denies.

A freeze carries why — an administrator, a lifetime rule, or a policy — and that reason is distinct from a policy refusal. "Account frozen" says nothing you write will work until an operator acts; "policy refused" says fix this write. An agent that cannot tell them apart retries the wrong one.

4 · Parsing #

Necessarily before policy: a diff cannot be computed without parsing, and policies are evaluated against the diff.

The diff carries before and after values, not only changed paths, with absence treated as a value. That is what lets a rule resolve create, update and delete uniformly — before.is_some() && after != before — with no action special-case anywhere.

5 · Per-repo queue #

The linearization unit is the repository, not the server. Order within one repository is load-bearing: its commits chain, each naming its predecessor. Order between two repositories is observable by nobody. So each repository has its own queue, different repositories evaluate concurrently, and head-of-line blocking is bounded to the agent that caused it.

The queue is bounded. A full queue is backpressure a caller can act on, not a memory leak.

6 · Policy tree — applicability index #

The tree indexes; what comes out of it is an ordered list. Candidates are selected on collection, path, action and subject kind. A write matching nothing pays only for the walk it already owed, because the diff it needed for indexing is the diff it needed anyway.

Selecting on kind at the tree is what keeps a policy scoped to agents from ever seeing a host's write.

7 · Evaluation #

Policies only deny. Any deny from any source is sufficient, so adding a policy can never widen what is permitted. That monotonicity is what makes the rest coherent: order is presentational rather than semantic, precedence between sources needs no rule, and failing closed is strictly conservative — a refusal on an unavailable evaluator is guaranteed to be at least as restrictive as a complete evaluation would have been.

Failing closed is scoped by the tree: if the evaluator serving one policy is down, only writes that policy would have judged are refused. Everything else is untouched.

Freeze is a transition on the repository's queue rather than a message sent beside it. The write that trips it drains that repository's pending writes with a rejection naming the freeze — which is also why a metapolicy that freezes agents for tripping policies never observes the denials its own freeze caused: those writes are drained without evaluation. That safety property falls out of the per-repo queue, and parallelising the drain would silently reintroduce the loop.

8 · Commit #

The only stage that holds the store lock. Evaluation happens outside it, so a slow policy never holds the single writer, and a denied write never needs un-committing — which the write-ahead log cannot do cheaply.

The sequence number is assigned at admission, not arrival. The policy version and the now that judged the write are recorded with it, because decisions are not reproducible — an evaluator may have a model in the loop, and an agent may delete data an evaluation read.

Where authentication sits #

Not as a stage of its own, and deliberately.

Verifying a token cryptographically is mechanism, and cheap. Deciding whether a token may be used is a policy question and belongs at stage 7: a token issued before its application was denied is still perfectly valid and still names a real account, and only a write-tree rule that can see client_id catches it. That is why client_id is on the write subject and not only on the grant subject.

So nothing before stage 7 spends a credential, and nothing before stage 7 needs to.

Shortcuts #

• Stages 1–3 short-circuit before any parsing. A halted server, a server not yet accepting writes, and a frozen account are all decided without reading the request body.
• Stage 6 short-circuits on an empty match, which is the common case.
• Stage 7 short-circuits the decision channel — once a Freeze is reached, evaluation stops. The observation channel does not short-circuit: every evaluator watching a matching surface sees every attempt and its outcome, including attempts an earlier policy already denied, because a metapolicy counting policy failures must see denials it did not cause.
• A freeze drains the rest of that repository's queue without evaluating any of it.

What is recorded #

• An allow costs a policy version and a hash of the matched set. A row per admitted write is where the volume is and buys nothing.
• A denial records an evaluation id, the policy version, which policies fired, the verdict, a hash of the payload, and a policy-authored reason — and no part of the refused payload. The refused data is by definition what a policy decided should not exist here; storing it would put it durably on disk in a log with its own retention, written by the mechanism meant to prevent it.
• Detail flows to the party that already has it: the rejection returned to the caller may be as specific as the policy likes, because the caller sent the data.