Policy evaluators #

The reference for the engines that judge a write: what one is, the six channels it answers on, the order they run in, where its state lives, and what happens when it fails. plan/policy.md is the reasoning behind this design; this page is what the code does. write-pipeline.md is where these stages sit in a write's journey.

Engines and policies #

An engine is a Rust implementation of the Evaluator trait (crates/didbot-policy/src/outcome.rs). It holds compiled statements and answers about subjects. It is the only part of policy that is Rust.

A policy is a bot.did.policy record (lexicons/bot/did/policy.json). Its document is a union member whose $type names the engine that reads it, and the statements inside it are that engine's own grammar. An operator writes one into their repository; the floor ships them in crates/didbot-policy-records/floor/. Both are read by the same code.

Three engines ship:

$type	Crate	What a statement is
bot.did.policy#regex	crates/didbot-policy-regex	an object whose kind says what it denies — a client, a path change, a changed value, a delete, an upload — with regular expressions for what to match
bot.did.policy#cedar	crates/didbot-policy-cedar	one forbid in Cedar text, carrying an @id and a @reason; the document's text holds all of them
bot.did.policy#count	crates/didbot-policy-count	an object naming the writes it counts, whether it counts each account or the whole server, its window and its limit

Each crate's own documentation holds its statement fields. A document of a $type no engine answers to is carried as far as the loader, which scopes it as unusable rather than dropping it — see Availability below.

The contract #

Six channels, all synchronous. No method is async, and crates/didbot-policy has no I/O and no clock of its own: an engine that needs either does it out of process, with the caller holding the deadline.

compile #

fn compile(&self, predicate: &serde_json::Value) -> CompileOutcome

Turns one statement into an artifact the engine can judge, and mints the CompiledId that names it. Three outcomes: compiled, compiled with warnings an operator should see, or rejected with a reason.

Called from the loader, once per statement per tree build (crates/didbot-policy-records/src/compile.rs), never from the write path. Validation is compilation: there is no separate validate step, because only the engine knows its own syntax. It must be idempotent — the same statement compiles again on the next build, and on an out-of-process engine's restart.

The caller keeps the id, never the artifact. An engine that holds one artifact per instance gets one instance per statement; an engine that keeps several must mint a distinct id per compile and dispatch on it (Language::new_evaluator, same file).

evaluate #

fn evaluate(&self, subject: &Subject<'_>, compiled: CompiledId)
    -> Result<Outcome, EvalError>

Judges one subject against one compiled statement: Outcome::Allow, Reject { reason } or Freeze { reason }, or an EvalError when it could not judge at all.

It may read only what Subject carries (crates/didbot-policy/src/subject.rs): the account and its attributes, the client, the collection, the action, the diff, the record before and after, the answers to the policy's declared lookups, and Universal, which holds this deployment's own DID, hostname and operator and the instant the decision runs at. It may not call a clock, reach a store, or fetch anything — the signature is the enforcement, since nothing reachable from its arguments can. What it reads of another repository was read before it was asked; see Lookups below.

The reason is the policy's own words. It is what the caller is told and what the evaluation log records, so it is the engine's job to say only what is safe to surface.

The tree short-circuits this channel: once a verdict cannot get more severe, the remaining candidates are not asked (crates/didbot-pds/src/policy_tree.rs, decide).

observe #

fn observe(&self, event: &Observed<'_>)

Every attempt on the policy's declared observation surface, with the subject, the compiled id and the outcome — whether or not evaluation short-circuited, and whether the write was admitted or refused. It runs after the decision, so a statement judging a count judges the count before this write and then counts it.

This is where an engine's state changes. evaluate takes &self and so does this, so an engine that accumulates does it here.

The write path calls it for every operation of a batch (crates/didbot-pds/src/writequeue.rs, submit_batch). Nothing after a freeze is judged or observed: the freeze drains that repository's queue.

lifecycle #

fn lifecycle(&self, event: &Lifecycle<'_>)

Frozen { account, reason }, Unfrozen { account }, Deleted { account }, delivered to every engine in the current tree (crates/didbot-pds/src/policy_tree.rs). It is how per-account state does not outlive its account: crates/didbot-pds/src/provision/lifecycle.rs calls deleted when an account goes.

An engine whose state caused a freeze must act on Unfrozen, or the freeze re-trips on the next write.

policy_lifecycle #

fn policy_lifecycle(&self, event: PolicyLifecycle) { }

Loaded, Superseded and Removed, per compiled artifact rather than per account, for an engine that keeps state per statement. It is defaulted to a no-op, so an engine that keeps none says nothing.

Delivery is deliberately unreliable: the tree decides what is routed anywhere, so an artifact nobody told the engine about costs memory and cannot change a verdict. Nothing here retries or acknowledges.

staleness #

fn staleness(&self) -> Staleness

Linearized for an engine whose answer is exact and cheap: it blocks rather than answer from a view that has fallen behind, so its lag bound is its latency bound. BestEffort { max_lag } for one that answers from what it has seen; a view further behind than max_lag is a health failure, and the engine reports it as EvalError::Unavailable, which the tree treats exactly as it treats a crash (crates/didbot-policy/src/outcome.rs).

All three engines here declare Linearized. Each policy carries the declaration too (ParsedPolicy::staleness), because one engine may one day serve statements of different kinds.

Lookups #

Not a channel of the trait: the answers arrive on the subject evaluate is handed. A policy record may declare lookups: records in other repositories its statements read, each under a name, with whose repositories to read (named is every account the written record names), the collection, and optionally a record key (#lookup in lexicons/bot/did/policy.json; Lookup in crates/didbot-policy-records/src/records.rs). They are read before the write is judged, never during it:

1. Lookups::of gathers what every policy in the build declares, and for_write answers which of them a write to one collection needs (crates/didbot-policy-records/src/lookups.rs). named_accounts says whose repositories a written record names (crates/didbot-policy-records/src/named.rs).
2. The route that accepted the write fills didbot-lookup's cache with those records, inside that crate's own deadlines, while it still has a runtime (Reader::prefetch in crates/didbot-serve/src/lookups.rs).
3. The write path reads the same cache, with no I/O, and attaches the answers to the subject as Subject::Write's lookups (WriteLookups in crates/didbot-pds/src/policy.rs). Each answer is a Lookup whose Found is the records, nothing, or unknown.

An engine reads an answer by the name the record gave it; the regex engine's denyLookup statement is one that does (crates/didbot-policy-regex). An unknown is never turned into a value: each statement says what an unknown means for it. The floor's declared-ai-preference record is the worked example.

The order, per write #

Matching the stages in write-pipeline.md:

1. On a poll tick, not on a write, the operator's records are listed and the whole tree is rebuilt: didbot_serve::policy_poll reads them, didbot_pds::policy_bindings::resolve splits every record into one declaration per statement per action, compile runs once for each, and the new tree is swapped into the gate. A write already being judged keeps the tree it pinned.
2. Lookups are read by the route that accepted the write, for the policies that could be asked about it — see Lookups above.
3. The subject is built by didbot_pds::policy::write_subject, which stamps Universal::now from the clock once, for this write, and carries the lookups' answers. Every engine reads the instant from there, and the evaluation log records the same one.
4. Stage 6, the applicability index: the tree walks the write's changed paths, collection, action and account kind, and produces the ordered list of candidate policies. A write matching none has paid only for the walk it already owed.
5. Stage 7, evaluate, in that order, skipping a policy no binding reaches this account with. Outcomes fold by severity, so the order decides only which reason is reported.
6. observe, over the observation surface of the same pinned tree, for every attempt.
7. The version: the write records the tree generation and the digest of the operator revision it was judged under (PolicyGate::version and judged_revision).
8. The evaluation log, for a denial only: the evaluation id, which policies fired, the verdict, a hash of the payload, the policy version, the instant, and the policy's own reason — never any part of the refused value (crates/didbot-pds/src/evaluation_log.rs).

Stages 6 and 7 run outside every store lock, so a slow engine holds none.

Where state lives #

An engine is constructed fresh for each statement on each tree build, and the tree is rebuilt on every poll tick. So state that must survive a tick cannot live in the engine.

didbot-policy-count is the worked example. Its counts live in one Counts per process (crates/didbot-policy-count/src/counts.rs), and each statement's count is held under the statement's own sorted JSON: a rebuilt engine compiling the same statement finds the count the last one left, and an edit that changes a limit keeps the count the old statement was keeping. Two statements written differently count for themselves, so an operator's own counting policy over the same surface can only refuse more.

The counts are memory only. A restart is a fresh window, for every account and for the deployment; that is an operator action the deployment records, and it is not reachable by the accounts the counts bound.

The bounds are declared rather than discovered: a ring of BUCKETS hourly counters and a head per account per statement, about 150 bytes with a DID for a key, and at most HELD of them — past that, counts whose whole ring has aged out are dropped. Counts for a deleted account go on lifecycle.

Availability #

Failure is scoped by the tree. An EvalError from evaluate, and a statement whose predicate could not be compiled at all (Predicate::Unusable), both refuse exactly the subjects that policy declared it covers, and nothing else: PolicyTree::decide turns each into a Reject on the spot (crates/didbot-policy/src/index.rs). A policy gating common writes therefore belongs in process and cheap, and anything slower belongs on a narrow branch, because everything under that branch shares its availability.

This is strictly conservative only because policies deny and never permit: a refusal on an engine that could not answer is at least as restrictive as the full evaluation would have been.

Outcome::Freeze is the third answer the contract allows: it drains that repository's queue and needs an operator to lift it. Every floor statement refuses the write and nothing more.

The floor and an operator's records #

Both are bot.did.policy records and both go through the same build (crates/didbot-policy-records/src/merge.rs, build): the floor is source 1, the startup-supplied set is source 2, the operator's records are source 3, and every one of them is read by the same reader, split by the same declarations, and compiled against the same engines by crates/didbot-policy-records/src/compile.rs. The floor's records are in crates/didbot-policy-records/floor/, with README.md beside them saying what each is for and where its numbers come from.

All three sources deny, so their order decides which reason is reported first and nothing else. An operator's record can narrow what the floor leaves and cannot widen it.

A built-in registers under builtin/{name}/{statement}/{action} (crates/didbot-policy-records/src/builtin.rs). That is not an at-uri, and bot.did.policyBinding.policies holds at-uris into the operator's own repository, so a binding has nothing to name: the floor is unaddressable by construction rather than by a check. crates/didbot-pds/tests/floor.rs holds that.

Adding an engine #

1. A crate beside the others, didbot-policy-<name>, implementing Evaluator.
2. A union member in lexicons/bot/did/policy.json and the matching arm in crates/didbot-policy-records/src/records.rs, so the reader carries the document.
3. A Split arm in crates/didbot-policy-records/src/set.rs, saying which subject shapes each statement decides.
4. A Language bridge and an ENGINES entry in crates/didbot-policy-records/src/engine.rs.
5. A policy in policy-site/recommended/ or crates/didbot-policy-records/floor/ with its cases beside it, which the harness in crates/didbot-pds/tests/cases/mod.rs runs through a real gate.