You can throw away the instance a service runs on and rebuild it from its declaration, boot image and type included. You cannot do that with its data volume. Most provisioning tools still handle both the same way, so a change of instance type, or a small mismatch on a disk, can turn into a plan that deletes your data.

caravan keeps them apart. You declare a target; caravan asks the provider what exists and works out, in a pure planner, the ordered steps that close the gap. If closing it would destroy something, you get a conflict and no plan. A provider backend runs the plan step by step and waits for each resource to settle before the next.

Three rules hold for every plan.

A plan never destroys the data volume. It won't delete, shrink or reformat it. If the volume doesn't match, you get a conflict to resolve.
Compute is disposable. Change the boot image or the instance type and the instance is stopped and replaced. Its data volume and routed address are detached first and kept.
Routine updates belong to the update channel. If an over-the-air update changes the image an instance runs, the instance keeps its provision tag and caravan leaves it alone. In-place updates are the job of uplink; caravan replaces compute only when the declared provision input changes.

Three tools manage one fleet, and each owns one thing. uniboot builds the images. It owns their layout, their slots and the content store, and it is the only tool that writes image bytes to a medium. uplink moves trust: it packages, signs, verifies and delivers releases, and it owns anti-rollback and the confirmation witness. Anti-rollback means a device refuses any release whose version is below a counter it keeps, so an old and vulnerable release cannot be put back. The confirmation witness is a statement the device signs with its own key after it has booted a release and confirmed it, naming that release's digest.

caravan owns the intent. It declares which target runs which release, plans, converges and refuses, and it never builds, signs or reads image bytes. The three tools name a release by one digest: the SHA-256 of the release's descriptor, which is the signed manifest listing the release's images by their own digests. A boot slot stores those 32 bytes, and caravan reads them as the hex after @sha256: in the target's image: reference. uniboot computes the digest at build time and uplink carries it. caravan asserts it (the provision tag is the digest inside the image: reference, so caravan hashes nothing), and the device reports it back.

The planner does no I/O and knows no provider, so the same planner serves every kind of target. Each kind has a backend that implements the Caravan.PROVIDER signature, with constraints of its own:

Local (caravan.vz) runs virtual machines under the macOS vz launcher and uses a state directory as the provider database. It has no firewall, address or registry.
Cloud (caravan.scaleway) manages the instance, the data volume, a firewall, a routed address and a registry. Plans show catalogue prices, and an instance is renewed by replacing it. A routed address is a public IP address reserved independently of any instance (a Scaleway flexible IP), so it can move from the instance being replaced to the new one while the domain's DNS record keeps pointing at it. A registry is a namespace in the provider's container registry, where images are pushed before an instance runs them.
Bench board (caravan.board) drives a physical board whose storage this host can see. The provider database lives on the board itself. A flash rewrites only the system partitions, so the state partition survives. Each target says how its board is powered (manual, always-on or a command); there are no per-board presets.
Field (caravan.spool) reaches a device during connectivity windows instead of over a standing connection. Commands wait durably in a spool that the transport drains when a window opens. What caravan observes is the device's own last check-in, and renewal: stage updates the device through the update channel without ever replacing it.

A target file stays with a device for its whole life. On the bench a board is renewal: replace, and it becomes renewal: stage when it ships. On a field target most spec fields assert facts instead of requesting them. The hardware is matched, and drift on a physical fact is a conflict you fix in the target file.

Installation #

Install with opam:

$ opam install caravan

If opam cannot find the package, it may not yet be released in the public opam-repository. Add the overlay repository, then install it:

$ opam repo add samoht https://tangled.org/gazagnaire.org/opam-overlay.git
$ opam update
$ opam install caravan

Usage #

A target is a YAML file. The provider is whichever member carries its own settings (scaleway:, vz:, board: or spool:). A target file can therefore only spell fields its provider has, and the decoder insists on the ones the provider requires:

name: camel
scaleway:
  zone: fr-par-2
instance-type: STANDARD2-A2C-8G
image: 11111111-2222-4222-8222-222222222222
volume-gib: 50
inbound-tcp: [80, 443]
registry: camel
bootstrap-file: ./bootstrap.yaml

The bootstrap document (bootstrap-file) is the configuration a new instance receives at first boot; on a cloud it is the user data. It may hold secrets, so caravan never looks inside it and every printer redacts it.

name: sat-1
spool:
  directory: /var/spool/caravan/sat-1
  intent-key: ./ops.pem
renewal: stage
instance-type: vehicle
image: release@sha256:1111111111111111111111111111111111111111111111111111111111111111
volume-gib: 1
address: false

Then:

$ caravan plan -c camel.yaml      # the actions that would run, billable marked
$ caravan apply -c camel.yaml     # show the plan, confirm, execute
$ caravan status -c camel.yaml    # what the provider reports; converged or not

plan prints, numbered, the actions that would close the gap. A step that starts a recurring charge shows its price when the provider's catalogue has one ([EUR 7.99/month], from Scaleway's server types), and [provider charge] when it does not. Each step is priced in whatever unit limits its target. A release staged over a windowed link, for instance, shows the bytes it has to carry ([1.2MiB over the link]). That figure is the size of the release descriptor plus the signed size of each image in it that the device did not list as held at its last check-in. A device that has never reported counts as holding nothing, so staging to it costs the whole bundle. The envelope's signature and CBOR framing are small constants and are left out. The listing ends with one total per unit. apply runs the plan once you confirm it; --yes skips the confirmation.

A directory of targets is a fleet, and caravan converges it in filename order behind a gate. The gate measures convergence. caravan has no idea what a healthy service looks like, so service health plays no part. A target is planned only when every target before it in filename order has an empty plan. The first target with a pending plan or a conflict closes the gate, and every target after it is Held for that pass. A phased rollout can therefore never get ahead of a target that refuses or has not yet confirmed. Credentials are resolved the same way the scw CLI resolves them: from the environment first, then from the profile.

Conflicts #

Some drift can only be closed by destroying something. The observed volume may be smaller than the spec asks for, or report different IOPS. (IOPS, input/output operations per second, is the rate a block volume is provisioned to sustain; on Scaleway it tells apart storage classes of the same size.) A stale attachment left by a deleted server may still hold the volume. A volume of the wrong storage class may sit under the expected name. The managed firewall may accept inbound traffic the target does not declare. The planner never turns any of these into a delete or resize step. It stops with a conflict that names the resource and the mismatch, and you resolve it in one of three explicit ways:

apply alone never resolves a conflict. It refuses and names the flag for each resolution the conflict offers, both flags when it offers both. An operator told only about --force would never learn that the volume can be deleted at all. plan prints the same resolutions step by step before it fails on the conflict.
apply --force runs the resolutions that lose nothing and prints each step. It detaches a data volume from every stale holder in one pass. It adopts a same-named resource that caravan did not create by tagging it. It also records the target's provision state on an instance that reports none.
apply --destroy-data runs the destructive resolution. A volume that cannot be reconciled (too small, different IOPS, the wrong class, or foreign) is detached from every holder, each holder being powered off first. It is then deleted and recreated empty on the next pass. In an interactive run you must type the volume's name again, and --yes does not skip that. Without a terminal, --yes together with --destroy-data is the authorisation. The provider call checks the same name again on the server side.

A foreign data volume is the one conflict that offers both fixes. You can adopt it by tagging it where it is, or delete it and recreate it empty. --destroy-data on its own takes the destructive fix. When both flags are given --force wins, so adoption is tried before deletion.

A plan by itself never contains a destroy step. Destruction comes only from an explicit resolution. There is always one other way out: change the target to match what exists and run plan again.

Neither flag answers two conflicts, because any fix caravan offered would assert something the provider contradicts. The first is an instance that carries more than one provision tag. It claims several states and proves none, even when the wanted one is among them, and it is what a delivery left half finished looks like. The tags are read sorted and deduplicated, so the verdict depends on the instance and never on the order the provider listed them in. A replace target resolves it by replacing the instance. A stage target refuses: staging and activating a release leaves the stale tag in place, and recording the wanted state would claim a convergence the instance contradicts. The fix there is to remove the stale tags. The second conflict arises when more than one resource claims the target's name. caravan will not choose between them, so it refuses and names every candidate.

Identity without a state file #

caravan keeps no state file; the provider is the only source of truth. Two mechanisms make that work.

Resources are identified by name and by a provenance tag. A target named camel looks up its resources under the names the planner gives them: instance camel, data volume camel-data, firewall camel-fw. A name alone proves no ownership. Provider names are not unique, and a resource with the same name may be a stale leftover or a plant. caravan therefore writes the identity tag caravan:camel on every taggable resource it creates, and a lookup needs both the name and the tag. A resource with the name but without the tag is a Foreign_resource conflict, and caravan does not quietly adopt it. Adopting a data volume decides what the node's persistent state is, so somebody has to make that decision, and apply --force makes it by writing the tag. Routed addresses have no provider name, so for them the tag is the only handle.

The provision state is an instance tag. When caravan creates an instance it also writes caravan:provision: followed, verbatim, by the digest in the target's image: reference, the one uniboot computed at build time. On every plan, observe reads the tags back and compares that identity with the one the current target file pins. If they are equal, the instance realises the current declaration, and caravan ignores the image it actually runs, since updates the instance applied itself are legitimate drift. If they differ, the declaration has changed since the instance was provisioned, and the plan replaces it: stop, detach the data volume and the address, delete, recreate from the new image, reattach. An instance with no tag at all also counts as drift. It has proved nothing, and treating it as converged would let an instance block its own delivery just by reporting less than one with a stale tag.

The comparison is between the declared identity and the recorded one, and never between the declared image and the observed image. That choice is what lets in-place updates over uplink live alongside declarative replacement. A reference without a digest pins no identity. A replace target then compares the observed image with the declared one, which suits a local path on a disposable target. A stage target is refused as soon as its file is read, because an identity that has to be echoed back over a link cannot be a local path.

The provider database #

The cloud backend can ask its provider, so it keeps no database of its own; the tags above are how it recognises what it created. The other three backends have no such API. Each keeps the database itself, in the one place that belongs to the target: a state directory on this host, the board's own boot partition, or the spool directory the fleet service writes.

Some other party can write to each of those directories, so no backend reaches them by path. Eio is OCaml's effects-based I/O library, and a capability there is a directory handle opened once; every later path is resolved against it, so code that holds it can reach only that subtree. Each backend resolves its directory once, opens it as such a capability and goes through it for every file. A join resolves one component at a time inside the subtree, and a component that leads out of it fails instead of being followed. A symlink planted inside therefore cannot send a read, a write or a delete elsewhere on the host, whether it sits where a file should be or in place of a directory the backend creates. The vz and spool backends take the switch that keeps their capability open (~sw, Eio's scope for resources, which closes the handle when it ends) and work only while it lives. The board backend instead opens its boot partition once per operation, because a flash unmounts that filesystem and the host cannot unmount one this process holds open.

Local (caravan.vz): each instance is a directory. It holds the boot disk (an APFS clone of the released image where the filesystem supports it), the tags, the list of attached volumes and the pid of the vz process. Each volume is a sparse file. The pid is stored with the process's start identity, so a recycled pid is never mistaken for the launcher. instance-type follows this backend's own grammar, <cpus>c-<memory>g, so 4c-8g means four CPUs and 8 GiB. The launcher reads the list of attached volumes once, at start-up, so attaching or detaching a volume needs the instance stopped; both refuse while its process runs.
Bench (caravan.board): before any operation, the backend has the host prove that the mount point the target names is a mounted partition of the device it names (with findmnt and lsblk on Linux, diskutil on macOS). It refuses when the partition records another board's serial number. The target declares the device and the mount point separately, and without this check a renumbered device could have one board's eMMC rewritten while the metadata landed on another's. A flash unmounts the partition, writes the device and mounts it again. Only then does it write all of the instance metadata (serial number, tags, image reference, name, flavour and volume record) through the filesystem that came back, and it reads the metadata again before it reports success. Nothing goes through a view the flash invalidated, and nothing recorded before the flash is lost. If the kernel holds the boot partition mounted and the backend was given no mount control, the backend refuses to write under it.
Field (caravan.spool): apply writes one file per intent into <spool>/outbox, named so that queueing the same intent twice does nothing. Every file is written under a dotted temporary name and then renamed into place. Dotted names are reserved for the spool's own files, so the transport never picks up a half-written file.

With a signing key, a detached COSE_Sign1 signature is published next to each intent (as its .sig sibling) and again under the SHA-256 of the bytes it covers. COSE_Sign1 is the single-signer signature structure of COSE (RFC 9052), encoded in CBOR, and "detached" means the signed bytes are not copied into it. Queueing a command again replaces the intent file in place, and a crash can interrupt that between the new intent and its sibling. The signature is therefore looked up by the digest of the intent bytes on disk. Whichever generation the crash leaves queued, old or new, is checked against its own signature and never against its neighbour's.

Some checks need no key, and they refuse a contact window on every spool, signed or not. The queued command must match the name that labels it. Its sequence number must be above the window already acknowledged. No bootstrap document may be left over that no queued intent pins (a bootstrap document is spooled under its digest, and a bootstrap intent pins that digest). A pinned operator public key answers what those checks cannot, namely who authorised the bytes. A window carries all of its commands or none of them, and a refused intent is moved aside as evidence so that it does not block every later window.

Convergence has to be proved; caravan does not take the report's word for it. With the device's public key pinned, a provision state counts only when the device's own signed witness of the release it booted verifies against that key. observe checks the stored witness again instead of trusting whatever the report path accepted.

Library #

The same engine is available as a library. Caravan.Plan.v is pure: given a spec and the observed state, it returns actions or a conflict, so the reconciliation logic can be tested without any provider. Caravan.Make (P) runs plans over any Caravan.PROVIDER:

module Engine = Caravan.Make (Caravan_scaleway)

(* Observe, plan, apply. A conflict and a backend failure both come back
   as values; nothing raises. *)
let converge backend spec =
  let engine = Engine.v backend in
  let error fmt = Fmt.kstr (fun message -> Error message) fmt in
  match Caravan.observe engine spec with
  | Error e -> error "%a" (Caravan.pp_error engine) e
  | Ok observed -> (
      match
        Caravan.Plan.v ~capabilities:(Caravan.capabilities engine) spec observed
      with
      | Error conflict -> error "%a" Caravan.Conflict.pp conflict
      | Ok plan -> (
          match Caravan.apply engine plan with
          | Ok applied -> Ok (Caravan.Applied.instance applied)
          | Error (action, e) ->
              error "%a: %a" Caravan.Action.pp action
                (Caravan.pp_error engine) e))

A plan is computed within its provider's capabilities, and the provider states them. Every Caravan.PROVIDER answers capabilities, an engine passes the value on as Caravan.capabilities, and Plan.v requires it with no default. A capability is a fact that changes the plan itself, as opposed to how a step runs. So far there is one: whether a volume can attach to a running instance. A provider whose instance reads its disk list only at launch says so, and the plan wraps the attach in a stop and a start, so a single apply still converges the target. An instance that is already stopped takes the attach as it is. The fields of a target are not capabilities. If a target asks for a routed address or a staged release that the backend cannot provide, the step refuses and names the field to clear.

Plan.t is a closed value. Plan.of_actions is the only way to build one from actions a caller writes. It refuses the first step that names a planned resource that no earlier step of the right kind creates, and says which step named which index. Plan.v emits nothing else, so apply never has a plan of its own to refuse. Every error it returns comes from the backend, labelled with the step it happened in and printed by Caravan.pp_error. That printer comes from the engine and not from the backend module, so a caller that made the backend's error type abstract can still print what went wrong.

apply reads no declaration. Each step carries every value it acts on: the name and type of the instance it creates, the identity and provision tags it writes, and the key the bootstrap document goes under. What an apply does to the provider is exactly what the approved plan showed, whatever the target file says by the time it runs. Fleet.plan and Fleet.progress implement the rollout gate. They are pure as well, so the ordering logic is tested with no fleet at all.

All backends share one shape: a v that takes labelled settings and returns (t, error) result, the flat Caravan.PROVIDER signature, and a submodule for anything else the backend offers. The library has one module per concept (Spec, Observed, Plan, Action, Conflict, Kind, Resource, Renewal, Release, Price, Bootstrap, Fleet). The modules a caller builds values of keep t abstract behind a v constructor and flat accessors. A new spec field then arrives as an optional argument and breaks no caller. Bootstrap documents are opaque Caravan.Bootstrap.t values that every printer in the library redacts, so a plan written to a log never leaks the payload.

API #

caravan.mli holds the Spec, Observed and Plan types, Conflict with its two resolution functions, the PROVIDER signature, Make (P : PROVIDER), and Fleet for the rollout order. The backends are caravan_vz.mli, caravan_board.mli, caravan_spool.mli and caravan_scaleway.mli. Each adds what only it has. The local backend has its flavour grammar and the board its power and mount control. The spool has a ground side and a transport side, and Scaleway has its resource mapping and secret store.

I took the idea from InfraKit (Docker, now archived), which paired a builder of immutable images, LinuxKit, with a toolkit that converged infrastructure to a declared spec. caravan does the same job next to uniboot, itself inspired by LinuxKit. The code shares nothing with InfraKit, and there are no plugin daemons here, just one observe, plan and apply with explicit conflicts. The provision tag is InfraKit's trick too: its group plugin tagged every instance with the SHA of its configuration and reconciled by comparing tags, so no state file was needed. Kubernetes does the same with its pod-template-hash label.
Terraform made plan-then-apply the usual workflow, with a plan a person reads before anything changes. caravan's CLI works that way, one target at a time.
If you come from containers, balena is the nearest whole-stack equivalent, with its own OS, an on-device supervisor that converges to a desired state, and a fleet console. caravan runs images and unikernels instead of containers. The same planner also provisions the cloud and the bench, which device platforms leave to other tools, and it does not assume a supervisor that is always connected.
Mender, RAUC, SWUpdate and Eclipse hawkBit cover what uniboot and uplink cover here: health-gated rollback on the device, signed delivery of updates, and servers for rollout campaigns. Those tools reserve a second full partition for the rollback copy (classic A/B). uniboot instead keeps every version as an immutable object in a content-addressed store and moves small pointers to the confirmed release, the release on trial and the last known good one, so a rollback costs a pointer and no partition. caravan sits above that layer and decides which target must converge to which declared input.
Device shadows and twins (AWS IoT, Azure IoT Hub) are the desired-versus-reported pattern that caravan's spec and observation follow, down to the rule that a device you have not heard from is not a device that has gone. caravan keeps the pattern free of any broker: the spool is a directory and the report is a file. The same reconciler also covers targets no shadow service models, namely the cloud itself and the bench.
The spool is the oldest pattern here. UUCP moved queued work over dial-up windows from /var/spool decades before IoT took up the term store-and-forward, and /var/spool is still the right name for the model.

Licence #

ISC License. See LICENSE.md for details.

Clone this repository

caravan #

Overview #