An experimental library to manage workers/tasks and clusters. This rely in crdt and swim libraries.
OCaml 97%
1%
JavaScript 1%
Dune <1%
CSS <1%
Dockerfile <1%
Shell <1%

README.md

hive #

A worker runtime for OCaml 5 + Eio: typed mailboxes, pure message handlers, supervision trees, scheduling, and one-shot tasks. Workers are local by default — a fiber, a bounded typed mailbox, a supervisor — and an optional cluster sub-library adds replicated state, a cluster registry and groups, remote messaging, and worker placement.

type _ msg =
  | Incr : int -> unit msg   (* cast *)
  | Total : int msg          (* call, replies int *)

module Counter = Hive.Server.Make (struct type 'r t = 'r msg end)

let handle : type r. r msg -> int -> (int, r) Counter.reaction =
 fun msg state ->
  match msg with
  | Incr n -> Counter.continue (state + n) ()
  | Total -> Counter.continue state state

handle is a pure function: it takes a message and a state and returns the new state, the reply, and any effects as data. It can be tested by calling it. The full program is examples/getting_started.ml, compiled as part of the build.

Contents #

Library Modules Purpose
hive Worker, Server, Supervisor, Task, Scheduler, Registry, Group, Pubsub, Mailbox, Addr, Outcome, Effects Servers with GADT-typed messages, restart strategies, timers, one-shot tasks, a type-safe local registry and groups, pubsub as an injected capability
hive.cluster Replica, Dist_registry, Dist_group, Remote, Placement, Placement_pinned, Placement_cp, Ring, Node, Cluster_intf, Wire A generic replicated value over gossip, cluster registry and groups, remote send with per-worker codecs, and three singleton-placement strategies (AP ring, AP pinned-to-a-node, and CP)
hive.cluster.swim adapter Membership and transport over swim gossip (UDP)
hive.cluster.crdt adapter Replicated JSON documents over crdt

hive.cluster depends on neither swim nor crdt: it is functorized over three module types (MEMBERSHIP, TRANSPORT, REPLICA), so implementations are swappable and the layer is testable against the in-memory fakes in fakes/.

Requirements and build #

OCaml ≥ 5.1. The core library depends on eio, kcas, and kcas_data. The adapter sub-libraries additionally require swim and crdt.

dune build
dune runtest
dune exec examples/getting_started.exe
dune exec bench/election_bench.exe
dune install

Documentation #

docs/README.md indexes the guides for building applications: getting started, servers, supervision, timers, registry/groups, clustering, and testing. AI coding agents should also read AGENTS.md and llms.txt.

Consistency properties #

  • Replication, the cluster registry and groups, and Placement are AP: reads are local, replicas converge after partitions heal, duplicates are possible during a split and are stopped on discovery, and there is no persistence — state lives only in memory. Workers under AP placement must tolerate duplicate execution.
  • Placement_pinned is AP but pins each worker to a node chosen by Node.id rather than the ring. Because at most one live member carries a given id, a split cannot duplicate it — only the side that can see the node runs it; if the node is absent the worker runs nowhere (strict, no fallback). There is no persistence: a crash loses in-memory state.
  • Placement_cp is CP: it is designed to run at most one instance of a placed worker, using majority quorums of expiring grants. It trades availability for that: nothing runs without a majority of the configured cluster size. Its safety argument assumes bounded clock rates and that a stopping worker stops; a worker whose entire process pauses past its lease can still act once while stale, so a side-effecting resource needs its own fencing if that must be impossible.
  • There is no consensus log in hive. Applications that need one should run an external implementation behind a worker.

Verification status #

The test suite covers the core with unit and property tests, and the cluster layer with simulated multi-node scenarios: partitions and heals, asymmetric membership views, silent message loss, crash-restarts, garbage and semi-valid message injection, and randomized fault schedules with invariants checked at each step. These simulations run on in-memory transports with a controlled clock; behavior on real networks has only been exercised by small loopback tests of the swim adapter. The CP placement protocol is tested, not formally verified. Benchmarks (in-process simulation, 1–100 nodes) show election and failover latency bounded by the lease setting rather than the node count.

APIs may still change.

License #

ISC.