title: embedding #

hydrant can run inside a program that isn't written in rust, through a small C api in ffi/. the embedded hydrant still serves its usual rest api and xrpc, and the program talks to it over that, ideally on a unix socket. the C side only starts hydrant and learns when it stops, so no events or callbacks cross the language boundary.

rust programs don't need any of this, use the library directly (see the statusphere example).

building #

cargo build --release -p hydrant-ffi

this produces the static library libhydrant_ffi.a in target/release (or target/<triple>/release when a target is set). the header is ffi/include/hydrant.h.

by default hydrant allocates with the program's own malloc. add --features alloc-jemalloc to give it the same tuned jemalloc the binary uses instead. its symbols are prefixed (_rjem_*), so only hydrant's allocations move to it and the program's malloc stays as it was.

settings #

hydrant_start takes a json object with the same settings the binary reads from its environment (see configuration), plus RUST_LOG. the process environment and any .env file are ignored, so the embedding program's own environment can't leak in.

HYDRANT_API_BIND is required, because the binary's default listens on every interface. use a unix socket, or none if the program doesn't need the api:

{
  "HYDRANT_DATABASE_PATH": "/var/lib/myapp/hydrant",
  "HYDRANT_API_BIND": "unix:/run/myapp/hydrant.sock",
  "HYDRANT_FILTER_SIGNALS": "sh.tangled.repo"
}

the socket is created owner-only (0600), so only the embedding program's user can reach the management endpoints, which are served on the socket and never on tcp unless HYDRANT_API_TCP_MANAGEMENT is set. HYDRANT_API_SOCKET_MODE loosens that, eg. 0660 for the socket's group too. HYDRANT_ENABLE_DEBUG and HYDRANT_DEBUG_PORT have no effect when embedded.

lifecycle #

• ignore SIGPIPE before hydrant_start. rust binaries do that at startup but a library can't, and on linux hydrant writing to a socket whose client already left would otherwise kill the whole process. the go wrapper does it for you.
• hydrant_start returns once the database is open and hydrant is starting. the api binds in the background, so poll it (eg. GET /stats) until it answers.
• hydrant_wait blocks until hydrant stops: when it fails (for example when its socket is already in use) it gives the reason, and after hydrant_shutdown it returns 0.
• hydrant_shutdown stops hydrant and waits for its database to close, which gives its memory back. anything it was in the middle of is cut off like a crash, and the next start picks up from its saved cursors. a hydrant that failed still holds its database until it's shut down.
• hydrant_free frees the handle once nothing is blocked in hydrant_wait on it.
• if a disk write fails, fjall refuses every write after it and hydrant_wait reports the database as poisoned. shut down and start again, which recovers it, and look at the disk if it keeps happening.
• only one hydrant can use a database at a time, a second hydrant_start on it fails until the first one is shut down.

go #

ffi/go wraps the C api for go programs. the go module is the repo root, so get it with:

go get tangled.org/ptr.pet/hydrant/ffi/go

the header comes with the module, but the static library doesn't, so build it and point CGO_LDFLAGS at its directory:

CGO_LDFLAGS=-L/path/to/hydrant/target/release go build ./...

h, err := hydrant.Start(map[string]string{
	"HYDRANT_DATABASE_PATH": "/var/lib/myapp/hydrant",
	"HYDRANT_API_BIND":      "unix:/run/myapp/hydrant.sock",
})
if err != nil {
	return err
}
go func() {
	<-h.Done()
	if !errors.Is(h.Err(), hydrant.ErrShutdown) {
		log.Printf("hydrant failed: %v", h.Err())
	}
}()

// later, eg. once it's done backfilling
if err := h.Shutdown(30 * time.Second); err != nil {
	return err
}

then reach the api with an http.Client whose transport dials the socket. nu tests/ffi_embed.nu builds the library and runs the go package's tests, which double as an example.