gazagnaire.org / ocaml-eio-fs
In-memory filesystem for Eio
Star 1
Fork 0
Watch
OCaml 94%
C 5%
1%
Perl <1%
Dune <1%
Shell <1%
HTML <1%
201 1 0
bench
bin
examples
fuzz
lib
test
.ocamlformat
dune
dune-project
eio-fs.opam
eio-fs.opam.template
LICENSE.md
README.md
README.md

eio-fs #

Filesystem helpers and an in-memory Eio backend.

A program written against Eio.Path should not care whether its files are on a disk or in memory. Tests often want them in memory, and so does a unikernel, which has no filesystem at all. This library provides that in-memory backend. Eio_fs.Fs.v () returns the root of a new, empty filesystem in memory. Eio_fs.Fs.load returns the root of a new one holding a copy of a directory tree, and Eio_fs.Fs.tree prints any Eio.Path tree. Two of these filesystems share no file.

For crash-consistency tests, Eio_fs.Chaos wraps a directory and injects faults into it: short writes, torn writes, failed syncs and power loss. A torn write stores some of its bytes and then reports a failure. To simulate power loss, Eio_fs.Chaos.Crash treats each data write as unsynced until an Eio.File.sync on the same file makes the earlier writes durable. It also keeps a journal of creations, renames, unlinks and the other directory operations, in the order the program issued them. When the simulated crash comes, unsynced data writes may be lost or reordered. Of the directory operations not yet committed, only a prefix survives, and each lost operation is undone completely.

Three more libraries come with the package. eio-fs.sync has durable file operations, eio-fs.unix the calls that only exist on POSIX systems, and eio-fs.path lexical operations on paths.

Installation #

opam install eio-fs

Usage #

Eio_main.run @@ fun _env ->
let fs = Eio_fs.Fs.v () in
Eio.Path.save ~create:(`Or_truncate 0o644) Eio.Path.(fs / "hello") "world";
assert (Eio.Path.load Eio.Path.(fs / "hello") = "world")

The code that uses fs sees an ordinary Eio.Path. It cannot tell which backend is behind it, so the same code runs against a test's in-memory files or a unikernel's scratch filesystem.

Embedding a file tree at compile time #

The eio-fs-crunch binary walks a directory tree and writes an OCaml module and its interface. The module contains every directory, file and symbolic link in the tree, as ocaml-crunch does. A dune rule regenerates the module when the asset directory changes:

(rule
 (targets assets.ml assets.mli)
 (deps
  (source_tree static))
 (action
  (run %{bin:eio-fs-crunch} -o assets.ml static)))

The generated assets.ml holds the tree as a value of type Eio_fs.Crunch.t. You never write these entries by hand:

(* Generated by eio-fs-crunch. Do not edit. *)

let entries : Eio_fs.Crunch.t =
  Eio_fs.Crunch.
    [
      Dir { path = "/"; perm = 0o755 };
      File
        {
          path = "/index.html";
          perm = 0o644;
          contents = "<h1>Hello from the image</h1>\n";
        };
    ]

let load () = Eio_fs.Crunch.load entries

At run time, Assets.load () rebuilds the tree in a new in-memory filesystem and returns its root. A unikernel or a test with no real filesystem can then read its static assets through an ordinary Eio.Path. This repository crunches examples/readme/static with the rule above, and the example below runs against the module it generates:

Eio_main.run @@ fun _env ->
let fs = Assets.load () in
assert (
  Eio.Path.load Eio.Path.(fs / "index.html") = "<h1>Hello from the image</h1>\n")

Provenance #

This code started as Patrick Ferris's eio_mem, which was derived from the Eio filesystem interfaces. The path handling comes from fpath. All three are under the ISC licence; see LICENSE.md.