merge-queue #
mq is a merge queue for Git that you run locally, on your laptop or on a
build host.
mq land fix rebases fix on its target, runs the target's checks on the
rebased tree and fast-forwards the target to it if they pass. It needs no
server and no network, and it keeps its state in the repository, where git
can read it. GitHub's merge queue and bors-ng do the same for pull requests,
on a server.
mq is still in development (version dev, before 0.1.0). The specification
in doc/ is ahead of the code; "Gaps" in
doc/design.md lists what is missing.
Contents #
Security #
Checks are scripts from the branch being landed, and mq runs them with your
rights, as Git runs a hook. Landing a change runs its code. When the system
scheduler starts the runs, it also runs outside the sandbox of whoever queued
it. Anyone with write access to the remote can forge a check result
(doc/trust.md).
On macOS, git config mq.confine auto runs each check under Seatbelt. Linux
has nothing yet, and the hook and the resolver are never confined
(doc/trust.md).
mq push uploads check results with the branches. It only pushes to a remote
listed with its exact URL in the rules' [push] section, and only if it can
tell the remote is private. Local paths are. GitHub is asked through its API
(with GH_TOKEN, GITHUB_TOKEN or gh auth token). Tangled is public, and
other hosts are refused.
Background #
CI on a pull request tests its head, and the merge produces a tree nobody
tested. mq moves a branch only to a tree it has just checked, and stores
the result per tree, check and build configuration, so the same tree is not
checked twice. The rules sit on their own branch, mq/config, as in Zuul's
config projects or Gerrit's refs/meta/config, so a change cannot edit the
rules it is judged by. doc/related-work.md compares
mq with other tools.
mq does not know about pull requests. If you review on a forge, keep doing
so and call mq land --dry-run from CI.
Install #
You need OCaml 5.2 or later. mq is in an opam overlay, which builds it from
the main branch of its repository:
$ opam repo add samoht https://tangled.org/gazagnaire.org/opam-overlay.git
$ opam install merge-queue-eio
That gives you mq, one executable: it is also the supervisor every check
runs under, started again by itself.
Usage #
Quick start #
A throwaway repository. test.sh stands in for a test suite and fails when
lib/ttl.ml stops trimming its input. mq init finds no pre-commit hook
and no build system it knows, so main has no check until a rule lands in
mq/config.
$ git init -q -b main app && cd app
$ mkdir lib && echo 'let of_string s = int_of_string (String.trim s)' > lib/ttl.ml
$ cat > test.sh <<'EOF'
> #!/bin/sh
> grep -q String.trim lib/ttl.ml || { echo 'ttl: " 30" is no TTL' >&2; exit 1; }
> EOF
$ chmod +x test.sh && git add -A && git commit -q -m "ttl: read a TTL"
$ mq init
created mq/config: DAG main, no checks
$ R=$(mq new add-test mq/config)
$ cat > "$R/mq.toml" <<EOF
> [check.test]
> command = "./test.sh"
>
> [branch.main]
> checks = ["test"]
> EOF
$ git -C "$R" commit -q -am "main runs test.sh"
$ mq land add-test
landed add-test into mq/config: 1 commit, checks rules (1s)
mq new prints the path of a fresh worktree. The first version breaks the
test, so main stays put. A failing check runs twice, to rule out a flake,
hence -2 in the log's name. mq status also says the tip of main was
never checked: it predates mq init.
$ W=$(mq new hex-ttl)
$ echo 'let of_string s = int_of_string ("0x" ^ s)' > "$W/lib/ttl.ml"
$ git -C "$W" commit -q -am "ttl: read hexadecimal"
$ mq land hex-ttl # exit 1
needs_work hex-ttl: check test failed (ttl: " 30" is no TTL)
log: /home/alice/app/.git/mq/logs/check-25c3bc4f10047094e08b05cce48c90701c234ea4-test-2.log
worktree: /home/alice/app.worktrees/hex-ttl.2
next: mq land hex-ttl
$ mq status
hex-ttl needs_work main check test failed (ttl: " 30" is no TTL) /home/alice/app.worktrees/hex-ttl.2
main idle
runs started by mq land · last ended 14:02 (exit 1)
$ echo 'let of_string s = int_of_string ("0x" ^ String.trim s)' > "$W/lib/ttl.ml"
$ git -C "$W" commit -q -am "ttl: trim before reading"
$ mq land hex-ttl
landed hex-ttl into main: 2 commits, checks test (1s)
Then, with mq events and plain Git:
$ mq events hex-ttl
14:02:09 queued hex-ttl for main, by alice on laptop
14:02:09 checking hex-ttl on main: test
14:02:11 needs_work hex-ttl: check test failed (ttl: " 30" is no TTL)
14:03:40 queued hex-ttl for main, by alice on laptop
14:03:40 checking hex-ttl on main: test
14:03:41 landed hex-ttl into main: 2 commits, checks test (1s)
$ git reflog show --format=%gs -1 main
mq land hex-ttl: 2 commits
$ git notes --ref=mq-checks show 'main^{tree}'
mq1 passed test tree=ee1e31e5866f3eb01199df8ee639cf10fa3f63b0 attempt=1 config=839b998c74ce69d6d2b50f62cb73dba6403a1b1d9a4a6f713b7d2ddfcf957d14 rules=27360193b6e142053af36eceaea64060f0e86503 machine=laptop mode=self confined=no seconds=1 at=2026-10-04T14:03:41Z log=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Progress goes to stderr, as a live panel on a terminal and one line per
change of state anywhere else. The outcome goes to stdout (JSON with
--json), and its first word sets the exit code. On a terminal, mq status, mq stats and mq events are drawn in mq's colours; piped, or
with --raw, they print plain lines, and with --json a JSON document:
| Exit | Outcome | Whose move |
|---|---|---|
| 0 | landed, passed (a dry run) or queued (--no-wait) |
nobody |
| 1 | needs_work: a check failed on the replayed tree, or a conflict waits |
the caller fixes the change |
| 2 | refused: the call cannot be made in this state |
the caller fixes the call |
| 3 | stuck: a bound was reached, a check ended in error, the replayed tree fails a check the target's tip already fails, or a checkout of the target holds a change the move would overwrite; the branch stays queued |
the operator |
| 4 | removed: another caller dequeued or dropped the branch, or queued it again at a newer tip or with other flags |
read mq status NAME |
| 5 | unavailable: the machine could not serve the call |
the operator |
| 125 | a defect of mq, or a command not implemented yet |
report it |
| 130 | interrupted: Ctrl-C at a terminal withdrew the branch |
the caller |
Other commands use the same words and codes. mq new prints a path, and mq status exits 0 whenever it can answer.
State in Git #
| State | Where Git keeps it | Read it with |
|---|---|---|
| A branch's base | its upstream, and a base: field in its description |
git branch -vv |
| A queued land | an annotated tag at refs/mq/queue/NAME, pointing at the queued tip |
git cat-file -p refs/mq/queue/NAME |
Its description and fields (mq new --desc, --field KEY=VALUE) |
branch.NAME.description |
git config branch.NAME.description |
| The rules | mq.toml on the branch mq/config |
git show mq/config:mq.toml |
| A check's result | a note on the tree it is about, under refs/notes/mq-checks |
git notes --ref=mq-checks show COMMIT^{tree} |
| A stopped conflict | a git rebase in progress in the branch's worktree |
git -C WORKTREE status |
| Each move | the branch's reflog | git reflog show main |
| What a land, a drop or a hand-out left | refs under refs/mq/, never under refs/heads/ |
git for-each-ref refs/mq/ |
The journal behind mq events, the check logs, the run lock and the build
slots stay in .git/mq/. A land writes nothing outside the repository and its
pool, so a sandbox that confines a process to them can run mq.
Branches and the DAG #
[check.test]
command = "./test.sh"
[branch.staging]
checks = ["test"]
[branch.main]
takes = "staging"
checks = ["test"]
[branch."release/*"]
checks = ["test"]
With these rules, new work lands in staging (or in the base given to mq new NAME BASE), and mq land staging --onto main fast-forwards main.
Branches made with git switch -c land the same way, and a stack lands from
the bottom. every = "1h" on an edge parses, but no schedule runs it yet.
Without a scheduler, mq land takes the run lock, checks its own branch and
exits. mq schedule start hands the runs to launchd, systemd or cron, which
makes mq land --no-wait work. Agents in sandboxes want this, since a check
an agent starts runs in its sandbox, where a refused socket looks like a
failed test (doc/runs.md).
The pool of worktrees #
Worktrees go to REPOSITORY.worktrees/NAME.N, or under mq.pool. N is never
reused, so a stale path in a shell cannot point at another branch. A land
deletes the worktree (unless --keep-branch), and on macOS keeps its _build
as a copy-on-write clone for the next one. mq.poolBudget caps those clones,
oldest out first. mq.freeSpace is the free space to keep; below it, with
nothing left to evict, mq new fails unavailable and prints both numbers.
Hook and conflict resolver #
A [hook] command gets every journal event on stdin, one CloudEvents JSON
line per call, in order, from a cursor in refs/mq/hook. It can refuse a
branch when it is queued. An issue tracker goes there, reading the fields of
mq new --field Issue=123 from the branch description, which is never pushed.
A conflict is left in the worktree as a git rebase in progress. Fix it and
run mq land NAME again. A [resolve] command can fix it instead, as git mergetool would; its result lands only if the checks pass
(doc/conflicts.md).
Several machines #
mq land never touches the network. mq push publishes the DAG's branches
and the rules, fast-forward only. mq pull fetches the shared history and
replays your unpublished landings on it, checking them again. Remotes are
local paths, file:// URLs or SSH. The rules must allow the push:
[push]
origin = "git@github.com:team/app.git"
Each push also backs up this checkout's queue refs and notes under
refs/mq/checkpoint/ on the remote (doc/machines.md).
Several repositories #
mq lands into one repository at a time. For a change to a library and to an
application that uses it, land the library through its queue, then the
application through its own. Nothing makes the pair atomic.
The application declares the library in .mqmodule, in git config syntax:
[dependency "lib"]
path = vendor/lib
url = https://example.org/lib.git
commit = <the 40 hexadecimal digits of a commit of lib>
path is the symlink in a checkout, url the library's repository (a local
path works), commit the library commit for this application commit. When
mq checks the application, vendor/lib is a read-only worktree of that
commit under POOL/.deps/, and the check result covers the lock with the
rest of the tree. To take a newer library, land a commit made with git config -f .mqmodule dependency.lib.commit COMMIT.
The rest of doc/commands.md is not
written yet: mq updating the lock itself, mq new --edit, one mq land
for every part, and mq init cloning the dependencies.
The manual #
doc/sessions.md has worked sessions,
doc/commands.md the commands,
doc/configuration.md the keys and
doc/design.md the implementation. mq --help --all lists
the operator commands and the plumbing.
API #
merge-queueis the model: total state machines that read no clock, file or process. See lib/, for example lib/rules.mli and lib/outcome.mli.merge-queue-eioruns the model's actions against Git and processes, and buildsmq. See lib/eio/.
Neither interface is stable before 0.1.0.
Contributing #
Run the quick start in a scratch directory and open an issue at https://tangled.org/gazagnaire.org/ocaml-merge-queue/issues if anything breaks or surprises you.
License #
ISC. See LICENSE.md.