Requires OCaml 5. The Metal backend requires macOS on Apple Silicon and the Xcode Metal toolchain, and the CUDA backend requires Linux and the CUDA toolkit. Downloading a model requires uv or hf on your PATH.

Commands #

Everything ships as humpty, built once per backend:

Command	Backend	Availability
`humpty-metal`	Metal	macOS on Apple Silicon
`humpty-cuda`	CUDA	Linux, with `DS4_CUDA=yes`
`humpty-cpu`	CPU	everywhere

The examples below use humpty-metal. Substitute humpty-cpu to run without a GPU.

dune build builds Metal on macOS and CPU everywhere. CUDA is off unless you ask for it, because dune cannot test for a toolkit that may not be installed:

DS4_CUDA=yes dune build

That compiles for the GPU doing the build. Set DS4_CUDA_ARCH (sm_89 for an L4, sm_90 for an H100) to build for another one, DS4_CUDA_HOME if the toolkit is not on the PATH.

A GPU serves the model from its own memory, so one that cannot hold the whole model reads the rest across the bus for every token. A 24 GB card running the 81 GB q2 generates around a third of what this machine's CPU backend does. Size the card to the model, or use humpty-cpu.

dune build
dune exec -- humpty-metal list                  # what models exist
dune exec -- humpty-metal download q4           # fetch one
dune exec -- humpty-metal chat "Explain monads in one sentence."
dune exec -- humpty-metal agent -d ./workspace  # interactive, with tools

Run humpty-metal <command> --help for the options of each.

Choosing a model #

Use the 0731 build of Flash, DeepSeek's 2026-07-31 retrain. Pick the largest quantisation your machine has memory for.

Memory	Alias	Target	Size
256 GB+	`q4`	`q4-imatrix-0731`	153 GB
128 GB	`q2q4`	`q2-q4-imatrix-0731`	98 GB
96 GB	`q2`	`q2-imatrix-0731`	81 GB

Models are stored under $XDG_DATA_HOME/ds4. With no --model, humpty uses the best of these that you have downloaded, so chat needs no arguments after download.

A target is named for the build it carries, and the short aliases follow the newest one. Asking for q4 gets whichever build is current, while q4-imatrix-preview still names the first Flash release. humpty list dims the builds that a newer one has superseded.

Two things affect the choice. The quantisations above are built with an imatrix, which weights the quantisation by measured activation statistics. mxfp4-0731 is not, so despite being larger it is not necessarily better. Memory use is also more than the model's size, because the KV cache grows with the context, and on Apple Silicon the GPU can address only about 75% of memory by default, which sysctl iogpu.wired_limit_mb reports.

The much larger PRO models are also listed, and need 512 GB.

Chatting #

chat sends one prompt and prints the reply.

humpty-metal chat "Hello?"
humpty-metal chat -m q2 "Hello?"                # a particular model
humpty-metal chat -m /path/to/model.gguf "Hello?"
humpty-metal chat --think high "Prove it."      # reason before answering
humpty-metal chat --ctx 16384 "..."             # a larger context

The model keeps nothing between runs.

The agent #

agent puts the same model in a loop with tools, so it can work across several turns.

humpty-metal agent -d ./workspace

It opens a terminal interface. The transcript fills the screen, a line beneath it shows how full the context is and how fast the model is running, and the prompt is at the foot.

Type a request and press Enter. The prompt clears at once, and anything typed while the model is working waits its turn instead of being refused, so a second thought can be written down as it arrives. The arrow keys walk back through earlier prompts.

Tab unfolds the tool output of the turns, which is folded to one line each by default, since it is bulky and rarely what you want to read afterwards. Ctrl-C leaves, as does ctrl-D at an empty prompt. The keys are listed above the prompt.

The layout follows the width of the window. Below about 76 columns the tool column goes, leaving the conversation the whole screen, and the line of vitals keeps whichever figures still fit.

If the workspace holds an AGENTS.md, it is added to the agent's instructions, so a project states its conventions once rather than in every request. The startup banner says when one is in use.

Tools #

list, tree, read, read_lines, find, grep and stat read the filesystem, write changes it, dns resolves hostnames and bash runs a shell command.

The filesystem tools reach only what they are granted. The agent starts with a capability for the workspace given by -d, and any directory outside it must be requested with open_dir, which prints the grant. A capability rejects .. and symlinks that lead out of it, so no path the model supplies can escape.

bash is the exception. A shell runs with this process's full access, so a command it runs is not confined to the workspace.

Context #

The agent starts with a context of 32768 tokens, which --ctx sets. Tool results are shortened to 4000 characters before being added to the conversation, because tool output is what usually fills a context.

A conversation that outgrows its context moves to a larger one, up to 262144 tokens, without reloading the model. Growing costs one pass over the conversation, so the context doubles rather than creeping. Beyond that ceiling the agent reports that it is full and keeps the conversation, rather than exiting.

Library #

The deepseek library is the same functionality without the command line. V4.generate sends one prompt, V4.Session keeps a conversation, Agent adds the tool loop, and Toolbox provides the tools. See the interface files in lib/ for the details.

Maintenance #

See ARCH.md for the repository layout, the vendored engine and how to update it, the FFI, and how to quantise a DeepSeek release yourself.

Clone this repository

ocaml-deepseek #