mem

Fork 0

Go 99%

Python <1%

Nix <1%

Shell <1%

24 1 0

Branches 1

main · 1mo ago Default

README.md

lard #

memory layer for my homelab; basically just a place to chuck data into from a bunch of different llm sources and then consolidate it

The canonical repo for this is hosted on tangled over at dunkirk.sh/lard

Running it #

# server: copy the example config, add your API key
cp config.example.toml ~/.config/lard/config.toml
# edit ~/.config/lard/config.toml, set llm.api_key
go run ./cmd/lard                    # listens on :7477

# client: point it at the server, then load everything you have
lard-client login                    # asks for the url, runs the device grant
lard-client backfill --root ~/code

# keep it fed in the background (macOS)
lard-client service install

client #

lard-client login    [--url URL] [--token TOKEN] [--root DIR...] [-f]
lard-client logout                        # revoke + forget credentials
lard-client status                        # server, auth, agent at a glance
lard-client backfill [--root DIR...]      # every session ever, idempotent
lard-client sync     [--workspace DIR...] # new sessions only
lard-client daemon   [--interval 5m]      # sync in a loop, for non-macOS init
lard-client service  install|uninstall|status
lard-client consolidate                   # force a pass now

server #

lard                        run the server
lard backup <dir>           copy every store into <dir>, live, without stopping
lard restore <dir> [-f]     put a backup tree back where the server reads it

Interfaces #

MCP at POST /mcp: get_context, memory_list, memory_read, memory_write, memory_append, memory_delete. add to crush:

{
    "mcp": {
        "lard": {
            "type": "http",
            "url": "https://lard.your.domain/mcp",
            "oauth": true
        }
    }
}

HTTP

GET    /healthz                liveness check (no auth)
GET    /whoami                 verify credentials; returns the caller's identity
GET    /context?project=<id>   profile + subject listing + this project's area
GET    /memory                 the subject listing
GET    /memory/{path}          a subject's markdown body
PUT    /memory/{path}          create or overwrite a subject
POST   /memory/{path}          append a line
DELETE /memory/{path}          delete a subject
POST   /ingest                 upload sessions
POST   /consolidate            trigger a pass
POST   /projects/resolve       hints → canonical project id
GET    /projects               project registry

paths are profile, areas/<name>, topics/<name>, people/<name>.

Configuration #

The server reads ~/.config/lard/config.toml (override with LARD_CONFIG). Every option can also be set as an environment variable; env always wins. See config.example.toml for a ready-to-edit starting point.

TOML key	env var	default	desc
`addr`	`LARD_ADDR`	`:7477`	listen address
`db`	`LARD_DB`	`~/.config/lard/lard.db`	sqlite path (sessions, facts, registry)
`memory_dir`	`LARD_MEMORY_DIR`	`~/.config/lard/memory`	subject files
`multi_user`	`LARD_MULTI_USER`	`false`	one isolated store per authenticated identity
`data_dir`	`LARD_DATA_DIR`	`~/.config/lard/users`	where tenant directories live
`primary_user`	`LARD_PRIMARY_USER`		identity owning requests with no OAuth identity
`llm.base_url`	`LARD_HYPER_BASE_URL`	`https://hyper.charm.land`	OpenAI-compatible endpoint for consolidation
`llm.model`	`LARD_MODEL`	`deepseek-v4-flash`	consolidation model
`llm.api_key`	`LARD_HYPER_API_KEY`		API key for consolidation (falls back to `HYPER_API_KEY`)
`auth.mode`	`LARD_AUTH`	`none`	`none` \| `token` \| `oauth`
`auth.token`	`LARD_TOKEN`		shared secret for `token` mode
`auth.auth_server`	`LARD_AUTH_SERVER`		authorization server URL for `oauth` mode
`auth.public_url`	`LARD_PUBLIC_URL`		lard's external url; goes in the OAuth metadata
`auth.allowed_users`	`LARD_OAUTH_USERS`		comma list of `me` urls allowed to call lard
`auth.required_scopes`	`LARD_OAUTH_SCOPES`		comma list of scopes every token must carry
`auth.resource_name`	`LARD_RESOURCE_NAME`	`lard`	friendly name shown on the AS consent screen (RFC 9728)
`auth.logo_uri`	`LARD_LOGO_URI`	(lard logo)	icon shown on the AS consent screen; `""` for none
`consolidate.after`	`LARD_CONSOLIDATE_AFTER`	`5m`	quiet period before a pass; `off` to disable
`consolidate.max_wait`	`LARD_CONSOLIDATE_MAX_WAIT`	`30m`	cap on that wait during constant uploads

multi-user #

Off by default: one server, one memory. Turn it on and every authenticated identity gets its own SQLite database and its own directory of subject files under data_dir. Nothing crosses between them, and no query in the store layer knows users exist.

multi_user = true
data_dir = ""                            # ~/.config/lard/users
primary_user = "https://you.example"

[auth]
mode = "oauth"                           # the identity comes off the token

Tenant directories are named <readable>-<hash>, derived from the identity's me url:

users/
  dunkirk-sh-8f21c0a3b7de/
    lard.db
    memory/{profile.md,areas/,topics/,people/}

Notes worth knowing before you flip it:

oauth mode is what makes this work. token and none put no identity on a request, so every caller would land on primary_user. lard warns at boot if you do that, and refuses to start if primary_user is also unset.
Your existing memory is adopted, not stranded. On the first multi-user boot, an existing db and memory_dir are moved into primary_user's tenant (wal and shm files included). It happens once and never overwrites a tenant that already exists. Turning multi_user back off does not move it back, so lard refuses to start rather than quietly serving a fresh empty database; the error names the directory to move.
Consolidation is per user. Each tenant gets its own quiet timer, so one person uploading an afternoon of sessions doesn't reset anyone else's clock.
auth.allowed_users still gates the door. Multi-user decides where an accepted caller's memory lives, not who is accepted. Leave the allowlist empty and anyone your authorization server vouches for gets a tenant.
Tenants stay open for the life of the process, one SQLite connection each. That is the right trade for a homelab; it is not a design for thousands of users.

backups #

lard backup /var/lib/lard/backup    # every store, live, nothing stops

The destination mirrors the live layout, so restoring is a move rather than a puzzle. Point your backup tool at that directory instead of at the running data directory.

backup/
  lard.db                       # single-user
  memory/
  users/<slug>/{lard.db,memory} # multi-user, one per tenant

Databases are copied with SQLite's VACUUM INTO, which reads the source inside one transaction, so the copy is a single point in time no matter what is being written meanwhile. This is the part worth being careful about: copying a database file directly, which is what a backup tool does, can read page 1 before a write and page 900 after it, giving you a file that was never a valid database. Checkpointing the WAL first does not help, because it says nothing about the writes that follow. That risk is the only real reason to stop a service for backup, and VACUUM INTO removes it.

Subject files are copied plainly, which is safe because every write to them lands via a rename. A .tmp file seen mid-walk is a write in flight and is skipped; the finished version is either already in the tree or in the next run.

restoring #

A backup tool restores the paths it archived, and what it archived is the staging directory, not the live one. So the last step is putting the tree back where the server reads it:

restic restore latest --target /tmp/r
lard restore /tmp/r/var/lib/lard/backup     # stop lard first

lard restore reads the same config the server does, so it knows whether to lay the tree out as one store or as tenants, and it refuses a backup of the wrong shape rather than putting data somewhere nothing reads it.

It will not overwrite data that is already there without --force, and even then nothing is deleted: what was there is renamed to <path>.superseded-<timestamp> first. Rolling back to last night is exactly when today might still be wanted.

auth #

token mode is a shared secret,oauth mode makes lard an OAuth 2.1 protected resource in front of any authorization server that supports introspection (rfc 7662) and serves OAuth metadata (rfc 8414). lard serves

/.well-known/oauth-protected-resource (rfc 9728) naming the authorization server
/.well-known/oauth-authorization-server as a redirect to the authorization server, so older mcp clients still discover it. a redirect rather than a proxy because clients check that the issuer matches where they fetched the document

every token must name lard in its audience (rfc 8707). that one check is the whole authorization model: a token minted for some other app the user authorized names that app, so it bounces here, and there is no client allowlist to keep in sync. clients ask for the audience by sending resource=<lard's url>, which they read out of the protected-resource document.

clients that need an identity register their own (rfc 7591) at the authorization server. lard hands out nothing and knows none of them by name. lard-client registers as a public client: the device code is its own proof of possession, so a login is one command on a fresh box with nothing to copy over.

lard-client registers itself once, then runs the OAuth device authorization grant (rfc 8628):

POST {as}/oauth/register           this machine claims its own client identity
POST {as}/auth/device              client gets a device code + user code + url
GET  {as}/device?code=XXXX-XXXX    user approves, from any browser anywhere
POST {as}/auth/token               client polls until the token appears

requirements on the provider: rfc 8414 metadata advertising device_authorization_endpoint and registration_endpoint, the device grant, and rfc 8707 resource indicators echoed as aud on introspection.

Clone this repository