llm-bridge is a self-hosted inference gateway. It exposes OpenAI and Anthropic APIs, discovers the models available through configured providers, and routes each request to the matching backend.

It supports:

Anthropic, Codex, and Google Antigravity subscription accounts through OAuth
OpenAI-compatible API providers and providers from the pi-ai registry
OpenAI Chat Completions and Responses APIs
Anthropic Messages and token-counting APIs
image generation and editing, audio transcription, and WebRTC Realtime call creation
Exa-compatible search backed by Codex
multiple provider accounts with per-key routing and conversation affinity
usage accounting, quota-aware account selection, conversation compaction, and an admin UI

Requirements #

Node.js 22.12 or newer
npm 10 or newer

Node 22.12 is required for the built-in node:sqlite module.

Quick start #

Install dependencies:

npm ci

Create config.toml with at least one provider:

host = "127.0.0.1"
port = 4040

[providers.openrouter]
base_url = "https://openrouter.ai/api/v1"
api_key_file = "/run/secrets/openrouter-key"

Start the bridge:

npm start -- --config ./config.toml

The server listens at http://127.0.0.1:4040. Without a keys file, it accepts unauthenticated requests only when bound to a loopback address.

Check the server and list its discovered models:

curl http://127.0.0.1:4040/health
curl http://127.0.0.1:4040/v1/models

Send an OpenAI Chat Completions request:

curl http://127.0.0.1:4040/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{
    "model": "MODEL_ID",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Use the bridge with an OpenAI client by setting its base URL to http://127.0.0.1:4040/v1. Set an Anthropic client's base URL to http://127.0.0.1:4040; it will call /v1/messages.

Providers #

Providers are configured under [providers.<name>]:

[providers.openai]
api_key_file = "/run/secrets/openai-key"
base_url = "https://api.openai.com/v1"

[providers.openrouter]
api_key_file = "/run/secrets/openrouter-key"
base_url = "https://openrouter.ai/api/v1"

Each provider accepts these fields:

Field	Purpose
`api_key`	API key stored directly in the config
`api_key_file`	File containing the API key
`base_url`	Provider API base URL
`auth_path`	OAuth credential file for a subscription provider
`model`	Default Codex model
`reasoning_effort`	Default Codex reasoning effort

Use api_key_file instead of api_key for deployed services. For OpenAI-compatible providers with base_url and a key, the bridge discovers models from the provider's /models endpoint. For providers supported by pi-ai, it registers models when the provider has credentials in the config or environment.

The complete root configuration is:

host = "127.0.0.1"
port = 4040
codex_fast_max_output_cost_per_mtok = 20

[providers.openrouter]
base_url = "https://openrouter.ai/api/v1"
api_key_file = "/run/secrets/openrouter-key"

The corresponding environment variables are:

Variable	Purpose
`BRIDGE_HOST`	Listen address; default `127.0.0.1`
`BRIDGE_PORT`	Listen port; default `4040`
`BRIDGE_PROVIDERS`	JSON provider object when no providers are present in TOML
`BRIDGE_STATE_DIR`	Persistent state directory
`CODEX_FAST_MAX_OUTPUT_COST_PER_MTOK`	Maximum output price for advertised Codex `:nitro` aliases
`BRIDGE_MODEL_REFRESH_MS`	Subscription model refresh interval

TOML settings take precedence over environment variables.

API keys #

Pass --keys to require bridge authentication:

npm start -- --config ./config.toml --keys ./keys.json

keys.json may contain one object or an array:

[
  {
    "key": "11111111-2222-4333-8444-555555555555",
    "name": "operator",
    "allowedModels": ["*"],
    "caps": ["admin"]
  }
]

key must be a UUID. allowedModels contains case-insensitive globs with * and ?. The admin capability grants access to the admin API and UI. Model access and admin access are separate; allowedModels: ["*"] does not grant admin.

Clients can authenticate with either header:

Authorization: Bearer 11111111-2222-4333-8444-555555555555

x-api-key: 11111111-2222-4333-8444-555555555555

A non-loopback bind requires at least one API key. The bridge refuses to start as an unauthenticated public proxy.

The admin UI can create, edit, and delete keys when the file passed to --keys is a writable regular file. Key writes are atomic and use mode 0600.

Subscription accounts #

Open http://127.0.0.1:4040/admin. When API keys are enabled, sign in with a key that has the admin capability. Add one or more accounts for:

Claude — Anthropic authorization-code flow
Codex — OpenAI device-code flow
Antigravity — Google authorization-code flow

The UI stores each account separately, refreshes its credentials, and updates the model catalog after login. It also manages per-key account pools, routing policies, compaction, API keys, model metadata, usage, and upstream errors.

A Codex account adds its live Codex model catalog. Eligible models also receive a :nitro alias for Codex Fast mode. A live Codex account can expose these media aliases:

codex-image and gpt-image-2
codex-transcribe and supported OpenAI transcription model IDs
codex-realtime and supported OpenAI Realtime model IDs

An Antigravity account adds its live Gemini catalog. Models ending in -tiered also receive an alias without that suffix.

Existing credential files can be configured directly:

[providers.anthropic-oauth]
auth_path = "/path/to/pi-ai/auth.json"

[providers.codex]
auth_path = "/path/to/codex/auth.json"

If no Codex path is configured, the bridge also checks CODEX_AUTH_PATH and ~/.codex/auth.json.

Inference APIs #

The bridge exposes these client-facing routes:

Method	Route	API
`GET`	`/health`	Health and model count
`GET`	`/v1/models`	Models available to the current key
`POST`	`/v1/chat/completions`	OpenAI Chat Completions
`POST`	`/v1/responses`	OpenAI Responses
`POST`	`/v1/messages`	Anthropic Messages
`POST`	`/v1/messages/count_tokens`	Anthropic token counting
`GET`	`/v1/usage?model=MODEL_ID`	Normalized provider usage
`POST`	`/v1/images/generations`	OpenAI image generation
`POST`	`/v1/images/edits`	OpenAI image editing
`POST`	`/v1/audio/transcriptions`	OpenAI audio transcription
`POST`	`/v1/realtime/calls`	OpenAI WebRTC call creation
`POST`	`/v1/realtime/transcription_sessions`	OpenAI transcription session creation
`POST`	`/search`	Exa-compatible search

Chat Completions, Responses, and Messages requests can use any discovered model. The bridge translates between API formats when the selected backend uses a different protocol. Streaming and tool calls are supported across the main text APIs.

Exa-compatible search #

POST /search uses a connected Codex account and accepts the common Exa Search request shape:

curl http://127.0.0.1:4040/search \
  -H 'content-type: application/json' \
  -H 'x-api-key: YOUR_BRIDGE_KEY' \
  -d '{
    "query": "latest work on coding agents",
    "type": "auto",
    "numResults": 5,
    "includeDomains": ["github.com"],
    "contents": {"text": true},
    "context": true
  }'

The optional model field selects an allowed Codex model. Without it, the bridge selects an available Codex reasoning model.

Account routing #

Each API key can use a separate pool of Anthropic, Codex, and Antigravity accounts. Four selection policies are available:

Policy	Behavior
`fill-first`	Use the first available account until its quota is exhausted
`lowest-usage`	Select the account with the lowest known quota usage
`round-robin`	Rotate new conversations across available accounts
`specific-plan-first`	Prefer a selected account, then fall back to the pool

Conversations stay on their selected account. Send a stable identifier on every turn for explicit affinity:

x-llm-bridge-conversation-id: conversation-123

The Responses API conversation field and OpenAI prompt_cache_key are also recognized. Without an explicit identifier, the bridge derives one from the opening request content.

When a conversation moves to another account, the bridge carries portable context across the account boundary. Per-key compaction supports three strategies:

remote — provider-native compaction
model-driven — a model-generated summary plus recent turns
snapcompact — local SnapCompact frames plus recent turns

Compaction can also run at a configured context percentage or after an idle interval. Conversation state is retained for the configured number of days.

State and backups #

The state directory is selected in this order:

The first non-empty component of STATE_DIRECTORY
BRIDGE_STATE_DIR
$HOME/.local/share/llm-bridge

It contains:

<state_dir>/accounts.json
<state_dir>/accounts/<account-uuid>.json
<state_dir>/usage.sqlite

Account files contain OAuth credentials. usage.sqlite contains token usage, routing state, conversation snapshots, and recent upstream errors. The directory uses mode 0700; credential and database files use mode 0600.

Back up the complete state directory with the service stopped. Include usage.sqlite-wal and usage.sqlite-shm if present. The state directory contains API credentials and conversation data.

NixOS #

The flake exports nixosModules.default and nixosModules.llm-bridge:

{
  inputs.llm-bridge.url = "git+https://tangled.org/okami.mom/llm-bridge";

  outputs = { self, nixpkgs, llm-bridge, ... }: {
    nixosConfigurations.host = nixpkgs.lib.nixosSystem {
      modules = [
        llm-bridge.nixosModules.default
        {
          services.llm-bridge = {
            enable = true;
            keysFile = "/run/secrets/llm-bridge-keys.json";
            settings = {
              host = "0.0.0.0";
              port = 4040;
              providers.openrouter = {
                base_url = "https://openrouter.ai/api/v1";
                api_key_file = "/run/secrets/openrouter-key";
              };
            };
          };
        }
      ];
    };
  };
}

The module runs the service as llm-bridge and stores state in /var/lib/llm-bridge.

Development #

Run the backend and Vite admin UI together:

npm run dev -- --config ./config.toml --keys ./keys.json

The backend listens on port 4040 and the Vite UI is available at http://localhost:5173/admin/.

Run the admin UI with in-memory development data and no backend:

npm run mock

Run checks:

npm run typecheck
npm test

Build the admin UI:

npm run build:admin

Clone this repository

llm-bridge #