+++
title = "Fetch Command"
description = "Download lexicons from remote repositories"
weight = 7
+++

The `mlf fetch` command downloads ATProto lexicons from remote repositories and converts them to MLF format.

## Usage

```bash
# Fetch all dependencies from mlf.toml (use lockfile if present)
mlf fetch

# Fetch and update all dependencies to latest versions
mlf fetch --update

# Strict mode: fetch from lockfile only (for CI/CD)
mlf fetch --locked

# Fetch a specific lexicon
mlf fetch <NSID>

# Fetch all lexicons matching a wildcard
mlf fetch <PATTERN>

# Fetch and save to dependencies
mlf fetch <NSID> --save
```

**Arguments:**
- `[NSID]` - Optional NSID or pattern to fetch:
  - Specific lexicon: `com.example.forum.post`
  - All descendants: `com.example.forum.*` (matches `post`, `comment`, `comment.reply`, etc.)
  - Direct children only: `com.example.forum._` (matches `post`, `comment`, but NOT `comment.reply`)
  - Real-world example: `app.bsky.feed.*`

**Options:**
- `--save` - Add the NSID/pattern to dependencies in `mlf.toml`
- `--update` - Update dependencies to latest versions (ignores lockfile)
- `--locked` - Require lockfile and fail if dependencies need updating (for CI/CD)

## Lockfile (`mlf-lock.toml`)

MLF uses a lockfile to ensure reproducible builds, similar to `package-lock.json` (npm) or `Cargo.lock` (Rust).

### Lockfile Format

```toml
version = 1

[lexicons."place.stream.richtext.facet"]
nsid = "place.stream.richtext.facet"
did = "did:web:stream.place"
checksum = "sha256:72c8986132821c7c6e3bd30d697f017861d77867b358e3c7850c19baef0a50d5"
dependencies = ["app.bsky.richtext.facet#byteSlice"]

[lexicons."app.bsky.richtext.facet"]
nsid = "app.bsky.richtext.facet"
did = "did:plc:4v4y5r3lwsbtmsxhile2ljac"
checksum = "sha256:db59d218c482774e617bb5d90d19ab75e2557f8cdebafe798be01b37d957d336"
```

### Fetch Modes

| Mode | Command | Behavior |
|------|---------|----------|
| **Fresh** | `mlf fetch` | No lockfile exists, performs full DNS lookup and fetch, creates lockfile |
| **Lockfile** | `mlf fetch` | Uses existing lockfile to guide fetch, updates lockfile if dependencies change |
| **Update** | `mlf fetch --update` | Ignores lockfile, refetches everything, updates lockfile with latest versions |
| **Locked** | `mlf fetch --locked` | Strict CI mode, uses only lockfile, verifies checksums, fails if no lockfile exists |

### When to Use Each Mode

- **Development**: Use `mlf fetch` (default) - respects lockfile for consistency
- **Update deps**: Use `mlf fetch --update` - gets latest versions
- **CI/Production**: Use `mlf fetch --locked` - ensures reproducible builds

## Transitive Dependencies

MLF automatically resolves and fetches transitive dependencies (dependencies of dependencies).

### Example

If `place.stream.richtext.facet` depends on `app.bsky.richtext.facet#byteSlice`, MLF will:
1. Fetch `place.stream.richtext.*` (your explicit dependency)
2. Parse the lexicons to find external references
3. Automatically fetch `app.bsky.richtext.*` (transitive dependency)
4. Record both in `mlf-lock.toml`

### Configuration

Control transitive dependency resolution in `mlf.toml`:

```toml
[dependencies]
dependencies = ["place.stream.*"]

# Enable/disable transitive dependency resolution (default: true)
allow_transitive_deps = true

# Enable/disable fetch optimization (default: false)
# When true, tries to collapse similar NSIDs into wildcards
optimize_transitive_fetches = false
```

## How It Works

The fetch command uses ATProto's lexicon discovery protocol to locate and download lexicons via DNS lookups and DID resolution, then converts them to MLF format and updates the lockfile.

## Examples

### Fetch All Dependencies

With an `mlf.toml` file:

```toml
[dependencies]
dependencies = ["com.example.forum.*", "com.example.social.*"]
```

Run:

```bash
mlf fetch
```

**Output:**
```
Fetching 2 dependencies... (mode: fresh, transitive deps: enabled)

Fetching: com.example.forum.*
Fetching lexicons for pattern: com.example.forum.*
  → Resolved DID: did:web:example.com
  → Using PDS: https://example.com
  → Found 3 lexicon record(s)
  Processing: com.example.forum.post
    → Saved JSON to .mlf/lexicons/json/com/example/forum/post.json
    → Converted to MLF at .mlf/lexicons/mlf/com/example/forum/post.mlf
✓ Successfully fetched 2 lexicon(s) for com.example.forum.*

→ Updated mlf-lock.toml

✓ Successfully fetched all 2 dependencies
```

### Update to Latest Versions

```bash
mlf fetch --update
```

This ignores the lockfile and fetches the latest versions of all dependencies.

### CI/CD with Locked Mode

```bash
mlf fetch --locked
```

**Output:**
```
Using locked dependencies from mlf-lock.toml
Fetching 2 lexicon(s) from lockfile...

Refetching: place.stream.richtext.facet
  → Using PDS: https://stream.place
  → Saved JSON (checksum verified)
  → Converted to MLF

✓ Successfully fetched all 2 lexicons
```

If no lockfile exists:
```
✗ No lockfile found. Run `mlf fetch` first to create mlf-lock.toml
```

### Fetch Specific Lexicon

```bash
mlf fetch com.example.forum.post
```

This downloads only the `com.example.forum.post` lexicon.

### Fetch with Wildcards

**All descendants (`.*`):**
```bash
mlf fetch com.example.forum.*
```
Downloads all lexicons under the `com.example.forum` namespace (recursive).

**Example:** If the namespace contains:
- `com.example.forum.post`
- `com.example.forum.comment`
- `com.example.forum.comment.reply`

The `.*` pattern fetches **all three**.

**Direct children only (`._`):**
```bash
mlf fetch com.example.forum._
```
Downloads only direct children of `com.example.forum` (non-recursive).

Using the same example namespace, the `._` pattern fetches:
- `com.example.forum.post` ✓
- `com.example.forum.comment` ✓
- `com.example.forum.comment.reply` ✗ (skipped, not a direct child)

### Fetch and Save

```bash
mlf fetch com.example.forum.* --save
```

This:
1. Downloads all `com.example.forum.*` lexicons
2. Adds `"com.example.forum.*"` to the dependencies array in `mlf.toml`
3. Creates `mlf.toml` if it doesn't exist



## Best Practices

1. **Commit lockfile** - Always commit `mlf-lock.toml` to version control
2. **Use --locked in CI** - Ensures reproducible builds in CI/CD pipelines
3. **Don't commit `.mlf/`** - Let each developer fetch independently
4. **Update explicitly** - Use `mlf fetch --update` when you want latest versions

## Comparison with npm/cargo

The fetch command follows patterns from popular package managers:

| Aspect | npm | Cargo | MLF |
|--------|-----|-------|-----|
| Install deps | `npm install` | `cargo build` | `mlf fetch` |
| Update deps | `npm update` | `cargo update` | `mlf fetch --update` |
| Strict mode | `npm ci` | `cargo build --locked` | `mlf fetch --locked` |
| Add dep | `npm install pkg` | `cargo add pkg` | `mlf fetch ns --save` |
| Config file | `package.json` | `Cargo.toml` | `mlf.toml` |
| Lock file | `package-lock.json` | `Cargo.lock` | `mlf-lock.toml` |
| Cache | `node_modules/` | `~/.cargo/` | `.mlf/` |