cuducos.me / mutualsky
👯 Are we mutuals or are we dancer? mutualsky.cuducos.me
Star 2
Fork 0
Watch
Go 100%
Dockerfile <1%
5 1 0
.tangled
.gitignore
AGENTS.md
api_test.go
api.go
consume_test.go
consume.go
did.go
Dockerfile
go.mod
go.sum
main.go
metrics_test.go
metrics.go
overlay.go
README.md
resolve_test.go
resolve.go
store_test.go
store.go
storefile.go
README.md

Mutualsky #

Tells whether two Bluesky accounts follow each other.

The graph is built from Jetstream app.bsky.graph.follow events — no Bluesky API involved.

Requirements #

Running #

$ go run .

With no stored sequence the stream replays the whole Jetstream archive; otherwise it resumes from the last stored sequence.

Try --help for more details.

Endpoint #

Responses carry no body.

POST /{a} #

Accept a JSON array of strings, each string is a DID or handle. Checks whether {a} (a DID or handle) is mutual with each of the other users.

A successful response contains two arrays in the same order of the input:

{
  mutuals: bool[],
  errors?: (string | null)[],
}

GET /{a}/{b} #

Whether two accounts follow each other. a and b are DIDs or handles — handles resolve through the _atproto DNS TXT record or the /.well-known/atproto-did file, whichever answers first (cached for an hour).

Response Meaning
200 they follow each other
404 they don't follow each other
422 handle does not resolve
400 malformed DID or handle

GET /ready #

Whether the stream has caught up with live events.

Response Meaning
200 caught up with live events
500 still backfilling

Contributing #

$ gofmt -l .
$ golangci-lint run
$ go test ./...

On-disk formats #

All state lives under the -db directory. CURRENT names the active generation (g0 or g1); each generation is an immutable snapshot, memory-mapped read-only. The wal holds recent ops. Compaction writes the inactive generation from scratch, fsyncs it, renames CURRENT to point at it, then truncates the wal. Integers are big-endian.

File What it stores Layout
plc.dat Interned did:plc accounts: decoded id to uid 19 B rows: 15 B id, 4 B uid; sorted by id
oth.dat Interned non-plc accounts (did:web, ...): hash to uid 20 B rows: 16 B SHA-256 prefix of the DID, 4 B uid; sorted by hash
oth.str, oth.off Original DIDs behind the oth.dat hashes oth.str stores each DID length-prefixed, one per row; oth.off stores one 8 B offset per row into oth.str, so a hash hit is checked against the actual DID
tmb.dat Deleted accounts 4 B uid rows; sorted
tid.dat Follow records: what each actor's rkey points at 16 B rows: 4 B actor uid, 8 B numeric rkey, 4 B subject uid; sorted by (actor, rkey)
adj.dat, adj.off Who each account follows adj.dat concatenates one blob per account: the uids it follows, ascending, delta-varint encoded (each uid stored as its difference from the previous). adj.off indexes the blobs with 12 B rows: 4 B uid, 8 B blob offset; sorted by uid. A blob spans its offset to the next row's offset (end of file for the last)
meta Checkpoint 13 B: 1 B seq-present flag, 8 B last Jetstream seq, 4 B next uid to hand out
wal Ops since the last compaction Records are uvarint payload length, then op kind, uvarint seq, and length-prefixed string fields

How they fit together:

  • Follows are recorded once, in tid.dat; adj.dat is the read-side index over the same data, rebuilt from it at compaction. Lookups (mutual, batch) only read adj.dat, never tid.dat.
  • Deleting or retargeting a follow needs tid.dat because delete events carry only the rkey: the current subject is resolved there before the edge changes. Canonical TIDs decode to their 8 B integer; other rkeys hash into a range TIDs never use.
  • Compaction keeps the tid rows minus deletions and tombstones, drops every edge to a tmb.dat uid, and regenerates adj.* from what remains.
  • After a crash the process opens the generation named in CURRENT, replays wal (truncating a torn tail), and resumes Jetstream from the recovered seq, so nothing is re-fetched from the archive.