A CLI for Semble link crawling and connection making
README.md

assemblinker #

assemblinker is a command-line tool for turning a web page into a network of Semble cards and connections on the AT Protocol.

Given a URL, it:

  1. Looks up whether the URL is already collected in Semble.
  2. Creates (or reuses) a network.cosmik.card record for the page on your PDS.
  3. Crawls the page for outbound links on different domains.
  4. Fetches each remote page and caches its title, Open Graph description, or a 300-character body excerpt.
  5. Shows you the first 10 outbound links one-by-one so you can:
    • connect – create a RELATED connection from the source URL.
    • save – save the remote URL as a new card and connect it to the source.
    • skip – do nothing.

Install #

npm install
npm run build

This installs dependencies, pulls the network.cosmik.* lexicons via @atcute/lex-cli, and builds the CLI to dist/.

Environment #

Copy .env.example to .env and fill in at least one authentication option.

cp .env.example .env
Variable Purpose
SEMBLE_API_KEY Used to check whether a URL is already collected in Semble. Create one at https://semble.so/settings/api-keys.
ATP_HANDLE Your ATProto handle (e.g. you.bsky.social).
ATP_PASSWORD An app password for that account.
PDS_URL The PDS to authenticate against, e.g. https://bsky.social.

If ATP_HANDLE, ATP_PASSWORD, and PDS_URL are not set, the CLI falls back to a browser-based ATProto OAuth loopback flow powered by @atcute/oauth-node-client. We recommend doing browser based OAuth, it's fun!

Usage #

Simple mode, just start assemblinking:

npm start -- https://bad-example.com/

You can also call the cli directly with a few commandline switches:

node dist/cli.js https://example.com/article
node dist/cli.js https://example.com/article --max 5   # links per batch
node dist/cli.js https://example.com/article --api-key $SEMBLE_API_KEY

Links are reviewed in batches of 10 (or the value of --max). After each batch, you'll be prompted to continue reviewing the remaining links.

Development #

npm run dev -- <url>        # run via tsx without building
npm run typecheck           # run TypeScript
npm run build               # build the CLI
npm run generate            # re-pull cosmik lexicons and regenerate types

Project layout #

  • src/cli.ts – main flow and argument parsing
  • src/auth.ts – password or OAuth session handling
  • src/pds.ts – PDS record creation for cards and connections
  • src/semble.ts – Semble API library-status lookup
  • src/crawler.ts – page fetch, metadata extraction, and outbound link discovery
  • src/cache.ts – global JSON cache at ~/.config/assemblinker/cache.json
  • src/interactive.ts – one-by-one CLI prompts
  • src/lexicons/ – generated atcute types from cosmik lexicons
  • lexicons/ – raw cosmik lexicon JSON files

Cache #

All fetched outbound link metadata and the results of each run (saved/connected URIs) are written to ~/.config/assemblinker/cache.json. If a URL is already saved or collected, it will still be presented as long as it is not already connected to the current source URL.

License #

MIT – see LICENSE.