ptr.pet / hydrant
very fast at protocol indexer with flexible filtering, xrpc queries, cursor-backed event stream, and more, built on fjall
rust fjall at-protocol atproto indexer
Star
Fork 3
Watch

Commits on test-13-fix

wip test-13-fix
dawn committed
[tests] use plc.klbr.net instead of plc.wtf
dawn committed
[ingest] change the unbounded channels to be bounded
dawn committed
[backfill] dont hold store / mst tree over db commit, drop early when we can
dawn committed
[lib] expose backlinks properly, add dids field support
dawn committed
[ingest,stream] dont read from db, use live event
dawn committed
[db] compact manually if we pruned enough events
dawn committed
[deps] update fjall and friends
dawn committed
[docs] update verbiage
dawn committed
[db] make count tracking more durable
dawn committed
[backfill] fix count tracking on resync
dawn committed
[docs] point to docs
dawn committed
[flake] remove conflicting rust deps from devshell override
dawn committed
[docs] update verbiage
dawn committed
[docs] move title headers into title frontmatter
dawn committed
[docs] fix rewrites
dawn committed
[db] improve ephemeral task, add test for the relay ttl to the gc test
dawn committed
[db] print pruned events log at info level
dawn committed
[docs] update links in docs readme
dawn committed
[docs] add building and deployment
dawn committed
[docs] add request body tables, restore config descriptions from readme
dawn committed
[docs] convert nested lists to headings in api docs
dawn committed
[flake] set VERBIAGE_TITLE=hydrant, update verbiage input
dawn committed
[ingest] store resync buffer commits with named tags
dawn committed
[docs] update verbiage to latest (redirect fix)
dawn committed
[docs] initial documentation + add verbiage to devshell
dawn committed
[firehose] periodically retry offline hosts (every 30 minutes)
dawn committed
[firehose,lib,api] add is_source_running, shortcircuit requestCrawl if source is already running to prevent task restart
dawn committed
[lib,api] add new host limit for hosts added via requestCrawl
dawn committed
[config] enable ephemeral mode by default in relay mode
dawn committed
[docs] remove gitnexus stuff from agents.md
dawn committed
[all] add indexer_stream feature to decouple the events stream from the indexer
dawn committed
[all] simplify code depending on block storage being enabled
dawn committed
[all] dont thread ephemeral bool through the code, use from app state
dawn committed
[indexer] add config option for only indexing links
dawn committed
[ingest,lib,api] add glob rules for pds tiers, TRUSTED_HOSTS -> TIER_RULES
dawn committed
[api] make sure cids are serialized as text in listRepos
dawn committed
[api] fix index page to show indexer if compiled with indexer
dawn committed
[api] also add _health
dawn committed
[firehose] jitter connection startup when we are spawning persisted / configged firehoses
dawn committed
[firehose] use cancel token instead of aborting task, fix bug where upserted tasks would remove the new task from the map
dawn committed
[ingest] make throttling less strict on firehose errors to better match ref behaviour
dawn committed
[lib,api] fix iterating through repos and getting repo info in relay mode
dawn committed
[docs] remove unnecessary method from safe to proxy endpoints
dawn committed
[api] add index file
dawn committed
[build] add rust toolchain toml and update flake to use nci for devshell
dawn committed
[lib] re-export public api deps in deps
dawn committed
[build] add dockerfile
dawn committed
[docs] better instructions for hosting in the readme
dawn committed
[ingest,lib,api] implement host status tracking for pds' and enforce account limit
dawn committed
[docs] update readme
dawn committed
[api] actually reject banned pds in requestCrawl
dawn committed
[firehose] only reset backoff to 0 if the stream has been healthy for over a minute
dawn committed
[firehose] send pings to hosts
dawn committed
[firehose] add jitter on throttles / backoffs
dawn committed
[firehose] print better stream closed errors
dawn committed
[firehose] treat every upgrade error as throttle reason
dawn committed
[firehose] 4xx is also throttle worthy now, also handle rustls AlertReceived better
dawn committed
[firehose,crawler] treat 1xx,2xx,3xx,5xx status codes as server error (aside from crawler, where 500 and 2xx is allowed)
dawn committed
[firehose] respond to pings, better treat close messages
dawn committed