diff --git a/README.md b/README.md index e2e6878..c8ab7fb 100644 --- a/README.md +++ b/README.md @@ -6,8 +6,11 @@ The runtime library and all generated client code are pure C11. The optional Lexicon generator is a development-time Python tool and is never linked into, embedded in, or required by applications using `libwolfram`. -**Status:** Early development — core transport, identity, repo, agent, OAuth, -and sync layers are implemented and tested, but it is not yet at full feature +**Status:** Broad, multi-layer coverage is implemented and tested — transport +(XRPC/WebSocket), identity (DID/handle), repo (DAG-CBOR/CAR/MST), agent +(wraps `com.atproto.*` + chat/ozone/moderation), OAuth (DPoP/PAR), sync +(firehose + Jetstream), moderation, DID PLC ops, rich text, syntax/validate/json, +and optional SQLite store persistence — though it is not yet at full feature parity with the official SDKs. ## Documentation @@ -17,6 +20,7 @@ Per-module usage guides (runnable C snippets): - [`docs/agent.md`](docs/agent.md) — high-level `wf_agent_*` API - [`docs/sync.md`](docs/sync.md) — repo CAR, firehose, commit verification - [`docs/validate.md`](docs/validate.md) — `wf_validate_value` / `wf_validate_record` +- [`docs/moderation.md`](docs/moderation.md) — `wf_mod_*` decision engine - [`docs/oauth.md`](docs/oauth.md) — OAuth/DPoP, PKCE, PAR, callback flow Topic guides: diff --git a/docs/moderation.md b/docs/moderation.md new file mode 100644 index 0000000..327541f --- /dev/null +++ b/docs/moderation.md @@ -0,0 +1,171 @@ +# Moderation engine (`moderation.h`) + +The moderation engine computes blur/alert/inform/filter decisions for subjects +(accounts, profiles, posts, notifications, feed generators, and user lists) from +labels, blocks, mutes, hidden posts, and muted words. It is fully **offline**: +it takes input data as C structs and produces decisions. No network I/O is +performed — you fetch the raw records/labels yourself (e.g. with the +[agent](agent.md) or [sync](sync.md) layers) and hand them to the engine. + +Reference implementation: `atproto/packages/api/src/moderation/` (TypeScript). + +Conventions used throughout: + +- Success is `WF_OK` (the `wf_status` enum); any other value is an error. +- Functions that allocate an owned struct come with a matching `_free` + (e.g. `wf_mod_prefs_free`, `wf_mod_labels_free`, `wf_mod_label_defs_free`, + `wf_mod_decision_free`, `wf_mod_ui_free`). Call it when done. +- Three JSON-ingestion helpers parse API-shaped JSON into engine structs so you + don't have to build the structs by hand. + +## Building preferences from JSON + +A user's `app.bsky.actor.defs#preferences` array (or a bundled +`app.bsky.actor.defs#moderationPrefs` object) parses into `wf_mod_prefs`: + +```c +#include "wolfram/moderation.h" + +/* `json` is the preferences array/object as returned by the API. */ +wf_mod_prefs prefs = {0}; +wf_status st = wf_mod_prefs_from_json(&prefs, json); +if (st != WF_OK) { /* handle error */ } + +/* `prefs` now owns adult_content_enabled, labeler list, global label prefs, + hidden posts, and muted words. Free when done: */ +wf_mod_prefs_free(&prefs); +``` + +Recognized shapes: `adultContentPref`, `labelersPref`, `contentLabelPrefs`, +`hiddenPostsPref`, and `mutedWordsPref` (plus their `moderationPrefs.*` +wrappers). + +## Interpreting a labeler's definitions + +Each labeler publishes `labelValueDefinitions` under +`policies.labelValueDefinitions` of its `app.bsky.labeler.service` record. +Parse them into `wf_mod_label_def` (with computed behaviors/flags/defaults) +via `wf_mod_label_defs_from_labeler`: + +```c +/* `labeler_json` is the labelValueDefinitions array; `labeler_did` is recorded + as `defined_by` on each definition. */ +wf_mod_label_def *defs = NULL; +size_t def_count = 0; +wf_status st = wf_mod_label_defs_from_labeler(labeler_did, labeler_json, + &defs, &def_count); +if (st != WF_OK) { /* handle error */ } + +/* Later: re-interpret a single raw definition if needed: + wf_mod_interpret_label_def(&out, identifier, defined_by, + blurs, severity, adult_only, default_setting); */ + +/* Free the array when done: */ +wf_mod_label_defs_free(defs, def_count); +``` + +You typically repeat this for every labeler the user subscribes to (from +`prefs.labelers`) and collect all definitions into one array for `wf_mod_opts`. + +## Parsing labels + +A `labels` array (each item with `src`, `uri`, `val`, `cts`) parses into +`wf_mod_label` via `wf_mod_labels_from_json`: + +```c +wf_mod_label *labels = NULL; +size_t label_count = 0; +wf_status st = wf_mod_labels_from_json(&labels, &label_count, labels_json); +if (st != WF_OK) { /* handle error */ } + +/* `labels` is now a caller-owned array. Free when done: */ +wf_mod_labels_free(labels, label_count); +``` + +Labels are attached to subject structs (see below) when computing a decision. + +## Computing a decision + +Build a `wf_mod_opts` carrying the user's preferences and the known label +definitions, then call a subject decider. Available deciders: + +- `wf_mod_decide_account` — account (block/mute/account-labels) +- `wf_mod_decide_profile` — profile (profile-labels only) +- `wf_mod_decide_post` — post (content labels + embeds + author) +- `wf_mod_decide_notification` — notification (account + profile + content) +- `wf_mod_decide_feed_generator` — feed generator +- `wf_mod_decide_user_list` — user list +- `wf_mod_decide_status` — account status (deactivated/takendown) + +```c +/* Assemble the options once per user/session. */ +wf_mod_opts opts = { + .user_did = my_did, + .prefs = prefs, /* from wf_mod_prefs_from_json */ + .label_defs = defs, /* from wf_mod_label_defs_from_labeler */ + .label_def_count = def_count, +}; + +/* Describe the post to be moderated. */ +wf_mod_subject_post subject = { + .uri = post_uri, + .cid = post_cid, + .author = { .did = author_did, .handle = author_handle, + .labels = author_labels, .label_count = author_label_count }, + .labels = post_labels, + .label_count = post_label_count, + .text = post_text, + .embed_type = NULL, /* or "app.bsky.embed.record" etc. */ + .embed_uri = NULL, +}; + +wf_mod_decision decision = {0}; +wf_status st = wf_mod_decide_post(&decision, &subject, &opts); +if (st != WF_OK) { /* handle error */ } + +/* Inspect the decision. */ +if (wf_mod_decision_blocked(&decision)) { /* block the post */ } +if (wf_mod_decision_muted(&decision)) { /* mute the post */ } + +/* Map to a UI for a view context. */ +wf_mod_ui ui = {0}; +wf_status st2 = wf_mod_decision_ui(&decision, WF_MOD_CTX_CONTENT_LIST, &ui); +/* ui.blurs / ui.alerts / ui.informs point at owned cause subsets. */ +wf_mod_ui_free(&ui); + +wf_mod_decision_free(&decision); +``` + +Decisions can be merged (`wf_mod_decision_merge`), downgraded +(`wf_mod_decision_downgrade`), and queried for blocking/mute causes. Use +`wf_mod_find_label_def`, `wf_mod_get_label_pref`, and `wf_mod_behavior_get` to +resolve individual labels, and `wf_mod_match_mute_words` to test post text/tags +against the user's muted words. + +## Persisted labels (`store.h`) + +When the library is built with `WOLFRAM_BUILD_STORE=ON`, the optional SQLite +store (`store.h`) lets you persist the labels you've seen so the engine can +moderate content offline: + +```c +#include "wolfram/store.h" + +wf_store *store = NULL; +wf_store_open(&store, "moderation.db"); + +/* Persist a label for a subject (uri + cid + the raw label JSON blob). */ +wf_store_save_label(store, post_uri, post_cid, label_src, label_val, label_cts); + +/* Load all labels attached to a subject. */ +wf_mod_label *loaded = NULL; +size_t loaded_count = 0; +wf_store_load_labels(store, post_uri, &loaded, &loaded_count); +/* ... feed `loaded`/`loaded_count` into a wf_mod_subject_* ... */ +wf_mod_labels_free(loaded, loaded_count); /* store copies its own; free the parse */ + +wf_store_close(store); +``` + +See [modules.md](modules.md) for the build-gating details and the optional +`WOLFRAM_BUILD_STORE_CRYPTO=ON` at-rest encryption (libsodium). diff --git a/docs/modules.md b/docs/modules.md index 9af89b1..fed7a23 100644 --- a/docs/modules.md +++ b/docs/modules.md @@ -14,13 +14,22 @@ | `wolfram/label.h` | Implemented | Label subscription (com.atproto.label.subscribeLabels) via WebSocket | | `wolfram/sync.h` | Implemented | Firehose subscribeRepos subscription, commit verification, CAR download, plus getBlob/getBlocks/getRecord/listBlobs/getHead/getLatestCommit/getRepoStatus/listRepos | | `wolfram/validate.h` | Implemented | Runtime Lexicon schema validation (records and named values), refs/unions/format keywords | -| `wolfram/agent.h` | Implemented | High-level BskyAgent-style API: session, posts, profile, social graph, feeds, **preferences**, **push registration**, notifications, blobs, server + app-password management | +| `wolfram/agent.h` | Implemented | High-level BskyAgent-style API: session, posts, profile, social graph, feeds, **preferences**, **push registration**, notifications, blobs, **video upload** (`wf_agent_upload_video`/`wf_agent_get_video_job_status`/`wf_agent_get_video_upload_limits`), server + app-password management | +| `wolfram/blob.h` | Implemented | Binary blob upload — image/blob POST (`wf_xrpc_upload_blob`) and dedicated **video** upload (`wf_agent_upload_video`, `wf_uploaded_blob_free`) | | `wolfram/chat_typed.h` | Implemented | Chat (DM) — `chat.bsky.convo`/`group`/`actor`/`moderation` write+query wrappers with chat-service endpoint resolution | | `wolfram/ozone.h` | Implemented | Ozone moderation-service / labeler helper (verify/emit labels, auth headers) | +| `wolfram/auth_client.h` | Implemented | Authenticated XRPC client — DPoP-binding OAuth-authenticated query/procedure/blob-upload (`wf_auth_client_*`) with session refresh and DPoP nonce retry | | `wolfram/plc.h` | Implemented | DID PLC operation build/sign/submit helpers (`wf_plc_*`): create/rotate/tombstone, signing-key and handle operations with ES256 signature + verification | | `wolfram/richtext.h` | Implemented | Rich text facets, grapheme detection, mention/link/tag parsing | | `wolfram/syntax.h` | Implemented | DID, handle, NSID, TID, AT URI, RFC 3339, BCP 47 validators | | `wolfram/atproto_lex.h` | Implemented | Generated lexicon endpoint wrappers (13K header, 74K source) | | `wolfram/moderation.h` | Implemented | Moderation decision engine — blur/alert/inform/filter for accounts, profiles, posts, notifications, feed generators, and user lists from labels, blocks, mutes, hidden posts, and muted words | -| `wolfram/store.h` | Partial/Optional | SQLite-backed session + repo-mirror persistence + persisted-label storage for the moderation engine (OFF by default; build with `WOLFRAM_BUILD_STORE=ON`) | +| `wolfram/store.h` | Partial/Optional | SQLite-backed session + repo-mirror persistence + persisted-label storage for the moderation engine (OFF by default; build with `WOLFRAM_BUILD_STORE=ON`; optional `WOLFRAM_BUILD_STORE_CRYPTO=ON` adds libsodium at-rest encryption) | +| `wolfram/threadgate_postgate.h` | Implemented | Threadgate / postgate record helpers — `wf_agent_create_threadgate`/`wf_agent_create_postgate` and `wf_agent_delete_record_by_uri` | +| `wolfram/embed_typed.h` | Implemented | Generated owning parser for embed records (`app.bsky.embed.*`) | +| `wolfram/feed_typed.h` | Implemented | Generated owning parser for timeline/feed records (`app.bsky.feed.*`) | +| `wolfram/feedgen_typed.h` | Implemented | Generated owning parser for feed-generator records (`app.bsky.feed.generator`) | +| `wolfram/graph_typed.h` | Implemented | Generated owning parser for graph records (`app.bsky.graph.*`) | +| `wolfram/list_typed.h` | Implemented | Generated owning parser for list records (`app.bsky.graph.list*`) | +| `wolfram/thread_typed.h` | Implemented | Generated owning parser for thread records (`app.bsky.feed.defs#thread*`) | | `tools/wf_lexgen.py` | Initial | Lexicon JSON to typed C data-model declarations | diff --git a/docs/roadmap.md b/docs/roadmap.md index 7c09280..cbbf735 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -57,6 +57,38 @@ tested). For what's still ahead, see [Next planned work](#next-planned-work). wrappers, and notification wrappers. Full input-validation tests. 25. Agent repo sync pipeline — offline mirror seed, verified incremental diff apply, mirror head query, operation inversion, local mirror record lookup. +26. DID PLC operation helpers (`wf_plc_*`, `plc.h`) — build/sign/submit + `plc` operations (create, rotate signing/handle/rotation keys, tombstone) + with ES256 signatures and verification. +27. Moderation decision engine (`moderation.h`) — blur/alert/inform/filter + decisions for accounts, profiles, posts, notifications, feed generators, + and user lists from labels, blocks, mutes, hidden posts, and muted words. + Offline; ingests API-shaped JSON (`wf_mod_prefs_from_json`, + `wf_mod_label_defs_from_labeler`, `wf_mod_labels_from_json`). +28. Generic JSON module (`json.h`) — canonical round-trip (`wf_json_canonicalize`) + and a JSON-Schema validator subset (type/required/properties/items, + enum/const, format, numeric bounds, string length/pattern, array + constraints, additionalProperties, anyOf/oneOf/not). +29. SQLite store persistence (`store.h`) — session + repo-mirror persistence and + persisted-label storage for the moderation engine; OFF by default, build + with `WOLFRAM_BUILD_STORE=ON`. With `WOLFRAM_BUILD_STORE_CRYPTO=ON` (libsodium) + the session credentials are encrypted at rest (XSalsa20-Poly1305, Argon2id). +30. Video upload (`blob.h` / `agent.h`) — dedicated video endpoint upload + (`wf_agent_upload_video`), job-status polling + (`wf_agent_get_video_job_status`), and upload limits + (`wf_agent_get_video_upload_limits`). +31. Chat typed wrappers (`chat_typed.h`) — `chat.bsky.convo`/`group`/`actor`/ + `moderation` write+query wrappers with chat-service endpoint resolution. + The full chat write surface is now implemented. +32. Ozone moderation-service / labeler helper (`ozone.h`) — verify and emit + labels, build service auth headers for the Ozone moderation service. +33. Generated typed wrappers — owning parsers for embed/feed/feed-generator/ + graph/list/thread records (`embed_typed.h`, `feed_typed.h`, + `feedgen_typed.h`, `graph_typed.h`, `list_typed.h`, `thread_typed.h`) plus + threadgate/postgate record helpers (`threadgate_postgate.h`). +34. Authenticated XRPC client (`auth_client.h`) — DPoP-binding OAuth-authenticated + XRPC query/procedure/blob-upload wrapper (`wf_auth_client_*`) with session + refresh and DPoP nonce retry. ## Next planned work @@ -64,11 +96,10 @@ tested). For what's still ahead, see [Next planned work](#next-planned-work). credentials (it SKIPs cleanly when `BSKY_HANDLE`/`BSKY_PASSWORD` are unset). - Broaden the `wolfram` CLI further (label subscription streaming) and wire `help `. -- Generic JSON module (`wolfram/json.h`): add a sorted canonical form if a use - case appears. - Continue cross-referencing `bluesky-social/atproto` and `rsky` for protocol - parity. The full `chat.bsky.convo`/`group`/`actor`/`moderation` write surface - is now implemented; remaining parity items include labeler service records. + parity. Remaining parity items include full labeler service record coverage + and tooling. +- Higher-level endpoint examples using generated clients. ## Dependencies