diff --git a/docs/POSTHOG.md b/docs/POSTHOG.md new file mode 100644 index 0000000000..ff721ab4ea --- /dev/null +++ b/docs/POSTHOG.md @@ -0,0 +1,162 @@ +# PostHog + +PostHog is additive product analytics. It does not replace the existing +operational or traffic systems. + +| System | Ownership | +| ----------------- | ---------------------------------------------------------------------------------------------------------------- | +| Lith + Silo/Mongo | Boot health, piece-run logs, errors, bundle performance, piece-hit counters, access logs, and storage operations | +| Google Analytics | Existing broad site traffic | +| PostHog | Minimized journeys, funnels, cohorts, and experiments | + +The browser integration is inert until Lith receives `POSTHOG_PROJECT_TOKEN`. +`POSTHOG_API_HOST` may be `https://us.i.posthog.com` (default) or +`https://eu.i.posthog.com`. The project token is public browser configuration; +personal API keys and OAuth tokens never belong in the repository or HTML. +Set `POSTHOG_SERVER_ENDPOINT_EVENTS=true` separately to enable anonymized, +batched endpoint-volume events. This second switch makes the higher-volume +server stream an explicit rollout decision. + +Initial capture is deliberately narrow: + +| Event | Properties | +| ----------------------- | ----------------------------------------------------------------------------- | +| `$pageview` | `ac_route`, with query, hash, published handles, and source removed | +| `ac_piece_opened` | built-in `piece` or `null`, `piece_kind`, minimized `route` | +| `$identify` | Auth0 `sub`; public `handle` when available; never email | +| `ac endpoint completed` | `endpoint`, method/status/latency buckets, analytics class, aggregate `count` | + +Autocapture, session replay, exception capture, performance capture, heatmaps, +surveys, and product tours are off. Embedded, packed, local, and preview/icon +renders do not initialize PostHog. Do Not Track is respected. + +Start with a journey from `$pageview` to `ac_piece_opened`, then break down by +`ac_route`, `piece_kind`, and built-in `piece`. Validate that published pieces +have `piece = null` before adding any dashboard or experiment. + +The server event is an anonymous aggregate, not a person event. It uses one +Lith-level distinct ID, disables person profiles and GeoIP, and combines equal +dimensions into ten-second batches. Never use it for unique-user counts or +person funnels; sum its `count` property for request volume. + +## Endpoint map + +Run the inventory from the repository root: + +```sh +node toolchain/analytics/posthog-inventory.mjs > /tmp/ac-posthog-inventory.json +``` + +The current tree contains 165 function source files, resolving to 164 unique +names: 160 statically detected handlers and four helpers or scripts. This is +not the same as the number of public routes. Lith also provides aliases, media +routes, host rewrites, operational routes, static files, and piece/index +fallbacks. `lithRouteFamilies` in the generated JSON maps those surfaces. + +Every function is classified by `shared/posthog-policy.mjs`: + +| Policy | PostHog treatment | +| -------------------------------- | ------------------------------------------- | +| `minimized-browser-or-aggregate` | Browser journey and/or bounded server count | +| `aggregate-status-only` | Bounded server count only | +| `inventory-only` | Mapped for context; emits no event | +| `existing-lith-silo-only` | Remains in Lith/Silo/Mongo | +| `disabled` / `review-required` | Fails the inventory test until reviewed | + +The server count contains only function name, HTTP method, status class, +duration bucket, analytics class, and count. It never reads or sends path, +query, request or response body, IP, user agent, user ID, authorization header, +raw error, or stack. Messaging, MCP, local-machine, admin, and existing +operational telemetry classes do not emit endpoint events. + +Adding a function source requires an analytics classification. Run: + +```sh +node --test \ + system/tests/product-analytics.test.mjs \ + system/tests/posthog-server.test.mjs \ + system/tests/posthog-inventory.test.mjs +``` + +The inventory test fails when the source/handler counts change or any function +lands in `review-required`. Update the reviewed policy and counts together. + +## Validation + +Before enabling the server switch, configure only the browser token and verify: + +1. Production `$pageview` events contain minimized `ac_route` values. +2. `/@handle/...`, `/$code`, and prompt-source routes become `/@published`, + `/$code`, and `/prompt`; no query or hash survives. +3. Published pieces have `piece = null`. +4. Identified profiles contain Auth0 `sub` and optional public handle, never email. +5. Replay, autocapture, exception, performance, survey, and tour data remain absent. + +Then enable `POSTHOG_SERVER_ENDPOINT_EVENTS=true` and verify `ac endpoint +completed` has only the documented properties. A useful HogQL request-volume +check is: + +```sql +select + properties.endpoint as endpoint, + sum(toInt64(properties.count)) as requests +from events +where event = 'ac endpoint completed' +group by endpoint +order by requests desc +``` + +For product behavior, create a funnel from `$pageview` to `ac_piece_opened` and +break it down by `piece_kind`. Endpoint aggregates answer system-usage volume; +Lith/Silo answer failures and diagnostics; neither should be substituted for +the other. + +Rollback requires no code or data migration: unset `POSTHOG_PROJECT_TOKEN` to +disable browser and server analytics, or unset only +`POSTHOG_SERVER_ENDPOINT_EVENTS` to retain browser journeys. + +## Product context + +- The front door is a prompt-driven creative computer. `imnew` registers, + verification establishes the account, `handle` claims a public identity, and + a piece name opens a program. +- The browser runtime lives in `system/public/aesthetic.computer`; pieces are in + `disks/`; Lith serves the site and API; Silo is the data and storage console. +- Built-in, published, and KidLisp pieces are different content classes. Raw + source, prompts, paintings, chat, and private account fields are content, not + analytics properties. +- Local tools include Slab/host tooling and the prox, fleet, paper, frame, + chat, DM, mail, and calendar MCP surfaces. Their existence and public product + role are useful context. Prompt/session content, contacts, messages, mail, + calendars, fleet host details, secrets, and local files are not analytics + inputs. + +Any local-tool analytics must be a separate opt-in proposal with a minimized +schema such as `{ tool_family, operation_class, outcome, duration_bucket }`. +Never include arguments, file paths, hostnames, handles, recipients, message +content, artifact contents, or raw errors by default. + +## Self-driving + +PostHog's setup command is: + +```sh +npx @posthog/wizard self-driving +``` + +Do not run it unattended. It can connect GitHub, enable replay and error +tracking, configure signal sources, and schedule scouts. Before activation: + +1. Verify production events and the privacy contract above. +2. Decide whether AI data processing is acceptable. +3. Resolve repository routing: Tangled is authoritative and GitHub is currently + documented as a read-only mirror, while PostHog requires a writable GitHub + repository to open PRs. +4. Grant only the selected GitHub repository, require human review and merge, + and leave deployments outside PostHog. +5. Review every proposed signal source. Replay, error capture, support, Slack, + and local MCP content stay off until each has its own privacy review. + +Self-driving creates a branch and PR for an actionable report; a human still +reviews and merges it. Start with one manually reviewed report, not broad +autonomous production mutation. diff --git a/lith/README.md b/lith/README.md index edc033768e..554df55484 100644 --- a/lith/README.md +++ b/lith/README.md @@ -3,21 +3,35 @@ Secrets and runtime env for the Aesthetic Computer monolith deploy. `lith/deploy.fish` expects: + - `aesthetic-computer-vault/lith/.env` That file is uploaded to: + - `/opt/ac/system/.env` Why `system/.env` on the server: + - [`lith.service`](/workspaces/aesthetic-computer/lith/lith.service) uses `EnvironmentFile=/opt/ac/system/.env` - The monolith serves the main site and API from the shared `system/` tree Minimum required keys: + - `NODE_ENV=production` - `CONTEXT=production` - `DEPLOY_SECRET=...` +Optional product analytics keys: + +- `POSTHOG_PROJECT_TOKEN=phc_...` — enables the privacy-minimized browser client +- `POSTHOG_API_HOST=https://us.i.posthog.com` — US or EU Cloud ingestion host +- `POSTHOG_SERVER_ENDPOINT_EVENTS=true` — separately enables anonymous endpoint aggregates + +See [`docs/POSTHOG.md`](../docs/POSTHOG.md) for the endpoint inventory, privacy +contract, event schemas, validation, and rollback. + Recommended workflow: + 1. Copy `.env.example` to `.env` 2. Fill in the real production values 3. Re-run `fish vault-tool.fish status` to confirm `lith/.env` is tracked diff --git a/lith/product-analytics.mjs b/lith/product-analytics.mjs new file mode 100644 index 0000000000..01c253d7a9 --- /dev/null +++ b/lith/product-analytics.mjs @@ -0,0 +1,150 @@ +import { + classifyPostHogFunction, + permitsPostHogEndpointAggregate, +} from "../shared/posthog-policy.mjs"; + +const CLOUD_HOSTS = new Set([ + "https://us.i.posthog.com", + "https://eu.i.posthog.com", +]); + +export function durationBucket(ms) { + if (ms < 100) return "under_100ms"; + if (ms < 500) return "100_499ms"; + if (ms < 2000) return "500_1999ms"; + return "2000ms_or_more"; +} + +export function statusClass(status) { + const numeric = Number(status); + if (!Number.isFinite(numeric) || numeric < 100 || numeric > 599) + return "unknown"; + return `${Math.floor(numeric / 100)}xx`; +} + +export function endpointAggregate(name, ms, status, method) { + const policy = classifyPostHogFunction(String(name || "unknown")); + if (!permitsPostHogEndpointAggregate(policy)) return null; + return { + endpoint: String(name || "unknown") + .replace(/[^a-z0-9-]/gi, "_") + .slice(0, 80), + method: String(method || "UNKNOWN") + .toUpperCase() + .slice(0, 10), + status_class: statusClass(status), + duration_bucket: durationBucket(Number(ms) || 0), + analytics_class: policy.class, + }; +} + +export function lithSurfaceAggregate(pathname, ms, status, method) { + const path = String(pathname || ""); + const gym = path.match(/^\/api\/(publish|history|rewind)-gym\/?$/); + const endpoint = gym + ? `lith-gym-${gym[1]}` + : /^\/media(?:\/|$)/.test(path) + ? "lith-media" + : /^\/frame(?:\/|$)/.test(path) + ? "lith-frame" + : null; + if (!endpoint) return null; + return { + endpoint, + method: String(method || "UNKNOWN") + .toUpperCase() + .slice(0, 10), + status_class: statusClass(status), + duration_bucket: durationBucket(Number(ms) || 0), + analytics_class: gym ? "gym" : "content-media", + }; +} + +export function createEndpointAnalytics({ + projectToken = process.env.POSTHOG_PROJECT_TOKEN, + apiHost = process.env.POSTHOG_API_HOST || "https://us.i.posthog.com", + enabled = process.env.POSTHOG_SERVER_ENDPOINT_EVENTS === "true", + fetchImpl = globalThis.fetch, + flushIntervalMs = 10_000, +} = {}) { + const active = Boolean( + enabled && + projectToken?.startsWith("phc_") && + CLOUD_HOSTS.has(apiHost) && + typeof fetchImpl === "function", + ); + let aggregates = new Map(); + let flushing = null; + + function queue(properties) { + if (!active || !properties) return false; + const key = JSON.stringify(properties); + const current = aggregates.get(key); + aggregates.set(key, { ...properties, count: (current?.count || 0) + 1 }); + if (aggregates.size >= 100) void flush(); + return true; + } + + async function flush() { + if (!active || flushing || aggregates.size === 0) return false; + const pending = aggregates; + aggregates = new Map(); + flushing = (async () => { + const batch = [...pending.values()].map(({ count, ...properties }) => ({ + event: "ac endpoint completed", + properties: { + ...properties, + count, + distinct_id: "ac-lith-endpoint-aggregate", + $process_person_profile: false, + $geoip_disable: true, + }, + })); + try { + const response = await fetchImpl(`${apiHost}/batch/`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ api_key: projectToken, batch }), + signal: AbortSignal.timeout(3000), + }); + if (!response.ok) throw new Error(`HTTP ${response.status}`); + return true; + } catch { + for (const [key, value] of pending) { + const current = aggregates.get(key); + aggregates.set(key, { + ...value, + count: value.count + (current?.count || 0), + }); + } + return false; + } finally { + flushing = null; + } + })(); + return flushing; + } + + function capture(name, ms, status, method) { + return queue(endpointAggregate(name, ms, status, method)); + } + + function captureSurface(pathname, ms, status, method) { + return queue(lithSurfaceAggregate(pathname, ms, status, method)); + } + + const timer = active + ? setInterval(() => void flush(), flushIntervalMs) + : null; + timer?.unref?.(); + + return { + active, + capture, + captureSurface, + flush, + stop() { + if (timer) clearInterval(timer); + }, + }; +} diff --git a/lith/server.mjs b/lith/server.mjs index d4a7aad33d..c3ea5fe64a 100644 --- a/lith/server.mjs +++ b/lith/server.mjs @@ -54,6 +54,7 @@ import { fileURLToPath, pathToFileURL } from "url"; import { createServer as createHttpsServer } from "https"; import { createServer as createHttpServer } from "http"; import { resolveFunctionName } from "./route-resolution.mjs"; +import { createEndpointAnalytics } from "./product-analytics.mjs"; const __dirname = dirname(fileURLToPath(import.meta.url)); const SYSTEM = join(__dirname, "..", "system"); @@ -73,6 +74,8 @@ if (existsSync(envPath)) { } } +const endpointAnalytics = createEndpointAnalytics(); + const PORT = process.env.PORT || 8888; const DEV = process.env.NODE_ENV !== "production"; @@ -138,6 +141,8 @@ function recordCall(name, ms, status, path, method, error) { errorLog.unshift({ time: s.lastError, fn: name, status, error: error || `HTTP ${status}`, path, method }); if (errorLog.length > MAX_ERROR_LOG) errorLog.length = MAX_ERROR_LOG; } + + endpointAnalytics.capture(name, ms, status, method); } function captureRawBody(req, _res, buf) { @@ -165,6 +170,21 @@ app.use((req, res, next) => { next(); }); +// Count only reviewed Lith-native product surfaces. The analytics adapter maps +// the path to a static name and discards params, queries, headers and bodies. +app.use((req, res, next) => { + const startedAt = Date.now(); + res.once("finish", () => { + endpointAnalytics.captureSurface( + req.path, + Date.now() - startedAt, + res.statusCode, + req.method, + ); + }); + next(); +}); + // --- Whistlegraph prompt routes --- // commands.json is regenerated with the site model and marks which codes may // safely occupy AC's bare command namespace. Every work also gets the explicit diff --git a/shared/posthog-policy.mjs b/shared/posthog-policy.mjs new file mode 100644 index 0000000000..ffcb3eaf31 --- /dev/null +++ b/shared/posthog-policy.mjs @@ -0,0 +1,279 @@ +const RULES = [ + { + class: "messaging-private", + pattern: + /^(ask|cal|chat|crm|email|mail|news|say|session|sotce-net|tell)|subscribe|push|clock/, + posthog: "inventory-only", + exclusion: + "No message, contact, recipient, calendar, or notification data.", + }, + { + class: "auth-account", + pattern: + /auth|authorized|device|login|signin|token|user|handle|profile|location|permahandle|delete-erase|update-tezos-address/, + posthog: "aggregate-status-only", + exclusion: "No tokens, email, account payloads, or authorization headers.", + }, + { + class: "commerce", + pattern: /billing|give|keep|mint|mug|paypal|print|shop|ticket/, + posthog: "aggregate-status-only", + exclusion: "No payment, wallet, address, or order payloads.", + }, + { + class: "local-machine", + pattern: + /agent-memory|machine|m4l|macpal|mcp-|ac-device|ff1|local-upload|os-install|os-native/, + posthog: "inventory-only", + exclusion: + "No host details, logs, local files, device identifiers, or credentials.", + }, + { + class: "operational-telemetry", + pattern: + /boot-log|bundle-telemetry|kidlisp-log|piece-hit|piece-log|metrics|reports|status|track-|menuband-logs|paper-hit/, + posthog: "existing-lith-silo-only", + exclusion: + "Keep raw logs, errors, stacks, IP and performance samples in Lith/Silo.", + }, + { + class: "admin-internal", + pattern: + /admin|backfill|fix-|migrate|oven|patch|presigned|redirect-proxy|register-|reload|update-build|verify-|warm-|whistlegraph|jas-tags/, + posthog: "inventory-only", + exclusion: + "No admin actions, internal payloads, secrets, or infrastructure details.", + }, + { + class: "content-media", + pattern: + /painting|piece|kidlisp|media|playlist|tape|sfx|stor|pixel|screenshot|art|flux|juke|manifestation|mockup|tv/, + posthog: "minimized-browser-or-aggregate", + exclusion: + "No source, prompts, media, filenames, handles, or artifact contents.", + }, +]; + +const PUBLIC_PRODUCT_NAMES = new Set([ + "apple-developer-merchantid-domain-association", + "api-docs", + "bdf-glyph", + "blank", + "bundle-html", + "cancelok", + "commits", + "docs", + "firebase-config", + "get-builds", + "get-plugins", + "index", + "logo", + "menuband-downloads", + "mood", + "og-image", + "og-preview", + "pop", + "run", + "vary", + "version", +]); + +// Reviewed against toolchain/analytics/posthog-inventory.mjs. Unknown names +// fail closed even if they happen to match one of the category patterns. +const REVIEWED_FUNCTION_NAMES = new Set([ + "ac-device", + "admin-rebake", + "agent-memory-ingest", + "api-docs", + "apple-developer-merchantid-domain-association", + "ask", + "atproto-user-stats", + "auth-cli-callback", + "auth0-events", + "authorized", + "backfill-painting-codes", + "bdf-glyph", + "billing", + "blank", + "boot-log", + "bundle-html", + "bundle-telemetry", + "bundle-telemetry-query", + "cal", + "cancelok", + "chat-heart", + "chat-messages", + "claude-token", + "client-media", + "clock", + "commits", + "crm", + "delete-erase-and-forget-me", + "delete-tape", + "device-auth", + "device-login", + "device-pair", + "device-pair-login", + "device-token", + "docs", + "email", + "export-piece", + "ff1-devices", + "ff1-pair", + "ff1-proxy", + "firebase-config", + "fix-tezos-network-prod", + "flux", + "get-builds", + "get-painting", + "get-plugins", + "get-tape", + "get-tape-status", + "give", + "give-image", + "give-portal", + "gives", + "handle", + "handle-colors", + "handles", + "index", + "jas-tags", + "juke-cloud", + "keep-confirm", + "keep-mint", + "keep-prepare", + "keep-prepare-background", + "keep-status", + "keep-update", + "keep-update-confirm", + "keeps-config", + "kidlisp-count", + "kidlisp-keep", + "kidlisp-list", + "kidlisp-log", + "local-upload", + "location", + "logo", + "m4l-plugins", + "machine-logs", + "machines", + "macpal-art", + "macpal-art-lib", + "macpal-status", + "mail-status", + "manifestations", + "mcp-remote", + "media-collection", + "menuband-downloads", + "menuband-logs", + "metrics", + "migrate-piece-paths", + "mockup-webp", + "mood", + "mug", + "mugs", + "nela-signin", + "news", + "news-api", + "news-guidelines", + "news-toll", + "og-image", + "og-preview", + "os-install-report", + "os-native", + "oven-complete", + "painting-code", + "painting-metadata", + "paper-hit", + "patch", + "paypal", + "permahandle", + "piece-commits", + "piece-dates", + "piece-fans", + "piece-hit", + "piece-log", + "piece-metadata", + "pieces-search", + "pixel", + "playlist", + "pop", + "presigned-url", + "print", + "profile", + "push", + "push-devices", + "redirect-proxy", + "register-build", + "register-plugin", + "register-push-token", + "reload", + "reports", + "run", + "say", + "screenshot", + "session", + "sfx", + "shop", + "sotce-net", + "store-clock", + "store-kidlisp", + "store-kidlisp-datomic", + "store-piece", + "stories", + "stretched-paintings", + "subscribe-to-topic", + "tape-draft", + "tell", + "test-tv-hits", + "ticket", + "track-media", + "track-media-stream", + "tv", + "tv-tapes", + "unsubscribe", + "update-build", + "update-painting-slug", + "update-tezos-address", + "user", + "user-tapes", + "vary", + "verify-builds-password", + "version", + "warm-gateways", + "whistlegraph-admin", + "whistlegraph-admin-lib", + "whistlegraph-og", + "whistlegraph-query", +]); + +export function classifyPostHogFunction(name) { + if (!REVIEWED_FUNCTION_NAMES.has(name)) { + return { + class: "review-required", + posthog: "disabled", + exclusion: "Classify this source before enabling any PostHog capture.", + }; + } + if (PUBLIC_PRODUCT_NAMES.has(name)) { + return { + class: "public-product", + posthog: "minimized-browser-or-aggregate", + exclusion: + "No request/response bodies, query strings, raw errors, or identifiers.", + }; + } + return ( + RULES.find((rule) => rule.pattern.test(name)) || { + class: "review-required", + posthog: "disabled", + exclusion: "Classify this source before enabling any PostHog capture.", + } + ); +} + +export function permitsPostHogEndpointAggregate(policy) { + return ["aggregate-status-only", "minimized-browser-or-aggregate"].includes( + policy?.posthog, + ); +} diff --git a/system/netlify/functions/index.mjs b/system/netlify/functions/index.mjs index 918cbff5f2..3136c3c462 100644 --- a/system/netlify/functions/index.mjs +++ b/system/netlify/functions/index.mjs @@ -20,6 +20,29 @@ import { defaultTemplateStringProcessor as html } from "../../public/aesthetic.c import { networkInterfaces } from "os"; const dev = process.env.CONTEXT === "dev" || process.env.NETLIFY_DEV === "true"; const BARE_KIDLISP_CODE = /^[0-9A-Za-z]{3,64}$/; +const POSTHOG_CLOUD_HOSTS = new Set([ + "https://us.i.posthog.com", + "https://eu.i.posthog.com", +]); + +function postHogBrowserConfig() { + const projectToken = process.env.POSTHOG_PROJECT_TOKEN?.trim(); + const apiHost = process.env.POSTHOG_API_HOST?.trim() || "https://us.i.posthog.com"; + if (!projectToken?.startsWith("phc_") || !POSTHOG_CLOUD_HOSTS.has(apiHost)) { + return null; + } + return { + projectToken, + apiHost, + uiHost: apiHost.startsWith("https://eu.") + ? "https://eu.posthog.com" + : "https://us.posthog.com", + }; +} + +function serializeBrowserConfig(config) { + return JSON.stringify(config).replaceAll("<", "\\u003c"); +} // Fire-and-forget piece hit tracking (don't await, don't block page load) async function trackPieceHit(piece, type) { @@ -803,6 +826,7 @@ async function fun(event, context) { // in the request url qury params... const qsp = event.queryStringParameters || {}; const previewOrIcon = "icon" in qsp || "preview" in qsp; + const posthogConfig = previewOrIcon ? null : postHogBrowserConfig(); const body = html` @@ -1191,6 +1215,13 @@ async function fun(event, context) { } }); + +