diff --git a/COSTS.md b/COSTS.md new file mode 100644 index 0000000..eed5fed --- /dev/null +++ b/COSTS.md @@ -0,0 +1,60 @@ +# COSTS.md + +> **Mandate.** Tracks the infrastructure cost attributed to **stream** +> (stream.waow.tech) and what we're doing about it. Numbers drift, so this +> file names the levers and the instruments, never a hardcoded figure. + +## current cost — fetch it live, never hardcode + +The daily snapshot (collected by my-prefect-server, surfaced at +https://hub.waow.tech) attributes this repo's resources to the `stream` +project: + +```bash +curl -s https://hub.waow.tech/api/costs.json | jq '{ + as_of: .generatedAt, + stream_monthly_usd: ( + [ .lineItems[] | select(.project == "stream") ] + | (map(.amount) | add // 0) / 100 + ), + lines: [ .lineItems[] | select(.project == "stream") + | {service, provider, usd: (.amount/100), estimated} ] +}' +``` + +Resources: the `stream-cx43` hetzner box (hel1) + its 3 TB volume, plus any +fly apps whose names contain `stream` (e.g. the loadgen). Attribution lives +in my-prefect-server `packages/mps/src/mps/costs/projects.py`; fix mappings +there, not numbers here. + +## the shape of the spend (measured 2026-08-09) + +- **the volume is the cost.** block storage bills per provisioned GB + (€0.0767/GB·mo gross); the sealed archive (~1.75 TB) plus grow-only + headroom dwarfs the server line. treemap snapshot of the full breakdown: + operator session artifact "stream · where the money goes". +- **memory is not a lever.** ~3 GB RSS on 16 GB; the rest is page cache over + a network-attached volume, which is what makes cold serving tolerable. + breakdown: `stream_manifest_resident_bytes` + tail + tombstone gauges + (grafana → "Memory contributors"). +- **format density is not a lever.** recompression of real blocks measured + ≤ 11% headroom (per-block l19 / whole-file l19 / trained dict) — not worth + a 1.7 TB rewrite, and upstream owns the format. + +## levers, in order + +1. compaction deletes-fold reclaiming segment bytes (throttled workers). +2. right-sizing the volume — hetzner volumes grow but never shrink, so this + is a copy-to-smaller-volume migration, sized after growth settles. +3. object-storage tiering for cold segments. go/no-go is empirical: + `stream_compaction_rewritten_age_seconds` reports whether steady-state + rewrites cluster in recent segments (tiering works) or scatter across + deep history (it doesn't). decide from the histogram, not from vibes, + and not while upstream format work is in flight. + +## changelog +- **2026-08-09** — initial notice. hetzner cost connector taught to price + volumes (my-prefect-server 665b2dd); `stream` project mapping added + (5c3f8d7). pending: HCLOUD token for the stream hetzner project in the + `hetzner-tokens` secret block — until it exists, snapshots show no + hetzner lines for this repo.