{"entries":[{"lastCommit":{"author":{"email":"1560508+ryoppippi@users.noreply.github.com","name":"ryoppippi","when":"2026-08-15T14:26:44.000000+01:00"},"committer":{"email":"noreply@github.com","name":"GitHub","when":"2026-08-15T14:26:44.000000+01:00"},"hash":"a323dcc233e259cd2bc3500bfa3cb4ff9d9a4f6d","message":"fix(grok): hide empty cache creation column (#1605)\n\n* fix(grok): hide empty cache creation column\n\nFocused Grok reports now suppress the cache-creation column when every selected row reports zero, while retaining cache reads, totals, JSON fields, and unified report columns. The shared table renderer exposes this as an opt-in display option so future non-zero Grok records remain visible. Add focused regression coverage and document the source-specific table behavior.\n\n* test(grok): harden cache column regression\n\nRead the shared renderer default in the regression test so the test guards the public default configuration. Link the Grok guide to the JSON output documentation for the stable cacheCreationTokens schema.","tree":"e814c0c4e72a1662b2cef7fec797cf95842e748a"},"mode":"0100644","name":"lib.rs","size":3240},{"lastCommit":{"author":{"email":"1560508+ryoppippi@users.noreply.github.com","name":"ryoppippi","when":"2026-08-10T19:37:01.000000+01:00"},"committer":{"email":"noreply@github.com","name":"GitHub","when":"2026-08-10T19:37:01.000000+01:00"},"hash":"215c61c78b643300ae7b99459068e213fdfbb395","message":"feat(grok): add Grok Build CLI usage adapter (#1593)\n\n* feat(grok): add Grok Build usage adapter\n\nParse completed Grok Build turns, map token usage and LiteLLM pricing, and expose focused and unified reports. Discover data from non-empty GROK_HOME or ~/.grok, add schema and CLI integration, and document the source.\n\n* fix(grok): report session activity and project path\n\nSession reports grouped by key then moved the group key into session_id,\nwhich left first_activity, last_activity and project_path unset. Accumulate\nwith SessionAccumulator like qwen/opencode/antigravity, and include session\nmetadata in the JSON report.\n\n* test(grok): expand adapter coverage to antigravity-level\n\nAdd fixture-backed tests for path discovery edge cases, token/pricing\nmapping, timestamp and summary metadata, cross-session dedupe, corrupt\nfile isolation, and session aggregation. Synthesize updates.jsonl rather\nthan committing real session trees.\n\n* fix(grok): bill from recorded costUsdTicks and drop reasoning from totals\n\nTwo corrections to the Grok adapter, both found by comparing its output\nagainst real ~/.grok logs.\n\nCost: every completed turn records `costUsdTicks`, a fixed-point USD amount\nwhere one tick is 1e-10 USD. The adapter discarded it and recomputed from the\npricing table instead. Grok bills each API request separately, but a\n`turn_completed` row only carries the sum over the requests in that turn, so a\nrecomputation cannot place the long-context tier boundary where Grok did and\nlands on either side of the real figure. Across 85 turns of local data the\nrecomputed total is $15.18 against an actual $17.40, and per-day it ranges from\n-46% to +12%. Feeding the ticks through `cost_usd` reproduces the invoice\nexactly in `display` and `auto`, while `calculate` keeps recomputing and `auto`\nstill falls back for turns that recorded no ticks.\n\nTokens: `reasoningTokens` were added to `extra_total_tokens`, which is the\nbucket for tokens *not* already covered by the usage sum. Grok reports\n`totalTokens == inputTokens + outputTokens` and its reasoning count never\nexceeds output, so reasoning is a subset of output and was being counted twice.\nThat inflated the grand total by 113,556 tokens over the same 85 turns.\n\nThe `turn_line` test helper moves its four token counts into a struct so it\nstays under the clippy argument limit.\n\n* chore(rust): reconcile grok branch with main\n\nMerging main brought the adapter count to 15 while the array annotation in the\n`agent_commands_are_exposed_by_independent_crates` test still said 14, so the\ntest crate stopped compiling. Also runs treefmt over the two files the grok\nbranch left unformatted, which the clippy and treefmt checks require.\n\n* feat(grok): read cacheCreationTokens and document interrupted turns\n\nGrok started writing `cacheCreationTokens` alongside `cachedReadTokens` around\nCLI 0.2.118 and still writes it in 1.0.0. The adapter hardcoded\n`cache_creation_input_tokens: 0`, so the field would be dropped the moment it\nwent non-zero.\n\nEvery turn observed so far reports it as zero, so there is no sample proving\nwhether it sits inside `inputTokens` or beside it. `cachedReadTokens` is\nprovably inside — session totals match `logs/unified.jsonl`, where\n`cached_prompt_tokens` is part of `prompt_tokens` — so the sibling field is\ncarved out of the same total. That keeps the three parts summing back to\n`inputTokens`, which is the invariant the adapter already relies on.\n\nAlso documents two limits that surfaced while checking real logs: a session\nkilled mid-turn never writes `turn_completed`, so its usage cannot be reported\nat all, and `logs/unified.jsonl` cannot stand in for it because it records no\nper-request model id.\n\n* fix(nix): include grok adapter in cargo artifact layer\n\nCo-authored-by: Codesmith \n\n* fix(grok): decode UTF-8 project paths and align docs with the single-root contract\n\nDecode percent triplets into bytes so multi-byte cwd segments survive, drop the\nalways-zero discriminator from the loader dedupe key, match the report test helper\nto the parser's reasoning contract, and correct the README surface list plus the\nGROK_HOME multi-root docs.\n\nCo-authored-by: Codesmith \n\n* fix(grok): keep cache-creation tokens in the fallback dedupe keys\n\nCo-authored-by: Codesmith \n\n* fix(grok): make module-local helpers private for hawk\n\nCo-authored-by: Codesmith \n\n* test(grok): cover wide scalars and invalid bytes in path decoding\n\nThe existing case decodes a two-byte scalar end-to-end. This adds a direct unit\ntest for a three-triplet CJK segment, and pins the behaviour for a percent\ntriplet that is not valid UTF-8: it degrades to the replacement character rather\nthan panicking or dropping the rest of the path.\n\n* fix(grok): dedupe only on eventId so distinct turns survive\n\nThe global dedupe rebuilt a content key for entries without an `eventId`, but\nthat key can only be built from what `LoadedEntry` carries, and reasoning tokens\nare not among them. `parse_session_files` deduped its own file using the full\nrecord including reasoning, so two eventId-less turns in the same second with\nidentical input, output and cache counts but different reasoning were kept by\nthe parser and then collapsed here, dropping one turn entirely.\n\nCross-file dedupe is about the same server event appearing in more than one\nsession export, which is what `eventId` identifies. Entries without one are now\nleft alone rather than matched on token counts.\n\nReported by an adversarial review pass. Every `turn_completed` in the logs\nchecked so far carries an `eventId`, so this was not reachable in practice.\n\n---------\n\nCo-authored-by: LeeShunEE <2457013396@qq.com>\nCo-authored-by: Kinema. <54445332+LeeShunEE@users.noreply.github.com>\nCo-authored-by: Codesmith ","tree":"4f14ca749802e6f1c4bb9cae158a76c7f7b7f554"},"mode":"0100644","name":"loader.rs","size":9453},{"lastCommit":{"author":{"email":"1560508+ryoppippi@users.noreply.github.com","name":"ryoppippi","when":"2026-08-15T12:05:10.000000+01:00"},"committer":{"email":"noreply@github.com","name":"GitHub","when":"2026-08-15T12:05:10.000000+01:00"},"hash":"b936c29211b9bc92d9cd3215b8b8d132c21de288","message":"fix(pricing): take models.dev rates from the authoring catalog (#1541)\n\n* fix(pricing): take models.dev rates from the authoring catalog\n\nmodels.dev publishes one catalog per provider, so every model appears once\nper provider that serves it. Resellers publish their own rates, and the\ngenerator picked between duplicates by how many fields an entry carried,\nwith a hardcoded preference for three provider ids. Two problems followed.\n\nThe committed snapshot billed resold models at marketplace rates. It held\n`moonshotai/kimi-k2.7-code` at 0.73 USD/Mtok, which is OpenRouter's\ndiscounted rate rather than MoonshotAI's 0.95 list rate, alongside a\ncorrect `kimi-k2.7-code` at 0.95; which one a lookup resolved depended on\nthe fuzzy key match. `moonshotai/kimi-k3`, `moonshotai/Kimi-K3` and\n`kimi-k3` each carried a different price for the same model.\n\nCoverage was also decided by a model-name pattern, `/claude|anthropic|kimi\n|moonshot/i`, so any family nobody had added to that pattern was unpriced\noffline even when models.dev published it and LiteLLM did not.\n`xai/grok-build-0.1` is the current example: models.dev carries it under\nits authoring catalog, LiteLLM has no entry at all, and `--offline`\nreported no cost for it.\n\nRank catalogs by trust instead. The authoring catalog wins, cloud\nplatforms that resell at list price plus a published regional premium come\nnext, and everything else is a last resort used only for models the\nauthored catalog still knows - which is what keeps retired releases such as\n`claude-3-5-haiku-20241022`, now listed by resellers only. Authorship comes\nfrom the `models//.toml` directory scan rather than a provider\nlist, so new authors need no maintenance; only `zai`, which serves models\nauthored under `models/zhipuai/`, needs an alias entry.\n\nThe name pattern is gone, and entries whose costs are all zero (flat-fee\nplans such as `kimi-for-coding`) or that price assets rather than output\ntext tokens are skipped instead.\n\nThe snapshot grows from 335 to 422 entries. No surviving entry changes\nprice; the 234 dropped keys are reseller-invented aliases and speed tiers\n(`kimi-k2.6-nitro`, `accounts/fireworks/...`, `TEE/...`) and dotted or\n`-thinking` spellings the fuzzy matcher already resolves to their base key.\n\nThe derived provider trust lists are emitted alongside the snapshot for the\nRust runtime loader, which sees only the live api.json and so cannot scan\nthe authored catalog itself.\n\n* perf(pricing): minify the embedded models.dev snapshot\n\nThe committed snapshot is indented so its regeneration diffs stay\nreviewable, and `include_str!` embedded it byte for byte, so the\nindentation shipped in every binary. At 422 entries the whitespace is\nabout 19 KiB of the 58 KiB file.\n\nStrip it in `build.rs` on the way in, the same place the LiteLLM snapshot\nis already compacted. The committed file keeps its formatting.\n\nTogether with the broader coverage this keeps the release binary flat:\n3,162,832 bytes before, 3,162,880 after, for 87 more priced models.\n\n* fix(pricing): rank models.dev catalogs when refreshing pricing online\n\nThe runtime models.dev fallback loaded providers straight out of an\n`FxHashMap`, and the first catalog to claim a model id keeps it. Which\ncatalog that was therefore depended on hash iteration order, so a reseller\ncould supply a model the authoring catalog also publishes, and the\nselection could change whenever models.dev adds or removes a provider.\n`grok-build-0.1`, for example, is served by 13 catalogs whose input rates\nrange from 0 to 1 USD/Mtok.\n\nRank the catalogs before loading, using the provider trust lists the\nsnapshot generator derives from the authored catalog, and sort by provider\nid within a tier so the result no longer depends on hash order at all.\n\nAlso skip entries whose input and output costs are both zero. Flat-fee\nsubscription catalogs publish those - `kimi-for-coding` is one, and it is\namong the 13 that serve `grok-build-0.1` - so loading one reported every\nrequest as free. The snapshot generator already skipped them; the runtime\nloader did not.\n\n* fix(pricing): keep asset-priced models out of the models.dev snapshot\n\nDropping the model-name filter let two kinds of non-token pricing into the\nsnapshot, where the runtime divides the rate by a million and multiplies it\nby token counts.\n\n`whisper-large-v3` prices per second of audio. It reached the snapshot at\n0.003 USD because the catalogs serving it advertise text output, and the\nprevious check only looked at the output side.\n\n`gemini-2.5-flash-image` and `gemini-3-pro-image-preview` price per\ngenerated image - 30 and 120 USD respectively. Google authors both with\nimage output, so the output check did reject Google's entry, but `302ai`\nlists the same models as text-only and its entry was embedded instead. A\nsloppier reseller description could therefore override the author's.\n\nRead the modalities off the authored catalog rather than the catalog being\nloaded, so one reseller's description cannot decide eligibility, and require\nthat the input modalities include text. A model that accepts no text is a\ntranscription or vision-only endpoint billed by duration; accepting audio,\nvideo, image or PDF alongside text is normal for chat models and is\ntokenised, so `kimi-k3` and `gemini-3-flash-preview`, which take video and\naudio input, stay eligible. An earlier attempt rejected any audio or video\ninput and dropped both of them.\n\nThe snapshot goes from 422 to 418 entries. The Rust test asserts both\ndirections against the committed artifact so a future regeneration cannot\nreintroduce either case.\n\n* fix(pricing): apply the snapshot's selection rules to live models.dev too\n\nThe asset-priced guard and the same-tier tie-break existed only in the\nsnapshot generator, so an online refresh could still price a model the\nembedded snapshot deliberately excludes, and could pick a different catalog\nthan the snapshot for the same model. Offline and default runs disagreed.\n\nThe live `api.json` records neither who authored a model nor the authored\nmodalities, so both have to be carried in from generation time. The\ngenerated artifact therefore grows an `assetPricedModelIds` list, and is\nrenamed from `models-dev-provider-trust.json` to\n`models-dev-catalog-rules.json` now that it carries more than provider\ntrust.\n\nThe loader applies it the same way the generator does: the authored verdict\ndecides where it exists, so a reseller describing an image model as\ntext-only cannot reintroduce a per-image rate, and entries the authored\ncatalog has never seen fall back to their own modalities, which is enough to\nreject an audio-only transcriber from any catalog.\n\nSame-tier ties now compare how much pricing detail an entry carries -\nexplicit cache-read, cache-write, context limit - before falling back to the\nprovider id, matching `shouldReplaceModelsDevPricingCandidate`. That needed\nthe loader to track which catalog claimed each id during a pass, so a better\ncandidate can replace a weaker one; ids absent from that record still belong\nto another pricing source and are left alone. Replacements do not increment\nthe loaded count, which stays a count of resolved models.\n\nReported by cubic on #1541.\n\n* style(pricing): collapse the nested modality check\n\nclippy::collapsible_if. Rewriting the two checks as slice patterns reads\nbetter than collapsing the condition and drops the redundant emptiness\ntests.\n\n* fix(pricing): keep separately priced tiers of the models we carry\n\nDropping reseller-only ids took the speed and effort tiers with them:\n`kimi-k2.6-nitro`, `kimi-k2.7-code-flex`, `glm-5.2-nitro`, `claude-opus-4-6-think`\nand about 140 others. Those are their own routes at their own rates, usually\ncheaper than the author's, so resolving them to the base model by name\nsimilarity over-reports them. `kimi-k2.6-nitro` went from 0.275 to 0.95\nUSD/Mtok - 3.5x - which is worse than the reseller duplicate this series set\nout to remove.\n\nKeep a reseller entry when its id is the authored id plus a suffix, and only\nfor bare ids: an id carrying a provider path is that provider's catalog entry\nfor a model some trusted catalog publishes anyway, which is the duplicate\ncase.\n\nA tier the author prices itself does not qualify. `claude-opus-5-fast` and\n`claude-opus-4-8-fast` exist in exactly one reseller catalog at 12 USD/Mtok\nwhile Anthropic publishes 10, so taking the reseller rate would over-report\nthem; the fast multiplier the runtime already applies is correct. The check\nreads those modes from the authoring provider's catalog entry, because\n`experimental.modes` lives in `providers//models/.toml` rather\nthan in the `models/` metadata.\n\nSnapshot 418 -> 559 entries, binary 3,162,832 -> 3,179,440 bytes (+0.5%).\n`ccusage daily --until 20260728` over a real 169-day Claude and Codex log\nstays byte-identical to main in both online and offline modes.\n\nThe broader alternative - keep every reseller id that does not reduce to an\nauthored one - was measured at 1804 entries and +136 KiB, because it drags in\nevery third-party host's copy of every open-weight model.\n\n* fix(pricing): match generation's candidate ordering in the live loader\n\nThree defects in the same-tier tie-break, all reported on #1541.\n\nThe detail comparison summed how many optional fields an entry carried, but\ngeneration compares cache-read, then cache-write, then context-limit presence\nin that order. A cache-read-only entry therefore lost online to one carrying\ncache-write and a context limit, while the committed snapshot picked the\nformer. A `ModelsDevClaim` struct with the fields declared in that order\nfixes it, because `derive(PartialOrd)` compares them in declaration order.\n\nA replacement that publishes no context limit left the replaced catalog's\nlimit in `context_limits`, reporting a window the selected rates do not\nbelong to. It is now removed, and only when the id was claimed in this pass,\nso another pricing source's limit is never touched.\n\nTwo source keys in one catalog can carry the same `id`, and the first to\narrive claimed it, which meant hash iteration order decided the winner again.\nGeneration walks each catalog in key order and keeps the first of an\nequal-strength tie, so the entries are sorted by key before loading.\n\nToday's models.dev response has no such duplicate, because `generateProviders`\nsets `id` to the source key, but nothing in the schema guarantees that.\n\n* fix(treefmt): stop typos rewriting model ids in generated snapshots\n\nOne Gemini id in the models.dev snapshot ends in a clipped \"no thinking\",\nwhich typos treats as a misspelling and corrects. The committed snapshot\ntherefore carried a model id that does not exist upstream, so that model\ncould never match and would report no cost - a silent data corruption in a\nfile nobody reads by hand.\n\nThis only surfaced now because the previous commits widened the snapshot from\nthe Claude and Kimi families to every token-priced model, which is what first\npulled a typos-sensitive id into it. CI caught it as a treefmt-check failure\non the generated file.\n\nExclude the three generated snapshots from typos alone; they stay in the JSON\nformatter, which is what keeps their regeneration diffs reviewable.\n\nThe same rewrite applies to prose, so the comment explaining this cannot\nspell the id out - it was rewritten on the first `just fmt` after being\nwritten.\n\n* fix(pricing): carry the authored catalog's verdicts into the live loader\n\nThe rules artifact listed only the models the authored catalog prices per\nasset, so the live loader could reject a model the snapshot carries: an\nauthored token-priced model whose serving catalog describes it as\nimage-output fell through to that catalog's modalities and was dropped, while\ngeneration had accepted it on the authored verdict. The asymmetry only ran one\nway, so `--offline` priced the model and a default run did not.\n\nCarry `authoredModelIds` as well. An authored id absent from the asset list is\nauthored as token-priced, which settles it whatever the serving catalog\nclaims, and only ids the authored catalog never listed still fall back to live\nmodalities.\n\nThe same artifact then closes the loader's other gap. Generation embeds a\nreseller entry only when the authored catalog knows the model or when the id\nnames a separately priced tier of one; the loader checked token pricing alone,\nso an online refresh priced 1705 ids the embedded snapshot deliberately omits,\neach at that reseller's own promotional or marked-up rate. Only the tiers an\nauthor prices itself were missing from the artifact, so with `authoredModes`\nalongside the authored id set the loader can apply\n`isEmbeddableModelsDevCandidate` too, and simulating it over the pinned catalog\nnow reproduces the committed snapshot exactly: same 559 ids, same rates. The\nartifact grows to 8.0 KB.\n\nAlso rank and tie-break catalogs by `provider.id` rather than the key they are\nfiled under, which is what `provider.id ?? providerId` gives the generator.\nmodels.dev assigns the id from the directory name today, so the two agree, but\nnothing in the schema requires it.\n\nReported by cubic on #1541.\n\n* fix(pricing): stop pruning models.dev ids and gate the fuzzy lookup instead\n\nPruning ids to keep the snapshot small was the wrong lever. `PricingMap::find`\nfalls back to substring matching that prefers the longest matching key, so\nremoving an id does not make its lookups fail - it makes them resolve to\nwhichever remaining key happens to match. Three pruning rules were tried and\neach mispriced something different, measured by pricing all 829 ids either\nside of the diff knows:\n\n- reseller ids dropped: `kimi-k2-7-code` resolved to `kimi-k2.7-code-highspeed`\n and doubled, from 4.95 to 9.90 USD for 1M+1M tokens, and about 40 Claude\n spellings such as `claude-4.5-sonnet` and `claude-3-7-sonnet-latest` went\n unpriced because the authored id is `claude-sonnet-4-5` and neither spelling\n contains the other\n- tiers marked exact-only on top of that: `moonshotai/kimi-k2.7-code-highspeed`\n then fell to the base rate instead\n- bare ids kept, namespaced ones dropped: 31 ids changed price, in both\n directions\n\nKeep every id the catalog publishes, so each resolves exactly and the fuzzy\nfallback only ever sees ids models.dev has never heard of - which is where it\nwas before this branch. The rules now decide only *which* catalog's rates an id\ngets.\n\nTwo kinds of key must still stay out of the fuzzy candidates, because they\nwould answer a lookup for a different model:\n\n- a separately priced tier: `kimi-k2.7-code-highspeed`, `kimi-k2.6-nitro`,\n `claude-opus-5-fast`\n- an id with no version in it: models.dev carries one called `auto`, which\n answered `codex-auto-review` - a Codex label that has to resolve through the\n adapter's date mapping rather than be priced directly. An existing test\n caught it\n\n`PricingMap` gains an `exact_only` set, excluded from the fuzzy scan in\n`find_entry` and `context_limit_entry`, and the snapshot marks those keys.\n\nSnapshot 335 -> 2362 entries, 476 of them exact-only. The binary grows\n3,162,832 -> 3,444,032 bytes, 8.9%. That is the price of the guarantee: across\nall 829 ids, 367 are newly priced, none lost pricing, and none changed price.\n\nTwo follow-ups the online path needed for the same reasons, folded in here\nbecause the workspace cannot author a separate commit GitHub will accept:\n\n`exactOnly` is a field this commit writes into the snapshot, and a live\n`api.json` has none, so nothing was ever added to `PricingMap::exact_only`\nonline: a refreshed run put every tier back into the fuzzy scan, where a live\n`kimi-k2.7-code-highspeed` answered `kimi-k2-7-code` and `auto` answered\n`codex-auto-review`. The loader now rederives the verdict from the rules\nartifact, asking the tier half about the catalog's source key and the\nunversioned half about the pricing key it resolves to. It also stops pruning\nreseller ids with `is_embeddable`, the rule this commit dropped from\ngeneration, which had the online refresh carrying 601 ids where the snapshot\ncarries 2362. The tier check folds `@` as `normalizeModelId` does, so the 25\nVertex regional ids such as `claude-opus-5@eu` are recognized as tiers of the\nmodel they shadow. Replaying the loader over the pinned revision's catalog now\nreproduces the committed snapshot exactly: the same 2362 ids, the same rates,\nthe same 476 exact-only keys.\n\nGeneration resolves `provider.id ?? providerId` before ranking a catalog, the\nway the loader does, so a catalog filed under an alias cannot be an owner\nonline and a reseller in the snapshot. models.dev assigns that id from the\ndirectory it scanned, so the artifacts regenerate byte for byte.\n\nCo-authored-by: Codesmith \n\nTest the two gates directly: unit tests for isUnversionedModelId in the\ngenerator, and a grok candidate test whose priced key no -build candidate\ncan reach, so it exercises the strip instead of the fuzzy lookup answering\nxai/ with a key the raw form already contains.\n\n* fix(pricing): keep the fuzzy lookup off exact-only ids\n\n`exact_only` kept a separately priced tier from answering another model's\nlookup, but not the reverse: the embedded models.dev snapshot is a\nseparate map consulted only after the primary LiteLLM one misses, so\n`claude-opus-5-fast` matched LiteLLM's `claude-opus-5` in the primary\nfuzzy scan and billed the tier at list price, 5/25 instead of 12/60. The\nsame shadowing hit the regional aliases, `claude-opus-5@eu` and friends.\n\nGate the fuzzy scan on whether a consulted catalog carries the requested\nid as exact-only, so it falls through to that exact entry instead. Only a\ncarried entry gates the scan, leaving ids nothing prices exactly on the\nfuzzy path they had.\n\nCo-authored-by: Codesmith \n\n* fix(pricing): deny the fuzzy scan when an alias names an exact-only id\n\nA `CCUSAGE_MODEL_ALIASES` entry is only tried after the map has answered\nfor the spelling the log recorded, so gating on that spelling alone left\nthe tier the alias names unreachable: `claude-opus-5-turbo` aliased to\n`claude-opus-5-fast` still fuzzy-matched LiteLLM's `claude-opus-5` and\nbilled the list rate. Carry the verdict for the resolved alias into the\nfirst lookup, in the context-limit path too. Exact entries for the\nrecorded spelling keep their precedence, and the alias, the network\ncatalog and the snapshot all still resolve fuzzily on their own terms.\n\nCo-authored-by: Codesmith \n\n* perf(pricing): deflate the embedded pricing snapshots\n\nDropping the coverage filters grew the embedded models.dev snapshot to a\nquarter megabyte of JSON, all of it shipped verbatim in every binary:\n3,162,832 bytes on main to 3,444,032, 8.9%. The JSON deflates to a ninth of\nits size, so compress all three embedded snapshots in build.rs and inflate\neach one once, on first use. The committed files stay indented for reviewable\nregeneration diffs.\n\n models-dev-pricing.json 259,174 -> 28,448 bytes\n binary 3,444,032 -> 3,179,904 (+0.5% over main)\n\nThe inflate side is miniz_oxide without default features, pure Rust; under\nthe fat-LTO opt-level=z release profile the decompressor costs a few KiB.\nInflation runs once per snapshot per process and is not measurable at\nstartup. Compression runs in build.rs at maximum effort, where its cost is\npaid only when a snapshot changes.\n\nVerifying the round-trip surfaced a pre-existing wrinkle: serde_json's\ndefault float parsing is not correctly rounded, so the minification\nintroduced in an earlier commit shifted eleven prices by one ULP on their way\ninto the binary - harmless at 1e-16 relative, but it broke the invariant that\nthe embedded data equals the committed data. The build script now enables\nserde_json's float_roundtrip feature, which is host-only and costs the\nshipped binary nothing; the inflated snapshots are byte-faithful to the\ncommitted files again.\n\nRe-verified against main across every id either snapshot knows: 367 newly\npriced, none lost, none changed. The two ids that moved in the raw probe,\nTEE/kimi-k2.5 and its -thinking sibling, no longer exist at the pinned\nmodels.dev revision, so both binaries guess through the fuzzy fallback and\ntheir guesses differ - the pre-existing unknown-id behavior, not a change in\nit. Real-log reports stay byte-identical to main in both modes.\n\n* fix(pricing): gate every separator spelling of an exact-only id\n\nmodels.dev writes a regional alias as claude-opus-5@eu, and the fuzzy scan\nreads @, . and - as one separator, so a request recording claude-opus-5-eu\npassed the raw-string exact-only gate and was billed at the base model's\nlist rate. Index exact-only ids under their normalized spelling too, and\nanswer such a request from the id it names rather than losing its rate.\n\nCo-authored-by: Codesmith \n\n* fix(pricing): index dash-spelled exact-only ids by their normalized form\n\nnormalized_pricing_key borrows a key already spelled with dashes alone, so\nthe normalized index skipped it and a request writing that id with . or @\nanswered neither membership question: the fuzzy scan then billed the base\nmodel instead of the exact-only tier.\n\nCo-authored-by: Codesmith \n\n* refactor(pricing): name the failing snapshot in build panics\n\nembed_snapshot runs for two snapshots and the LiteLLM path used the same\nmessage, so a build failure did not say which file it failed on. Reported\nby CodeRabbit on #1541.\n\n* feat(pricing): price long-context tiers from models.dev data\n\nGrok, GPT-5.x and Gemini bill requests above a context boundary at higher\nrates, and `--mode calculate` ignored that everywhere the data came from\nmodels.dev: the generator dropped `cost.tiers` on the floor, and the only\ntwo-stage rates the runtime knew were a hardcoded table for six GPT-5\nmodels. On a real Grok Build log, 94 of 172 requests exceed the 200K\nboundary and were billed at half the correct input rate.\n\nThe generator now emits each model's long-context band in the upstream\n`cost.tiers` shape, so the Rust loader parses the embedded snapshot and a\nlive `api.json` response with the same code. `Pricing` holds one above-base\nband, so the band with the lowest context threshold is kept - it is the one\na request crosses first, and the handful of models publishing a second,\nhigher band would otherwise contribute nothing.\n\nModels whose base rates come from LiteLLM get the band filled in from the\nmodels.dev snapshot after every load, replacing the hardcoded\n`builtin_long_context_rates` table. The overlay consults exact ids only\n(after date-suffix and alias resolution): a fuzzy match here could graft one\nmodel's tier onto another's base rates. `long_context_split_threshold`,\nwhich the Codex aggregation uses to bucket per-request context sizes, now\nreads the same data.\n\nDeleting the table also fixes two prices it had frozen: OpenAI cut\ngpt-5.6-terra to 2/12 USD/Mtok (tier 4/18) and gpt-5.6-luna to 0.2/1.2\n(tier 0.4/1.8) after it was written, and `put_builtin_pricing` was\noverwriting the corrected rates LiteLLM already publishes - luna reported\nat five times its price. Built-in entries are now a last resort for keys\nthe snapshots miss, never an overwrite, with one exception kept: z.ai's\nGLM cache facts (no cache-write billing, own cache-read rate) are patched\nonto entries whose snapshot did not publish cache rates, base rates\nuntouched.\n\n* fix(cost): select the long-context tier by the whole context\n\nThe two-stage branch compared only `input_tokens` against the model's\nthreshold, but adapters normalize usage into the Claude shape where that\nfield is the uncached remainder. The vendor picks the tier by the request's\nwhole context, cached or not, so a turn re-reading megabytes of cached\nprompt with a few thousand fresh tokens was billed at the short-context\nrate. Grok logs are the extreme case: prompt caching leaves days where\ncache reads outnumber fresh input fifty to one, so effectively no request\never crossed the boundary.\n\nSum uncached input, cache reads and cache writes for the comparison. The\nCodex path already compares OpenAI's raw `input_tokens`, which includes\nthe cached share, so the two paths now agree on what a request's size is.\n\nBilling itself is unchanged - each bucket still bills at its own rate -\nonly the tier selection sees the full context.\n\n* fix(grok): resolve pricing candidates exactly before falling back to fuzzy\n\n`PricingMap::find` falls back to substring matching, so once the snapshot\ncarries `grok-4.5`, the raw candidate `grok-4.5-build` fuzzy-hits it on the\nfirst iteration and every later candidate is dead code - including the\n`xai/`-prefixed spellings a user pricing override may be keyed by.\n\nTry every candidate exactly first, then repeat the walk with the fuzzy\nlookup. An override naming the request's model precisely now beats a fuzzy\nhit on a shorter spelling, and with no override the result is unchanged:\nboth passes land on the same grok-4.5 rates.\n\n* fix(pricing): merge dotted and dashed spellings onto one models.dev entry\n\nmodels.dev catalogs spell the same version with dots or dashes, and each\nspelling got its own snapshot entry from whichever catalog published it.\nThe two entries then tied in the fuzzy lookup - equal length, arbitrary\ntie-break - so `grok-4.5-build` resolved to `grok-4-5`, a reseller's flat\ncopy, instead of `grok-4.5`, xAI's tiered one, and every long-context Grok\nturn was billed at the base rate even after the tiers landed.\n\nSpellings now contend for one slot keyed by the normalized id, in the\ngenerator and in the live loader alike, and the losing spelling's entry is\nremoved outright - entry, exact-only flag and context limit - or both would\nstay findable and the tie would return. The winner also gains a comparison\nstep: within a trust tier, a catalog publishing a long-context band beats\none with more cache detail, because the band is the rate data hardest to\ncome by. Lookups by the losing spelling still resolve through the fuzzy\nnormalization, now onto the surviving entry.\n\nThe snapshot shrinks from 2363 to 2159 entries, all spelling duplicates.\n\n* docs(grok): describe long-context tier estimation\n\nThe estimate now applies xAI's long-context rates when a turn's whole\ncontext crosses the model boundary, so the guide stops claiming the tiering\nis not reproduced and states the per-turn approximation instead.\n\n* test(pricing): carry the tier flag in generator candidate fixtures\n\noxlint's type-aware pass requires the new field the runtime comparison\ngained; node --test does not type-check, so it passed without it.\n\n* fix(pricing): normalize catalog verdict lookups and stop the last overwrites\n\nReview findings from cubic on #1541, all in the new tier and selection code.\n\nThe authored catalog's verdicts - asset-priced, authored, its modalities -\nwere keyed by the exact source id, so a catalog spelling a model with\ndifferent separators or case dodged them: a reseller listing\n`gemini-2-5-flash-image` as text-only would have put its per-image rate\ninto the snapshot. The verdict maps and the rules artifact now hold\nnormalized ids, and every lookup normalizes the same way generation does,\ncase folding included; the live loader's claim slots fold case too, so\ncase-variant spellings merge online exactly as they merge offline.\n\nGeneration also disagreed with the runtime about an authored model that\npublishes no modalities: the generator fell back to the serving catalog's\ndescription while the loader kept every authored id, so a mislabeling\ncatalog could make the snapshot skip what a live refresh keeps. Authored\nids without authored modalities are now token-priced in both paths.\n\nThree built-in entries still overwrote refreshed pricing - the two\nclaude-3-5-haiku spellings and z.ai's free glm-4.5-flash - and now go\nthrough the fallback and patch helpers like the rest. The GLM cache patch\nalso repairs a defaulted cache-write on entries whose cache-read rate was\npublished: `Pricing` records only read explicitness, but the derived\ndefault is recognizable exactly, being this same computation.\n\nThe long-context sum saturates instead of wrapping, so a corrupt JSONL\nline cannot flip a request under the threshold or abort under overflow\nchecks.\n\n* fix(pricing): key empty declared ids by their source and order same-id catalogs\n\nTwo live-loader gaps against generation, reported by cubic on #1541.\n\nA live catalog can declare `id` as an empty string, and the loader stored\nthe model under \"\" - a name no lookup ever asks for - where the generator's\n`selectModelsDevPricingKey` falls back to the source key. Empty ids now\nfall back the same way, for the pricing key and for the exact-only verdict\nalike.\n\nTwo catalogs can also declare the same provider id, which tied the sort and\nleft their order to hash iteration. The unique provider map key settles the\ntie now, matching the order of generation's provider-key walk.\n\n* fix(pricing): track cache-write explicitness instead of inferring it\n\nThe z.ai cache patch recognized a defaulted cache-write by its value - it\nequals `input * 1.25` - which misreads a catalog explicitly publishing that\nsame number. The heuristic was a dodge around `Pricing` only recording\ncache-read explicitness; recording both is the honest shape, so every\nconstructor now says where its cache-write rate came from: published data\nor the derived default. The patch repairs reads and writes independently\nand only ever the derived ones.\n\nReported by cubic on #1541, and it also closes the round-3 finding this\nheuristic had answered halfway.\n\nThe Grok guide's troubleshooting note still credited LiteLLM for\n`--mode calculate` estimates; it now matches the Pricing bullet above it.\n\n* fix(pricing): break exact claim ties by source key within a declared id\n\nTwo catalogs declaring the same provider id are one provider to generation,\nwhich breaks exact-strength ties by `(sourceProviderId, sourceModelId)` -\nand with the ids colliding, by the source model key. The live loader kept\nwhichever catalog arrived first instead, so a same-strength duplicate could\nresolve to a different rate online than in the snapshot. The claim slot now\nrecords the declared id and source key it was claimed under, and an exactly\nequal claim yields to a smaller source key from the same declared id.\nDiffering declared ids were already settled by arrival order, which matches\ngeneration's ascending walk.\n\nReported by pullfrog on #1541.\n\nAlso narrows the new `cache_create_explicit` field to `pub(crate)`: nothing\noutside the crate reads it, and the hawk gate fails the build for exactly\nthat.\n\n* style(pricing): make the cache-write flag private\n\nhawk: pub(crate) was still wider than its uses, which never leave the\npricing module.\n\n---------\n\nCo-authored-by: Codesmith ","tree":"8f735a3cde72f3bfd19d398eb7ef077a38c8047a"},"mode":"0100644","name":"parser.rs","size":43827},{"lastCommit":{"author":{"email":"1560508+ryoppippi@users.noreply.github.com","name":"ryoppippi","when":"2026-08-10T19:37:01.000000+01:00"},"committer":{"email":"noreply@github.com","name":"GitHub","when":"2026-08-10T19:37:01.000000+01:00"},"hash":"215c61c78b643300ae7b99459068e213fdfbb395","message":"feat(grok): add Grok Build CLI usage adapter (#1593)\n\n* feat(grok): add Grok Build usage adapter\n\nParse completed Grok Build turns, map token usage and LiteLLM pricing, and expose focused and unified reports. Discover data from non-empty GROK_HOME or ~/.grok, add schema and CLI integration, and document the source.\n\n* fix(grok): report session activity and project path\n\nSession reports grouped by key then moved the group key into session_id,\nwhich left first_activity, last_activity and project_path unset. Accumulate\nwith SessionAccumulator like qwen/opencode/antigravity, and include session\nmetadata in the JSON report.\n\n* test(grok): expand adapter coverage to antigravity-level\n\nAdd fixture-backed tests for path discovery edge cases, token/pricing\nmapping, timestamp and summary metadata, cross-session dedupe, corrupt\nfile isolation, and session aggregation. Synthesize updates.jsonl rather\nthan committing real session trees.\n\n* fix(grok): bill from recorded costUsdTicks and drop reasoning from totals\n\nTwo corrections to the Grok adapter, both found by comparing its output\nagainst real ~/.grok logs.\n\nCost: every completed turn records `costUsdTicks`, a fixed-point USD amount\nwhere one tick is 1e-10 USD. The adapter discarded it and recomputed from the\npricing table instead. Grok bills each API request separately, but a\n`turn_completed` row only carries the sum over the requests in that turn, so a\nrecomputation cannot place the long-context tier boundary where Grok did and\nlands on either side of the real figure. Across 85 turns of local data the\nrecomputed total is $15.18 against an actual $17.40, and per-day it ranges from\n-46% to +12%. Feeding the ticks through `cost_usd` reproduces the invoice\nexactly in `display` and `auto`, while `calculate` keeps recomputing and `auto`\nstill falls back for turns that recorded no ticks.\n\nTokens: `reasoningTokens` were added to `extra_total_tokens`, which is the\nbucket for tokens *not* already covered by the usage sum. Grok reports\n`totalTokens == inputTokens + outputTokens` and its reasoning count never\nexceeds output, so reasoning is a subset of output and was being counted twice.\nThat inflated the grand total by 113,556 tokens over the same 85 turns.\n\nThe `turn_line` test helper moves its four token counts into a struct so it\nstays under the clippy argument limit.\n\n* chore(rust): reconcile grok branch with main\n\nMerging main brought the adapter count to 15 while the array annotation in the\n`agent_commands_are_exposed_by_independent_crates` test still said 14, so the\ntest crate stopped compiling. Also runs treefmt over the two files the grok\nbranch left unformatted, which the clippy and treefmt checks require.\n\n* feat(grok): read cacheCreationTokens and document interrupted turns\n\nGrok started writing `cacheCreationTokens` alongside `cachedReadTokens` around\nCLI 0.2.118 and still writes it in 1.0.0. The adapter hardcoded\n`cache_creation_input_tokens: 0`, so the field would be dropped the moment it\nwent non-zero.\n\nEvery turn observed so far reports it as zero, so there is no sample proving\nwhether it sits inside `inputTokens` or beside it. `cachedReadTokens` is\nprovably inside — session totals match `logs/unified.jsonl`, where\n`cached_prompt_tokens` is part of `prompt_tokens` — so the sibling field is\ncarved out of the same total. That keeps the three parts summing back to\n`inputTokens`, which is the invariant the adapter already relies on.\n\nAlso documents two limits that surfaced while checking real logs: a session\nkilled mid-turn never writes `turn_completed`, so its usage cannot be reported\nat all, and `logs/unified.jsonl` cannot stand in for it because it records no\nper-request model id.\n\n* fix(nix): include grok adapter in cargo artifact layer\n\nCo-authored-by: Codesmith \n\n* fix(grok): decode UTF-8 project paths and align docs with the single-root contract\n\nDecode percent triplets into bytes so multi-byte cwd segments survive, drop the\nalways-zero discriminator from the loader dedupe key, match the report test helper\nto the parser's reasoning contract, and correct the README surface list plus the\nGROK_HOME multi-root docs.\n\nCo-authored-by: Codesmith \n\n* fix(grok): keep cache-creation tokens in the fallback dedupe keys\n\nCo-authored-by: Codesmith \n\n* fix(grok): make module-local helpers private for hawk\n\nCo-authored-by: Codesmith \n\n* test(grok): cover wide scalars and invalid bytes in path decoding\n\nThe existing case decodes a two-byte scalar end-to-end. This adds a direct unit\ntest for a three-triplet CJK segment, and pins the behaviour for a percent\ntriplet that is not valid UTF-8: it degrades to the replacement character rather\nthan panicking or dropping the rest of the path.\n\n* fix(grok): dedupe only on eventId so distinct turns survive\n\nThe global dedupe rebuilt a content key for entries without an `eventId`, but\nthat key can only be built from what `LoadedEntry` carries, and reasoning tokens\nare not among them. `parse_session_files` deduped its own file using the full\nrecord including reasoning, so two eventId-less turns in the same second with\nidentical input, output and cache counts but different reasoning were kept by\nthe parser and then collapsed here, dropping one turn entirely.\n\nCross-file dedupe is about the same server event appearing in more than one\nsession export, which is what `eventId` identifies. Entries without one are now\nleft alone rather than matched on token counts.\n\nReported by an adversarial review pass. Every `turn_completed` in the logs\nchecked so far carries an `eventId`, so this was not reachable in practice.\n\n---------\n\nCo-authored-by: LeeShunEE <2457013396@qq.com>\nCo-authored-by: Kinema. <54445332+LeeShunEE@users.noreply.github.com>\nCo-authored-by: Codesmith ","tree":"4f14ca749802e6f1c4bb9cae158a76c7f7b7f554"},"mode":"0100644","name":"paths.rs","size":6640},{"lastCommit":{"author":{"email":"1560508+ryoppippi@users.noreply.github.com","name":"ryoppippi","when":"2026-08-15T14:26:44.000000+01:00"},"committer":{"email":"noreply@github.com","name":"GitHub","when":"2026-08-15T14:26:44.000000+01:00"},"hash":"a323dcc233e259cd2bc3500bfa3cb4ff9d9a4f6d","message":"fix(grok): hide empty cache creation column (#1605)\n\n* fix(grok): hide empty cache creation column\n\nFocused Grok reports now suppress the cache-creation column when every selected row reports zero, while retaining cache reads, totals, JSON fields, and unified report columns. The shared table renderer exposes this as an opt-in display option so future non-zero Grok records remain visible. Add focused regression coverage and document the source-specific table behavior.\n\n* test(grok): harden cache column regression\n\nRead the shared renderer default in the regression test so the test guards the public default configuration. Link the Grok guide to the JSON output documentation for the stable cacheCreationTokens schema.","tree":"e814c0c4e72a1662b2cef7fec797cf95842e748a"},"mode":"0100644","name":"report.rs","size":7395}],"lastCommit":{"author":{"email":"1560508+ryoppippi@users.noreply.github.com","name":"ryoppippi","when":"2026-08-15T14:26:44.000000+01:00"},"committer":{"email":"noreply@github.com","name":"GitHub","when":"2026-08-15T14:26:44.000000+01:00"},"hash":"a323dcc233e259cd2bc3500bfa3cb4ff9d9a4f6d","message":"fix(grok): hide empty cache creation column (#1605)\n\n* fix(grok): hide empty cache creation column\n\nFocused Grok reports now suppress the cache-creation column when every selected row reports zero, while retaining cache reads, totals, JSON fields, and unified report columns. The shared table renderer exposes this as an opt-in display option so future non-zero Grok records remain visible. Add focused regression coverage and document the source-specific table behavior.\n\n* test(grok): harden cache column regression\n\nRead the shared renderer default in the regression test so the test guards the public default configuration. Link the Grok guide to the JSON output documentation for the stable cacheCreationTokens schema.","tree":"e814c0c4e72a1662b2cef7fec797cf95842e748a"}}