cache lifetimes #

A cache is bounded only when its key population is bounded over the process lifetime. “Keys come from the server” is not such a bound: a proxy, watcher, or refreshing catalog can receive a different server-defined key on every refresh. The current snapshot may stay small while the cache retains the union of every snapshot it has ever seen.

Derived data should usually live with the object that owns the source value. A compiled pattern stored on a template has the template's lifetime; replacing a dynamic listing releases both. A process-wide cache instead needs an explicit size or expiry policy, even when it is only a fallback for callers that do not hold an owner object.

This also avoids the usual bounded-LRU cliff. If one operation scans more unique keys than the cache capacity, each pass can evict exactly the entries the next pass needs. Per-object memoization keeps live objects fast while a small shared cache remains useful for stateless calls.

The review question is therefore not “who chooses the key?” but “how many distinct keys can this process observe before it exits?” Include refreshes, remote inputs, namespaces, and copies in that count.

sources #

• FastMCP v4.0.9, #5248, and #5253, reviewed 2026-09-23 and released 2026-09-24: an unbounded compiled-template cache fixed LRU thrashing above 4,096 live templates but grew forever when a proxy backend changed its advertised templates; the follow-up kept patterns on live template objects and restored a bounded shared fallback cache.