Tagged caching

Caching LLM Responses Safely: What to Cache and What Never To

Caching LLM answers saves tokens and latency — and can leak one user's data to another if done carelessly. A safe design: cache only answers that are identical for everyone, key on normalized question plus content version, expire, and lock the cache down.