• [...] Rebuilding the top-level system prompt or tools array between requests in the same conversation.
Many people unknowingly do this (at a high cost to them because of the cache busts), this change will finally force them to stop.
Especially if you're generating your system prompt via a template that can change mid conversation, it's so easy to fall into this trap.
• [...] Rebuilding the top-level system prompt or tools array between requests in the same conversation.
Many people unknowingly do this (at a high cost to them because of the cache busts), this change will finally force them to stop.
Especially if you're generating your system prompt via a template that can change mid conversation, it's so easy to fall into this trap.