Together AI teamed with Ollama to explain how to keep long coding-agent sessions fast without reprocessing the full transcript every turn. The obvious reading is that prefix caching solves agentic inference.
I do not buy that it solves it. I think it exposes how brittle agents still are.
The claim itself is accurate as far as it goes. Zain Hasan and Parth Sareen of Ollama describe an agent pattern with large input and small output, where each turn resends the system prompt, tools, files and history to get back a short tool call. Prefill dominates that work, they write, while decode only streams a few dozen tokens. Ten equal turns without caching process 55 units of prompt versus 10 with caching, and 100 turns process 5,050 versus 100, according to their idealized math.
Cache matches from the first token and stops at the first change.
That one rule carries the whole harness section, and it is useful. Put a timestamp, git status or changing directory in the system prompt and every turn becomes a full re-read, the authors warn. Load a new tool definition into the top of the prompt and you invalidate everything after it. Raise reasoning effort mid-run in GLM-5.3-Flash and you do the same, because that model writes effort into the first system block. The fix list is strict: keep the system prompt stable, append tool outputs at the end, truncate files with explicit ranges, and pick effort once. Their timezone example makes the point, when an agent reads lines 1 to 100 of a 640-line calendar.py and misses convert_timezone at line 420.
To be fair, the harness advice is concrete and most teams break at least one of these rules. What is missing is proof on the engine side. The published text stops just as it reaches whether the cache is still resident and routed to the same GPU. There are no cache-hit rates, no time-to-first-token curves, no cost per 100-turn session. Scale makes that gap matter, since community deployments list GLM-5.3 at 744B parameters and GLM-5.3-Flash at 321B, with NVIDIA now listing an FP4 local quant for self-hosting. Caching a 300B-plus session across 100 turns is not free memory.
If you want long agents, fix your harness before you rent bigger GPUs.





