Every step of an agent, re-sends the whole conversation, and a browser agent's conversation grows with each page and screenshot. If that history is never served from cache, you pay full price for the same tokens over and over. OpenAI's write-up of Asana's browser test agent (published 9 October 2026) shows a fix any solo builder can copy: keep the start of the context byte-for-byte stable so it caches, and prune old material in batches instead of on every step.
The pattern
Cache-stable context with batched pruning means two rules. First, cache the growing history, not just the fixed instructions. Second, let disposable material (screenshots, raw page dumps) pile up to a threshold, then cut it back in one go, so the earlier history stays unchanged for long stretches between cuts.
In Asana's case, the agent had been caching only "its fixed instructions and tool definitions, but not the growing history of page text and screenshots it gathered", according to OpenAI's case study. It also "dropped older screenshots and trimmed text at nearly every step". Because each edit changed the history, caching the history by itself would not have helped. The fixes the study tested included "extending caching to the agent's browsing history" and "removing screenshots in batches rather than at every step". The best-performing policy let screenshots accumulate to 20 before cutting back to the most recent one, which, in the page's words, "kept earlier history unchanged for longer stretches between removals".
The result, per OpenAI: 89% of the input came from cache at 5% of the uncached price. On the same model (GPT-6.1 Sol) with the larger history budget, the new caching and screenshot policy alone cut cost 4x, from $1.97 to $0.47 per run. The page's 76x headline, and its run-time drop from at least 22.5 minutes to about four, also include a switch from an earlier model.
When to use it and when not
Use it when:
Your agent runs many steps per session (browsing, research, QA, long coding sessions).
Each step appends bulky, short-lived observations such as screenshots, HTML or tool output.
Steps happen close together, inside the provider's cache lifetime.
Skip it when:
Calls are one-shot or the prompt is short. Providers set minimums: OpenAI's guide gives 1,024 tokens for GPT-5.6 and later, and Anthropic's minimum varies by model.
Steps are minutes apart. Anthropic's cache has a 5-minute default lifetime (a 1-hour option costs extra); OpenAI keeps a prefix eligible for 30 minutes after its most recent write or reuse on GPT-5.6 and later.
You must rewrite early context every step (for example, a running summary at the top). Move it to the end or accept the misses.
How it works (input → step → output)
Input: fixed instructions and tool definitions, then the append-only history of observations and actions.
Step: send the full context with a cache marker at the end of the history. Everything up to the previous step's marker matches the cached prefix, so only the newest turn is billed above the cached rate. Both providers match on exact prefixes: OpenAI says "cache reuse requires the entire rendered prefix to match", and Anthropic says cache hits require "100% identical prompt segments, including all text and images".
Prune check: if retained screenshots exceed the threshold (20 in Asana's study), drop all but the latest. That one call misses the cache and rewrites it; the following calls hit again.
Output: the model's next action, plus cache usage numbers you log per call.
Components
Stable header: instructions and tools, never edited mid-run. On Anthropic, changing tool definitions invalidates the entire cache.
Append-only history: no timestamps, counters or reordering inside earlier turns.
Cache marker: automatic on OpenAI; on Anthropic, a
cache_controlbreakpoint (up to four per request).Pruner: a threshold rule for screenshots and a text budget. Do not starve the agent: in Asana's study, raising the history budget from 120,000 to 480,000 characters took runs producing an answer from three of 18 to all 18.
Meter: per-call cache usage, covered below.
Memory across sessions is a separate layer; see the mem0 memory stack.
Minimal implementation
HEADER = [system_instructions, tool_definitions] # never edit mid-run
history = []
MAX_SCREENSHOTS = 20
TEXT_BUDGET_CHARS = 480_000
def step(task):
msgs = HEADER + history
mark_cache_breakpoint(msgs[-1]) # Anthropic: cache_control; OpenAI: automatic
resp = llm.call(msgs)
log(resp.usage.cached_tokens, resp.usage.input_tokens)
obs = run_tool(resp.action) # page text + screenshot
history.append(resp.message)
history.append(obs)
if count_screenshots(history) > MAX_SCREENSHOTS:
history[:] = keep_latest_screenshot_only(history) # one batched cut
if total_chars(history) > TEXT_BUDGET_CHARS:
history[:] = drop_oldest_text(history) # also batched
return resp
The key line is the prune condition: prune only when a threshold is crossed, because pruning on every step changes the history and breaks the cache.
Reading cache hit rates from the response
Both APIs report caching per call.
OpenAI: read
usage.input_tokens_details.cached_tokensandcache_write_tokens, alongsideusage.input_tokens. Hit rate iscached_tokens / input_tokens. The OpenAI guide's example shows 12,000 of 15,000 input tokens cached on a second request.Anthropic: the response splits input into
cache_read_input_tokens,cache_creation_input_tokensandinput_tokens, whereinput_tokenscounts only tokens after the last breakpoint. The docs give total input = cache reads + cache writes + input_tokens, so hit rate iscache_read_input_tokensdivided by that total.
Plot hit rate per step. A healthy run shows a sawtooth: high between prunes, one dip at each prune, then recovery. A flat low line means something near the top changes every call.
A worked cost example
Take Claude Sonnet 5.5 at Anthropic's listed prices as of 18 October 2026: $2 per million input tokens, $2.50 per million for 5-minute cache writes and $0.10 per million for cache hits. Suppose one step sends 100,000 input tokens, of which 95,000 were cached on the previous step and 5,000 are new.
No caching: 100,000 × $2/M = $0.20 for the step.
Cache-stable: 95,000 × $0.10/M = $0.0095 for reads, plus 5,000 × $2.50/M = $0.0125 to write the new turn, about $0.022 in total.
That is roughly 9x cheaper per step, before output tokens. Each prune rewrites the shortened history at the write rate, so keep prunes rare. On OpenAI the logic is similar: the guide says cached tokens are "discounted up to 95%", and for GPT-5.6 and later cache writes cost 1.25x the standard input rate.
Failure modes and evals
Image churn: on Anthropic, adding or removing images anywhere in the prompt affects message-block caching. Eval: misses should line up with prune events only.
Over-pruning: a small text budget saves tokens and loses answers, as the 3-of-18 result shows. Eval: completion rate on a fixed task set at two budgets.
Cache expiry: slow tools push steps past the TTL. Eval: log the gap between calls.
Cache-busting checklist
Timestamps, request IDs or user names in the system prompt. Move them after the stable content, as the OpenAI guide advises.
Reordering or regenerating tool definitions. OpenAI lists tool names, descriptions, schemas and ordering as part of the cached prefix.
Editing old turns, such as trimming text in place. Append instead.
Running summarisation or compaction every step. OpenAI notes compaction and truncation "can change the prefix and reset cache reuse".
To tune the harness itself once you have these metrics, see Harness^2 on self-improving agent harnesses.
Build-it-this-weekend exercise
Pick one multi-step agent you already run and choose 10 fixed tasks.
Run it as-is and log cost, cached tokens and completion per task.
Move every changing value out of the header, add a cache marker at the end of history, and replace per-step pruning with a threshold (start at 20 screenshots).
Re-run the same 10 tasks at two text budgets and compare cost per completed task, not cost per run.
Sources
From today's Daily Briefing → DeepSeek's $0.75 Search API, 71.5% of AI Tokens Go Open-Weight, and How Reddit Lost Its Grip on ChatGPT's Answers