New data: same model, same task, identical output quality, and one run cost four times more than the other. The only difference was how the context got handled.
A Redditor on r/PromptEngineering, u/ClickOk5811, wanted an actual number instead of a guess. So the author ran the same coding task twice, in two separate sessions, same model both times. One session got worked the way most people default to: paste full files whenever something changes, and let the conversation run long without ever summarizing earlier turns. The other session was deliberate. Only the functions relevant to the current step went into the prompt. A short summary stood in for the earlier turns instead of the full transcript. And the stable parts of the prompt, system instructions and reference material, stayed byte-identical across every single call.
The careless session cost 4x more than the deliberate one. Output quality didn’t meaningfully change between the two. If anything, the trimmed version stayed more on-target, likely because there was less irrelevant material sitting in context and competing for attention on each response.
Where the Gap Actually Came From
Here’s the part worth sitting with: caching drove more of that 4x gap than the file trimming did. Prompt caching only kicks in when the cached portion of a prompt matches the previous request byte for byte. Break that match by even one character and the discount disappears, quietly and completely.
That failure mode is easy to trigger without noticing. A timestamp or a session-specific detail placed near the start of a prompt kills the cache hit. Nothing throws an error. Nothing in the response flags it. The bill just stops reflecting the discount it should have gotten, and most people never think to go check.
This lines up with a blind spot the original poster called out directly. Prompt engineering discussion spends a lot of energy on getting the wording right. Almost none of it goes toward the fact that context shape carries its own cost curve. That curve is completely separate from whether the wording is any good, and most people never account for it.
3 Practical Applications
- ⚙️ Order your prompt so caching survives. Put system instructions and reference material first, and keep that exact block unchanged across every call in a session. Push anything that varies, timestamps, session IDs, current file state, to the end of the prompt instead of the front. This one change is what separated the two test sessions the most.
- ⚙️ Summarize instead of replaying the full transcript. Once a session runs long, swap the earlier turns for a short summary of what already happened. The model doesn’t need every prior message to stay on track. It needs the conclusions that still matter for the current step.
- ⚙️ Trim context down to what the current step touches. Stop pasting entire files on every edit. Pull in only the functions or sections that are actually relevant to the task sitting in front of you right now. Everything else is just noise the model has to sort through.
Tips and Pitfalls
- Caching needs an exact byte match on the cached prefix. One misplaced timestamp and the session pays full price with zero warning.
- There is no error message when a cache miss happens. The only way to catch it is checking your own usage numbers after the fact.
- Trimming isn’t only about tokens saved. Less irrelevant material in the window seems to help the model stay focused on the actual task too.
- Stable content goes first, dynamic content goes last. Getting that order backwards is the single most common way people break caching without realizing it.
- Don’t assume a long conversation is free just because you’re not pasting new files. An unsummarized transcript still gets sent, and billed, on every single turn.
I’ve run into a quieter version of this while scaling up automation work. It’s the kind of cost that stays invisible until you actually sit down and measure it, exactly like the original poster did here. As u/MeringueAdmirable381 put it in the replies, people obsess over clever wording, but context hygiene is probably the bigger deal once a session runs long.
If you’re running long coding sessions with Claude, check where your stable content sits in the prompt. That’s worth five minutes before your next round of wording tweaks, and it might matter more than the tweaks themselves. The original poster wrote up the full before-and-after numbers, worth a look if you want the actual data behind all of this.
Frequently Asked Questions
Q: Does trimming context actually hurt output quality?
Nope. In fact, it often helps. When you keep only the functions that matter and swap summaries in for full transcripts, there’s way less noise competing for the model’s attention. The messy version with all the files wasn’t noticeably better output-wise; if anything, the trimmed one stayed more on-target.
Q: How does the cache silently break without telling you?
Caching needs the cached portion to stay byte-for-byte identical. Drop a timestamp or session ID at the start of your prompt instead of the end, and you get a silent cache miss. No error, nothing in the response warns you. Your bill just doesn’t get the discount it should.
Q: Should I spend more time on wording or context structure?
Context structure wins, especially over long sessions. Everyone optimizes for clever wording, but context hygiene (only including what actually matters, not pasting full files every time) moves the needle way more on cost. Good wording still helps, but disciplined context beats it.
Q: How much of the 4x cost difference was caching versus file trimming?
Caching was the heavy lifter. File trimming helped, sure, but the real savings came from keeping stable parts (system instructions, reference material) byte-identical across calls so they’d actually cache.
Ran the same coding session twice, once with careless context, once deliberate. Same output quality, 4x cost difference.
by u/ClickOk5811 in PromptEngineering