Most AI pipelines handle a changed input the same way: rerun the whole prompt, wait for the whole output, pay for the whole output. One engineering team looked at that habit and built something faster.
Key Idea
The trick is called Revision Prompting, shared by Reddit user u/Dry_Rabbit_1123 in r/PromptEngineering. Instead of feeding the model a fresh input every time something changes, you feed it the old input, the old output, and a diff of what changed. Then you ask for a patch, not a rewrite.
Think translation pipelines, invoice parsing, contract redlines, anything that runs the same instruction on updated data over and over. A single typo fix used to trigger a full regeneration of the entire output, with the model quietly rewording paragraphs that never needed to change. That’s non-determinism doing what it does, and it shows up in places you wouldn’t expect. A legal team correcting one date field ends up with a document where three unrelated clauses read differently on the next pass. A localization pipeline fixing one mistranslated word regenerates the whole page and half the surrounding sentences shift tone. None of that was requested. It just happens because the model is starting from scratch every single time, with no memory of what “correct” already looked like.
Revision Prompting treats the old output as ground truth instead of throwing it away. The model’s job shrinks from “write this document” to “tell me what changes.” That’s a much narrower task, and narrower tasks are cheaper, faster, and more predictable.
Old Way vs New Way
🔴 Old way: input changes, full prompt reruns, full output regenerates, you pay for every token including the untouched 90%. Every run is a fresh roll of the dice on wording, formatting, even structure.
🟢 New way: input changes, model sees only the diff, model returns a patch, you apply it to the old output, untouched sections stay byte-identical. The parts of your output that didn’t need to change literally can’t change, because the model never touches them.
The team behind this shared real numbers: processing time dropped by roughly 80%, cost dropped by about 65%. Results depend heavily on the task, some pipelines will see more, some won’t benefit at all. A pipeline where inputs change by a sentence or two sees the biggest win. A pipeline where every update rewrites half the source document won’t see much of one, because there’s not much stable content left to protect.
There’s a second win that matters just as much: consistency. Full regeneration means the output looks slightly different every run, even when nothing meaningful changed. A patch only touches what actually changed, so your output diffs become something you can actually review. That matters for anyone doing code review on AI-generated content, or anyone who has to explain to a stakeholder why last week’s version and this week’s version don’t match on a line that was never supposed to move.
How to Try It
- Keep your original input/output pairs around. If you’re already logging pipeline runs, you probably have this already. If you’re not, start now, you’ll need at least one prior pair before Revision Prompting can work at all.
- When the input updates, generate a diff. Unix diff format works for plain text, JSON Patch works for structured data. Pick whichever format your model already handles well, don’t force a format just because it’s the standard one.
- Prompt the model with the instruction, the old input, the old output, and the diff. Ask for a patch, not a new output. Be explicit about the format you want the patch returned in, that’s where most early failures come from.
- Apply the patch programmatically to the stored output. Don’t hand this step back to the model, a deterministic apply step is what keeps the untouched sections actually untouched.
- If a large chunk of the input changed, skip all of this and rerun the prompt normally. Revision Prompting is built for small, incremental changes, not full rewrites. Set a threshold, if more than some percentage of the input changed, fall back to the old full-regeneration path automatically.
One caveat worth taking seriously: several commenters flagged the same risk, you need a way to tell “patch failed” apart from “patch applied but silently dropped a section.” Build a validator before trusting this in production. Don’t assume the patch always lands clean. A cheap first pass is diffing the patched output’s length and structure against expectations before it ships anywhere near a customer.
Worth Testing This Week
If any part of your stack reruns the same prompt on slightly different data, this is worth testing. Start with your cheapest, most repetitive pipeline, measure the before and after, and see if the savings hold up for your task.
Frequently Asked Questions
Q: When should I actually use Revision Prompting?
It’s best for small, surgical input changes: a typo fix, one field update, that kind of thing. If half your input changed, just re-run it normally; the savings disappear and you risk the model missing cross-dependencies. Test it on your pipeline with small diffs first and see if it actually saves you money.
Q: How do you know the patch didn’t break something?
Yeah, that’s the hard part. You can’t automatically tell if the model said “this section doesn’t need changing” or “I missed that the input changed here.” Some teams compare the patched output against a full re-run, use checksums on critical sections, or spot-check manually. The validation strategy really depends on how risky it is if something goes wrong.
Q: Does this work for everything, or just certain tasks?
It’s task-dependent. The author confirmed it works great for some pipelines (like translations and data extraction) but fails on others. How well it works depends on what you’re asking the model to do and which model you’re using. Definitely test it on your specific use case first.
Q: What’s the actual failure rate? Does the patch ever miss changes?
That’s still mostly anecdotal right now. No solid numbers from the community yet. It definitely varies by task and model. If you try this, set up monitoring to catch when patches diverge from a full re-run. That’ll give you real data on whether it’s worth the risk for your pipeline.
Revision Prompting: A trick to avoid regenerating the whole output when only 10% of the input changed.
by u/Dry_Rabbit_1123 in PromptEngineering