TL;DR: A Reddit user built a second prompt that audits AI research claims before they get written up, and it turns out most summaries fail not at finding info but at overstating what that info actually proves.
Here’s the thing nobody talks about with AI research tools. The model isn’t bad at finding sources. It’s bad at knowing the difference between “this source mentions it” and “this source proves it.” u/Harshit-24 over in r/PromptEngineering ran into that exact wall and built a fix worth stealing.
The Problem
Ask an AI to summarize research and you get confident-sounding paragraphs. Sounds airtight. Except half the time the model jumped from a vague mention to a hard conclusion, and nothing in the output tells you that happened. Two new job listings in Europe becomes “European expansion” with zero pushback, even though those could just as easily be remote roles or backfills.
The Fix: A Separate Verification Pass
Instead of one prompt that finds and writes, run two. First pass gathers sources. Second pass audits every claim before it’s allowed anywhere near a final summary. Here’s the actual prompt:
You are the verification pass, not the writer. For every material claim, return: the claim; its source; the exact supporting passage; whether the source directly supports it, merely suggests it, or does not support it; the strongest counterevidence; any freshness risk; the missing evidence; and the next verification step. Do not repair unsupported claims from memory. End each row with KEEP, WEAKEN, or REMOVE. Only synthesize claims marked KEEP.
Why It Works
- 🔍 Forcing an exact passage stops the model from treating a broad homepage as proof of a specific number.
- ⚖️ “Strongest counterevidence” makes the model actively hunt for reasons it might be wrong, instead of just cheerleading its own conclusion.
- 🚦 KEEP / WEAKEN / REMOVE is a hard gate. Nothing gets polished into confident prose until it survives the audit.
Use Cases
- Competitive intelligence, where one misread signal turns into a wrong strategic call
- Investment or market research, where “suggestive” and “proven” have very different price tags
- Any agentic research setup where the same model both discovers and writes, which is exactly the setup that lets unsupported claims slide through
Prompt of the Day
Run your next research summary through the two-pass split above. Discovery pass gathers, verification pass grades, and only KEEP survives to the final write-up.
Worth trying on whatever research workflow you’re running right now. What would you add to the audit to make it even harder to fake?
This audit prompt made my research summaries less confident—and more useful
by u/Harshit-24 in PromptEngineering