Yesterday a Redditor shared a habit that sounds almost too simple. Run every important prompt through Claude, GPT and Grok before you trust the answer. Then they packaged the habit into a Windows app called CouncilAI. The app is the new part. The habit is the part you can use today, for free, with three browser tabs.
What’s new
CouncilAI sends one prompt to several models at once. You get three modes:
- Side by side: compare the answers next to each other. This is the fastest way to spot where the models split on tone, length or facts.
- Sequential: chain the models, so one answers and the next one picks it up. Think of it as a relay race where each runner is allowed to fix the last runner’s mistakes.
- Council: one model reads all the answers and combines them into a single response. It works like an editor who has read three drafts and keeps the best parts of each.
It runs on your own API keys, so there’s no subscription. You pay only for the tokens you use, which for a short prompt across three models is usually pennies. It’s a young project, and the post had one upvote when I saw it. I haven’t tested the app myself. The idea behind it is worth stealing whether or not you ever install it.
The twist
The app isn’t the interesting part. The disagreement is.
When three models give you three different answers, most people pick the one they like best. This person does something else. They treat the disagreement as a bug report on their own prompt. If the models can’t agree, your prompt probably left room for three readings. Say you wrote “make it shorter.” One model cuts a sentence, another cuts half the email, and a third rewrites it as bullets. None of them is wrong. Your prompt just never said how short, or what to keep. You never had to ask a model to review the prompt. The models showed you where it was vague just by answering it.
They noticed two more things along the way:
- Models fail in different ways. One ignores your format instruction and another follows it so hard it gets stiff. Side by side, you can see which lines of the prompt are doing the work and which ones are being quietly skipped.
- Chaining beats one long prompt. One model drafts, a second critiques and improves, and a third writes the final version. That tends to win on emails and summaries, because each step has one small job instead of five competing ones.
Try it by hand in 10 minutes 🧪
- Pick one prompt you actually rely on. A client email template works, or a summary prompt. Choose something you run often, since that’s where a small fix pays back the most.
- 🔍 Paste it into three different models. Same words, fresh chat each time, with no memory or custom instructions tilting the result.
- Compare where the answers split. Is it tone, length, structure, or the facts themselves? Write the split down in one line, like “length differs by 3x.”
- ✂️ Rewrite only the vague line that caused the split. Don’t pile on extra instructions. If the length split, add a number. If the tone split, add one example sentence in the voice you want.
- Run all three again. Repeat until the answers mostly agree. Two or three rounds is normal.
- 🔗 For the final polish, chain it: model A drafts, model B critiques, model C finalizes. Tell model B to list three weaknesses before it rewrites anything, so it doesn’t just rephrase.
Pro tips
- Use models from different families. One commenter pointed out that reviewers from the same family share blind spots. Only count agreement as real when two different families land on the same thing independently.
- Keep the models from seeing each other’s answers during the test. Otherwise you’re measuring copying, not clarity. Save the cross-reading for the chaining step, after you’ve already collected the independent answers.
- Don’t chase perfect agreement. Facts and format should line up. Style can differ. If you force identical wording, you’ll end up with a stiff, over-specified prompt that breaks the moment you switch models.
- Save the version of the prompt that passed. Keep a small notes file of your tested prompts and the line you fixed in each. After a few weeks you’ll see the same kinds of vagueness showing up again and again, and you’ll start catching them before you hit send.
Not everyone buys into this. Another commenter tested everything at the top tier for months and settled on one max plan. That’s a fair call if you only write for one model. The multi-model test pays off when the prompt has to survive different models, or when it’s going into a workflow that will run a thousand times. A tiny ambiguity that costs you nothing in one chat can quietly cost you hundreds of bad outputs at scale.
Your move 🚀
Grab one prompt today and run it past three models. Then tell me which line of it caused the split. I’d bet it’s the one you thought was obvious. If you’d rather skip the tab juggling, CouncilAI is on the Microsoft Store: https://apps.microsoft.com/detail/9N5L747K559D?cid=reddit_promptengineering
Frequently Asked Questions
Q: How do you stop your reviewer models from sharing the same blind spots?
One commenter keeps the models from seeing each other’s answers and picks two different model families, since models from the same family tend to miss the same things. They only trust a finding when both models flag the same underlying fact on their own. Keeping the reviewers read-only helps too, and every finding gets checked against the actual project before anyone acts on it.
Q: Is it worth testing across models, or should you pick one and tune for it?
One commenter tested everything for months, then settled on a single top-tier plan and hasn’t looked back. They still keep a second subscription because they think Google has the best data and likes its model. Others say comparing several models is the fastest way to find where a prompt is vague, so the right choice depends on whether you want depth with one model or a prompt that holds up across all of them.
Q: Is there a tool that checks prompts and context files for you?
One commenter recommended reporails, which they said has been checking rules for about a year and recently added a UI for managing context sessions. It’s a good starting point if you want rule-based checks instead of comparing outputs by hand. Try it on one of your own prompts first to see whether its results match what you’d flag yourself.
Testing the same prompt across Claude, GPT and Grok changed how I write prompts
by u/Fun_Statement_6108 in PromptEngineering