Most people fight AI hallucinations by asking the model to read a page and confirm what it found. That sounds reasonable, but it has a flaw. The confirmation is one more thing the model generated. A different approach skips the model’s opinion and checks the claim against the raw source with a plain string match. It’s simpler, and it holds up much better.
This idea comes from a Reddit post in r/PromptEngineering. The author used Claude to do SEO for a brand-new site. The worst failures were never bad advice. They were confident made-up facts: pages assumed to be indexed, stale prices, menu paths that don’t exist, and schema that didn’t match the page. The site still went from zero to 4.4k Google clicks in three months, but only after they wrote rules to stop each of those failures.
The Key Idea: Don’t Let the Model Grade Its Own Homework
A summary is generated text. If you ask Claude to read a page and tell you whether it mentions a price of €25, it can say yes when the answer is no. A check like "€25" in html cannot do that. It returns true or false against the actual source, and the model has no say in it.
Old Way vs. New Way
The old way: Paste the page, ask Claude to summarize it, then ask whether the claim is true. When the source is blocked or unclear, the model fills the gap with a confident guess. Scores come out as a number with no explanation.
The new way: Treat the model as the one making claims and treat code as the one verifying them. Every claim gets checked against raw source text. Anything unreadable gets labeled as unreadable. Every score is tied to a written rubric you can inspect.
The difference is who holds the authority. In the old way, the model both makes and approves its own claims. In the new way, those jobs are split.
Three Rules You Can Copy 🔍
1. Verify with a literal string match, not a summary. Each claim the model makes gets tested against the raw HTML or text. If the exact string isn’t there, the claim fails.
2. Give the model a way to say “I can’t know that.” If a page sits behind a login, a bot challenge, or robots.txt, the output should say “unreadable” instead of guessing. The same goes for rankings and search volume. The model shouldn’t predict them, because nothing can do that reliably.
3. Tie every score to a written rubric. One failed check caps the score, so you can see exactly why a site got its number. Without a visible rubric, an audit score means nothing.
Practical Steps
- List the factual claims your workflow depends on: prices, URLs, indexed status, menu paths, schema fields.
- For each claim, write a check that runs against the raw source, such as a substring match or a regex.
- Add an explicit “unreadable” or “unknown” output option to your prompt, and say when to use it: login walls, bot challenges, blocked pages.
- Remove any request for predictions you can’t verify, like rankings or search volume.
- Write your scoring rubric in plain text. Define which failed check caps the score and at what level.
- Every time the model invents something, turn that failure into a new rule. That’s how the author built their list.
The same logic works well outside SEO. One commenter, u/No_Engine6211, said they use a similar approach for data extraction and called the string match trick severely underrated. People keep trying to get the model to read and understand, when they should just be checking. Another commenter, u/FragrantArt8270, pointed out that asking for cited sources also forces the model to back up what it says. Citations pair well with string matching, since you can verify the quoted text actually appears in the source.
A quick note on the source: the author packaged these rules into a paid set of 14 Claude skills and admits to being biased. You don’t need that to use the idea, since all three rules work in any prompt.
Your Turn
Pick one workflow where Claude states facts you later have to double-check. Find the one claim that has burned you most often, write a string match for it, and add an “unreadable” escape hatch to the prompt. Run it on your next batch and see how many confident answers drop out. Then tell us in the comments: what’s the best trick you’ve found to stop a model from inventing facts?
Frequently Asked Questions
Q: Does citing sources actually stop a model from making things up?
Citations help, but a model can attach a convincing source to a claim it invented. Treat every citation as something to verify: check that the quoted text literally appears in the source, the same way you’d check a price or a menu label. If the text isn’t there, the claim fails.
Q: Should the rubric checks run separately or in one prompt?
The author ties each score to a written rubric where one failed check caps the total, so you can see exactly why a site got its number. The post doesn’t say whether to split the checks into separate runs or combine them in one prompt. Choose the setup where every check still leaves a visible result you can audit, because a score with no trail is the thing to avoid.
Q: How do I get a model to admit it can’t access a page?
Give it a named status for that case, such as ‘unreadable’, and say plainly that this answer is better than a guess. One reader spent months fighting invented pricing pulled from cached copies of paywalled pages, and the fix was giving the model permission to say it couldn’t access the page. Do the same for things nobody can predict reliably, like rankings or search volume.
Q: Does the string match idea work outside SEO?
Yes. One practitioner uses it for data extraction, and the logic carries over to any job where the output has to exist in the source, such as prices, names, dates, or quotes. Fewer moving parts also means fewer things to break.
Summaries hallucinate. What stopped Claude from making up facts in my SEO work was a literal string match
by u/DutyOnly4308 in PromptEngineering