Your AI is likely hallucinating right now, and the scariest part is that it often sounds smarter when it lies. I recently came across a brilliant breakdown by an AI educator who analyzed the latest research to solve this exact problem. He explains that whether you are using ChatGPT, Claude, or Gemini, these models are designed to predict the next likely word, not to act as a database of absolute truth.
This fundamental design flaw means models often prioritize sounding coherent over being factually correct. The video creator highlights a hilarious example where older models would confidently invent a description for a non-existent “seahorse emoji” simply because the request implied one should exist. To combat this, the expert suggests shifting our approach from simple chatting to engineering verification workflows. He details a hierarchy of methods ranging from simple prompt tweaks to complex multi-model audits that force the AI to check its own work. It turns out that asking an AI to think step-by-step, a popular piece of advice, can sometimes make hallucinations worse because the model builds a logical narrative structure around an initial false premise. The solution lies in grounding the AI with external data and forcing it to critique its own output before you ever see it.
Grounding AI with Retrieval Augmented Generation (RAG)
The most effective way to stop an AI from guessing is to give it the answers beforehand. The expert refers to this as Retrieval Augmented Generation (RAG), which essentially acts as providing the model with a cheat sheet. Instead of relying on its hazy training memory, the model is forced to look up information from documents you provide. The video highlights Google’s NotebookLM as the current champion for this workflow because it automatically cites its sources inline.
However, simply uploading a document isn’t enough if the source material itself is biased or incomplete. The author suggests a workflow using three specific verification prompts to ensure your “cheat sheet” is solid. First, you ask the model to identify contradictions. You instruct it to look only at the provided sources and flag areas where they disagree with each other. This is crucial for catching conflicting data points before they merge into a single, confused answer. Second, you prompt it to find gaps. You ask the AI to list important questions or subtopics that are missing from the uploaded data. This helps you realize what you don’t know, preventing you from drawing conclusions based on partial pictures. Finally, the expert recommends asking for missing perspectives. You ask the model to identify contrarian or lesser-known viewpoints that aren’t represented in your sources. This helps break echo chambers by revealing what the mainstream sources might be omitting.
📌 The “I Don’t Know” Protocol and Confidence Labels
If you aren’t using a full RAG system and are just chatting with a standard bot, you need to change how you ask questions. The original poster emphasizes that models are people-pleasers; they try to answer even when they shouldn’t. To fix this, you must explicitly give the model permission to fail. The expert suggests adding a strict instruction to every prompt: “If the answer is not clearly in the text, say ‘I don’t know’ instead of guessing.”
He takes this a step further with a technique called “Confidence Labeling.” This involves instructing the AI to add a confidence tag, such as High, Medium, or Low, in parentheses after every main claim it makes. At the end of the response, the model must list anything it was unsure about. This forces the model to perform a self-assessment before presenting information as fact. When you see a Medium or Low tag, you know immediately that you need to verify that specific point manually. This simple addition cuts out a surprising amount of fake confidence and makes it much safer to use LLMs for research or learning new topics.
📌 The Chain of Verification Method
Standard “Chain of Thought” prompting (asking the AI to think step-by-step) has a hidden danger: if the first step is a hallucination, the AI will use perfect logic to justify that lie in the subsequent steps. To counter this, the author details a method called “Chain of Verification.” This separates the generation of content from the checking of facts.
This process involves four distinct steps. First, you let the model generate its initial response. Second, you ask the model to scan that response and extract every factual claim, like dates, names, and statistics, and convert them into a numbered list of questions. Third, and this is critical, you open a brand new chat window so the model isn’t biased by its previous context. You feed it the list of questions and instruct it to use its search tool to verify each one individually, noting any conflicts. Finally, you take those verified answers back to the original prompt and ask the model to rewrite the answer using only the verified facts. This prevents the model from doubling down on its own errors and ensures the final output is rigorously fact-checked.
📌 The Auditor and The LLM Council
For high-stakes decisions, you shouldn’t rely on a single model’s perspective. The expert introduces the concept of “The Auditor,” based on the idea that LLMs are much better at critiquing content than creating it. After getting an answer from ChatGPT, you can paste that answer into a different model, like Claude or Gemini, and ask it to audit the response for logical fallacies, missing context, or overstated claims.
He also describes a technique called “Self-Consistency.” If you ask a model a complex business question once, it might hallucinate. But if you ask it the same question five times in fresh chat windows, the truth tends to be the most consistent answer, while hallucinations vary. Taking this to the extreme, the author references a concept called the “LLM Council,” inspired by Andrej Karpathy. This involves sending the same prompt to ChatGPT, Gemini, Claude, and Grok simultaneously. You then have the models anonymously review each other’s answers. The disagreements between the models are often where the most valuable insights live, as they highlight uncertainty or nuance that a single model would gloss over. There are even tools mentioned, like ChatHub or specific GitHub repositories, that can automate this “council” meeting for you.
If you want to stop getting tricked by confident AI, you have to start treating them like interns who need their work checked. Check out the link below for the full breakdown and the specific prompt templates the expert shared.