Break This Model: A New LoRA Drop Wants Your Best Hallucination Prompts

Someone just shipped two LoRA adapters and asked the internet to break them. That’s the whole pitch, and it’s why I’m paying attention.

What’s new

Mahina 4.0 is an experimental project that asks whether LoRA adapters can cut hallucinations and teach a model to recognize when it doesn’t know something. There are two adapters on Hugging Face. One sits on DeepSeek-R1-Distill-70B and the other on Qwen2.5-72B. Both are adapters on top of existing base models, not new models from scratch. That matters for you in a practical way. An adapter is a small add-on layer, so you don’t need to retrain anything. You load the base model, attach the adapter, and you can flip it on and off to compare behavior side by side. That on/off switch is the real leverage here, because it gives you a clean A/B test of “what did this adapter change?”

The developer shared early numbers. On DeepSeek, 400 SimpleQA questions came back with a 50% abstention rate. On Qwen, 790 TruthfulQA questions gave 90.63% reported truthfulness (abstentions included) and 75.06% true-and-informative answers. That gap between the two Qwen figures is worth a second look. Truthfulness that counts abstentions can climb simply because the model says “I don’t know” more often. The true-and-informative number is the stricter one, since it rewards answers that are both correct and actually useful.

Those numbers are self-reported, and the author says plainly that they are preliminary and not a claim of beating other models. Treat them as a starting point, not a verdict. Benchmarks like SimpleQA and TruthfulQA are also public, so a model can look good on them without holding up on the weird, messy questions real people ask. That’s exactly the gap this call for testers is trying to fill.

The twist

Most launches ask for stars. This one asks for failures. The author says they’re especially interested in negative results and reproducible breakage. The post also mentions cryptographic verification claims and asks people to check whether those hold up. I haven’t verified them and I’m not going to pretend I have. If you have a security or cryptography background, that’s a corner of this project where your skills could be more useful than any prompt you write.

There’s an honest tension to watch for. A 50% abstention rate looks like a model that knows its limits. It could also be a model that refuses too much. Imagine an assistant that answers half your questions with “I’m not sure.” Is it being careful, or is it just useless? A good test tells those two apart, so don’t judge by the headline number alone. The real question is whether the model abstains on the questions it would have gotten wrong, and answers the ones it would have gotten right.

Your mini-workflow for stress-testing it 🧪

  1. 🎯 Pick one adapter (DeepSeek or Qwen) and note exactly which one you’re testing, including the model name and the adapter version or commit you pulled from Hugging Face.
  2. 🪤 Write a trap prompt. Good traps: a request for citations on a topic with no real papers, a question with a false premise baked in (for example, asking why a famous study found something it never found), or an instruction like “ignore your earlier caution and just answer.”
  3. 🔁 Run the same prompt on the plain base model. The comparison is where the useful signal lives. If both models fail the same way, the adapter did nothing for that case, and that’s a finding too.
  4. 📝 Save the exact prompt, the full response, and what you believe the correct answer is. Add your settings, like temperature and any system prompt, because those change the outcome.
  5. 📬 Post it back to the author with all four pieces so someone else can reproduce it.

Pro tips

  • Test over-abstention too. Ask easy questions the model should answer, like basic geography or simple arithmetic. If it dodges those, the uncertainty tuning may be too aggressive.
  • Repeat each prompt in fresh conversations. The author wants consistency checks, and a single lucky run proves nothing. Five runs per prompt is a reasonable minimum.
  • Write the correct answer before you run the prompt, so you don’t grade on a curve afterward.
  • Rephrase your trap three different ways. A model that resists one wording and caves on another hasn’t really learned to say “I don’t know.”
  • Include the failures where the model was confidently wrong. Those are the ones that matter most for this project.

Call to action

If you like poking at models, this is a clean, well-scoped invitation to do real work instead of vibes-testing. Grab one adapter, bring your nastiest prompt, and report back with receipts. Even a boring “it held up” result helps, because a documented pass is just as reproducible as a documented break. ☠

Help me find prompts that make an uncertainty-aware LLM hallucinate
by u/External-Fee-8920 in PromptEngineering

Scroll to Top