New research says giving people an AI assistant made them worse at answering questions and more sure of their wrong answers. According to Hacker News, researchers from three French and Italian universities found that access to AI advice crushed people’s willingness to say “I don’t know,” dropping it from 44% to just 3%. Accuracy fell too. Confidence went the other way.
This is the part that should stop you cold: people got worse and felt better about it.
What the Researchers Actually Did
The team, led by Valerio Capraro at the University of Milano-Bicocca with Chiara Marcoccia of École Normale Supérieure and Walter Quattrociocchi of Sapienza University of Rome, ran a clever setup. They picked questions AI models usually get wrong. Think visual trivia, like the color of a team’s uniform in Bend It Like Beckham.
Then they handed participants Step 3.5 Flash, a model that reliably flubbed those exact questions. That choice was deliberate. If people got worse after asking the AI, you couldn’t wave it away as “they wisely trusted a reliable tool.” The tool wasn’t reliable. People trusted it anyway.
The Numbers
Here’s what happened when AI advice entered the room:
- Willingness to say “I don’t know”: 44% down to 3%
- Accuracy: 27% down to 9%
- Confidence: 30% up to 76%
“People became much worse, the accuracy was only one third, but they were twice as confident,” Capraro said. Some participants who would have nailed the answer on their own asked the AI and got it wrong.
The researchers tried paying people for correct answers. Money helped a little, not much. Willingness to admit ignorance crept from 3% to 8%. Accuracy went from 9% to 16%. Both still sat far below the no-AI baselines of 44% and 27%. Cash couldn’t buy back people’s judgment.
Why This Matters
This isn’t the first warning shot. Wharton researchers earlier this year coined “cognitive surrender” for the same pattern: people accepting wrong AI answers 80% of the time while feeling more confident than folks working without AI.
What the new study adds is sharper. It’s not just that people trust bad AI answers. It’s that the mere availability of AI seems to switch off the habit of noticing what you don’t know. That habit matters. As Capraro put it, saying “I don’t know” is “the recognition of the limits of our own knowledge.”
He’s especially worried about kids growing up with these systems before they build critical thinking skills. And the design trend isn’t helping. Google’s AI search overhaul replaced links with confident AI summaries, and Common Sense Media this week called that an “unacceptable risk” for students. The pattern is consistent: AI products are built to answer, never to say “I don’t know.” The people using them are picking up the same reflex.
What You Can Actually Do with This
You don’t need to ditch AI. You need a habit or two around it.
- Form your own answer first, then check the AI. If you skip your own take, you can’t tell when the machine is leading you off a cliff.
- Treat confident output as a claim, not a verdict. These models rarely hedge, even when they’re wrong. The tone is not evidence.
- Ask the AI to show its reasoning or sources. If it can’t back the answer, that’s your “I don’t know” signal.
- Watch for the questions AI is bad at: niche visual details, fresh events, anything needing real-world grounding.
One limitation worth naming: the researchers rigged the test with questions the model fails at. Real life gives you a mix, and AI genuinely nails plenty of tasks. The point isn’t that AI makes everyone dumber across the board. It’s that a confident wrong answer can override your own correct instinct, and you won’t feel it happen.
The fix is boring and it works. Keep your own judgment in the loop. Get comfortable saying “I’m not sure.” That two-word phrase might be the most valuable skill left once the machines answer everything.
You can find the full study details at the original source.