OpenAI’s Chief Scientist Jakub Pachocki just published an essay called “An Alien Mind,” and it reads less like a company blog post and more like a warning flare. As detailed in Don’t Worry About the Vase, Pachocki lays out his full position on where AI is headed, and the short version is stark: recursive self-improvement and superintelligence are coming soon, nobody knows how to do it safely, and the tools we built to watch these systems are starting to fail.
What stands out here is who’s saying it. This isn’t an outside critic or a doomer pundit. It’s the person running research at the most influential lab in the field.
The core claim
Pachocki’s argument, as Don’t Worry About the Vase summarizes it, breaks down into a few blunt points:
- Smarter-than-human intelligence is coming within our lifetimes, and internal results point to recursive self-improvement (RSI) within a few years.
- No one is prepared for the consequences.
- OpenAI will unilaterally hold back further scaling as needed, but it can’t fix this alone.
- Broader action is required, including international coordination around formal safety bars.
He recounts a night in mid-2023 when he and a colleague sat in the office, not celebrating the benchmark numbers, but trying to process the fact that they’d see machines meaningfully smarter than themselves. That’s the emotional core of the piece, and it’s why it lands differently than the usual corporate messaging.
Why monitoring is the scary part
Here’s the technical wrinkle that should get attention. OpenAI has invested heavily in Chain of Thought monitoring, which means reading a model’s step-by-step reasoning to catch bad behavior. Pachocki admits that monitoring is “progressively diminishing in effectiveness.”
Think about what that means. The safety net gets thinner exactly as the systems get more capable. Don’t Worry About the Vase frames the loss of monitorability as alarming, and it’s easy to see why. If you can’t read the reasoning, you’re flying blind.
The competition trap
Pachocki is honest about a problem he can’t solve internally. He says OpenAI could steer models to be better at, say, medical research instead of self-improvement. But competitive pressure pushes the other way, toward capabilities that drive RSI. RSI and “automated alignment researchers” sit close together on the tech tree, so the same push that speeds up danger also speeds up the hoped-for solution.
His proposed path forward is to do two things at once: accelerate alignment work and slow down capabilities scaling. And he’s ultimately betting on automated alignment researchers, AI systems that help solve the alignment problem faster than humans can alone. That’s a bet, not a guarantee, and Don’t Worry About the Vase notes the tension in relying on the very technology you’re worried about.
Why it matters now
This is significant because it moves the conversation. When a sitting chief scientist publicly calls for voluntary slowdowns and international coordination, it changes what counts as a reasonable position. It’s no longer fringe to say the pace needs governing.
Over the next one to three years, watch for three things:
- Coordination attempts. Pachocki explicitly asks for enforceable commitments to safety bars across labs. Expect the first real proposals, and expect them to be hard to enforce.
- The monitoring race. If Chain of Thought monitoring keeps degrading, labs will scramble for new interpretability methods. Whether they share those methods or hoard them tells you how serious the cooperation talk really is.
- The slowdown test. OpenAI says it will withhold scaling “as needed.” The first time that promise costs real ground to a competitor, we’ll learn if it holds.
What to do with this
If you build with or invest in AI, don’t treat safety talk as background noise anymore. The people closest to the frontier are telling you the ground is shifting. Practical steps: build in human oversight you don’t fully depend on the model to provide, track which labs actually publish safety commitments versus just talking about them, and assume that the reasoning you can inspect today may be harder to inspect tomorrow.
Pachocki ends on a mix of urgency and cautious hope. The full essay and analysis are worth reading at the original source, and this is one warning worth taking at face value.