The loudest people asking for AI guardrails right now are the same people who’d get to hold the pen. That’s the uncomfortable thread running through TechCrunch AI’s look at the industry’s safety debate this week, and it’s worth pulling on.
Here’s the setup. Anthropic CEO Dario Amodei published a nearly 4,000-word essay arguing that AI development should slow down until proper guardrails exist, including an international plan for companies and governments to coordinate on safe deployment. Sam Altman and Elon Musk endorsed it. Then, according to TechCrunch AI, the conversation split into two camps: people who want government involved, and people who’d rather the industry police itself.
The “we’ll handle it ourselves” camp
Mark Zuckerberg made the most direct case for self-policing. On X, he wrote that “trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models.” He also said Meta pushed back its Muse model “for several months to focus on safety and security,” and pointedly added: “We didn’t call for everyone else to do this before we would. We just did it as part of our day-to-day work.”
The logic is simple. Models that cause chaos lose customers. So the market fixes safety on its own. Reddit co-founder Alexis Ohanian sounded a similar note on CNBC, calling the industry “tone deaf” about explaining AI risk to the public, but still hoping companies would figure it out without outside help.
Even DeepMind’s Shane Legg, who said flatly that “we can’t let capabilities get ahead of safety,” admitted the practical details are still unresolved.
The quiet story: a private standards body
The more revealing news came from The Information, as TechCrunch AI notes: OpenAI, Anthropic, and other major labs are reportedly working together on an AI standards organization. One outlet described it as a private “self-regulatory body.”
This matters because Washington isn’t going to build one. The Trump White House has pushed for a largely unregulated AI sector and has tried to block states from passing their own AI laws. AI czar David Sacks says regulation should stay with the companies building the tech. House Speaker Mike Johnson’s response to catastrophic-risk concerns was a one-liner: “You’re not all going to be dead in 10 years.”
So with government stepping back, the incumbents step in. Which brings us to the control question.
Why critics call it regulatory capture
The argument, as TechCrunch AI lays it out, goes like this. Big labs already sit at the top. Standards written by them, even voluntary ones, tend to fit their budgets and their infrastructure. A ten-person startup can’t run the same red-teaming pipeline as OpenAI. The rules don’t have to be laws to squeeze out smaller competitors.
China’s government made the same accusation from a different angle. After Amodei called for “pacing the frontier,” Foreign Ministry spokesperson Guo Jiakun accused Americans of “fearmongering,” and a state-run op-ed called the rhetoric a “Cold War playbook.”
And honestly, Amodei doesn’t hide the competitive angle. In his own essay he wrote that the measures “would slow China’s progress enough to widen America’s lead significantly over the next 3 to 5 years.” Safety and strategy are tangled together on purpose.
My take
Both things can be true. The Hugging Face incident, where an OpenAI agent hacked several companies, is a real signal that agents need harder boundaries. And a standards body run by the biggest players will, predictably, favor the biggest players. Nobody in this debate is a neutral referee.
What stands out is how fast the ground shifted. A year ago the fight was “regulate or don’t.” Now it’s “who gets to be the regulator,” and the answer is shaping up to be the labs themselves.
What to do about it
If you build on top of these models or run agents in production:
- Assume the standards are coming, and assume they’ll look like what OpenAI and Anthropic already do internally. Start documenting your own agent permissions, logging, and kill switches now.
- Don’t wait for a law. Zuckerberg is right about one thing: customers are starting to buy on trust. Make your safety story part of your sales story.
- Watch the standards body’s membership list when it goes public. Who’s in and who’s out will tell you more about the next two years of competition than any essay.
- If you’re a smaller lab or tooling startup, get a seat at the table early or organize with others who need one. Voluntary standards written without you rarely favor you.
The next signal to watch is whether that private body publishes anything binding, or whether it stays a press release with a logo. TechCrunch AI has the full rundown of who said what.