Here’s the risk hiding in plain sight: the people writing federal law are quietly outsourcing pieces of it to chatbots, and there’s almost nobody checking their work. According to Futurism AI, tools like OpenAI’s ChatGPT, Anthropic’s Claude, and xAI’s Grok have become everyday utilities inside Congress. The catch is that these tools hallucinate, and the safeguards that should catch those errors mostly don’t exist on Capitol Hill.
The evidence isn’t hypothetical. Futurism AI reports that the office of Representative Anna Paulina Luna (R-FL) accidentally pasted raw Claude output into the public record of the National Defense Authorization Act. The document literally read “11:25 AM???? Claude responded:” in the middle of a defense bill. Luna didn’t flinch.
“A lot of staff will use ChatGPT, Claude, Grok. I personally like Grok better, but my staffer prefers Claude,” she told the Washington Post, per Futurism AI. “There’s nothing illegal about him doing a summary.”
She’s right that it’s not illegal. That’s exactly the problem.
Why this matters now
What stands out here is the gap between adoption and oversight. Staffers in both the House and Senate are using these tools freely, and Futurism AI notes there’s practically no formal rulebook governing how. No staffer has been disciplined for AI misuse, because the rules are a patchwork at best. When the tool is a research assistant, that’s manageable. When aides use it unsupervised to draft actual legislative language, a hallucinated clause or a garbled citation can slip into a bill and nobody notices until it’s already law.
This lands in the middle of a bigger push. Futurism AI ties the trend to the Trump administration’s drive to get every corner of government adopting AI at speed. Adoption is racing ahead. Guardrails aren’t. And the people best positioned to regulate AI are the same ones leaning on it without a safety net, which suggests federal AI rules will stay stuck for a while.
The pattern across every profession
Congress isn’t special here, and that’s the unsettling part. Futurism AI points to lawyers caught citing fake cases, consulting firms shipping nonsensical reports, and teachers using AI to grade likely AI-written essays. The tech has gone everywhere at once. The difference is stakes. A bad consulting deck costs a client money. A bad clause in the National Defense Authorization Act is a different order of consequence.
Not everyone is careless about it. Futurism AI highlights Sydney Broome, an aide to Representative Cleo Fields (D-LA), who set a personal rule: never use AI to write documents from scratch. She picked that habit up from a policy she ran into back in college. The irony writes itself. Congress has no such rule, but a college classroom did.
What practitioners and businesses should take from this
If lawmakers can let AI slop into a defense bill with zero process, your organization can too. The fix isn’t banning the tools. It’s building the boundary Congress skipped.
- Draft assist, not draft author. Use AI to summarize, research, and reshape, not to write final language from scratch. Broome’s rule is a good default.
- Require a human verification step for anything that ships externally or carries legal weight. Someone owns the output, not the model.
- Log where AI touched the work. You can’t audit what you can’t see. A simple disclosure norm beats a silent free-for-all.
- Write the policy before the incident. The offices getting embarrassed are the ones that had no rule until something leaked into the public record.
The deeper lesson is about sequence. Adoption is easy and instant. Oversight is slow and boring, so it gets skipped, until a chatbot timestamp shows up in a federal bill. If the institution writing the nation’s AI laws can’t govern its own AI use, don’t assume anyone’s coming to govern yours. Build the guardrail yourself.
More detail on the congressional AI free-for-all is available at the original Futurism AI report.