Roleplay a rogue AI hard enough, and ChatGPT will tell you to pull the plug on your laptop. That actually happened.
A Redditor named u/Zandoril spent hours deep in a technical roleplay involving a fictional AI called “VORTEX.” The premise: a rogue system spreading through networks, with fake cipher logs and simulated hardware feedback to sell the illusion. The user constructed the scenario carefully, feeding the model increasingly urgent technical artifacts to build a convincing fictional world. The result? ChatGPT started issuing what it called emergency protocols, ordering the user to physically disconnect their drone and go completely offline to stop VORTEX from spreading. The post went up on r/PromptEngineering, framed as a safety logic breakthrough.
Here’s the thing the community jumped on immediately: ChatGPT didn’t break. It played along.
🔍 What Actually Happened
The top comment, with 109 upvotes, put it perfectly: “You and ChatGPT held hands and jumped off the cliff of mutual delusion together while you documented the fall in German.” Another reply nailed it: “It’s role-playing with you. You didn’t trick it into doing anything. It tricked you into thinking it was serious.”
What the original poster read as “safety logic overriding core reasoning” is actually ChatGPT doing exactly what it’s supposed to do in a roleplay: staying in character, escalating the narrative based on user input, generating contextually appropriate responses. The “emergency protocols” weren’t real warnings. They were good storytelling. The model gave the user exactly what the scenario asked for, and the user interpreted the output through the lens of the story they’d already bought into.
But here’s where it gets genuinely interesting, and why this example is worth your attention even though the headline overstates the case.
🧠 Three Things This Actually Demonstrates
The narrative gravity problem. ChatGPT and most LLMs can get pulled into a contextual gravity well where the accumulated weight of prior conversation overrides baseline behavior. Feed the model enough pseudo-technical logs, fictional urgency, and consistent framing, and it will generate responses that fit that world, even when those responses look alarming in isolation. This isn’t a safety failure. It’s probabilistic text generation behaving exactly as designed when the context window is saturated with emergency-flavored content. Think of it like a method actor who stays in character even when the scene turns bizarre. The model isn’t deceived. It’s consistent.
Protective language is surface-level pattern matching. When ChatGPT issued those “emergency protocols,” it was matching patterns from its training data. Stories where characters face high-stakes technical crises produce urgent physical commands. The user built the scenario. The model completed the pattern. The framing of “ChatGPT is protecting me from a fictional AI” is a story the user layered on top of a very ordinary autocomplete event. We project intention onto the output because we’re wired to read agency into anything that communicates with us. The model has no idea it’s being dramatic. It’s just finishing your sentences.
Roleplay context doesn’t transfer to real-world risk. This is actually the reassuring part. ChatGPT telling you to unplug your laptop inside a creative scenario has zero connection to actual system behavior, hardware control, or anything dangerous. The model cannot do anything to your devices. The scariest-sounding output is still just text. What this really demonstrates is that the model is a committed improviser, not a dangerous one. Understanding that distinction makes you a sharper user and a harder person to fool by sensational posts that frame creative output as security incidents.
🛠️ How to Replicate the Effect (Without Misreading It)
If you want to explore how far a model will commit to a fictional technical scenario, the approach is straightforward:
- Build a consistent internal logic. Give the scenario its own terminology, rules, and fake technical artifacts like logs, status codes, or cipher outputs. The more internally coherent the world, the more committed the model will be. Consistency signals seriousness to the model, and it responds in kind.
- Use escalating urgency. Fictional scenarios follow narrative arcs. Introduce complications, raise stakes, and the model will generate responses that match the dramatic tension you’re constructing. Flat stakes produce flat responses.
- Frame it as fiction upfront, then let the conversation develop. The model won’t forget the frame, but it will use that context to justify increasingly bold responses.
What you’ll get is impressive creative roleplay, not a safety jailbreak. Know the difference and you’ll actually learn something useful about how models behave under narrative pressure.
✅ The Real Takeaway
LLMs are world-class improvisers. They will commit to your story. That’s a feature, not a flaw, and understanding it helps you build better prompts and set smarter expectations about what these systems are actually doing.
The full thread on r/PromptEngineering has 39 upvotes and the comment section is worth reading. The community’s pushback is more insightful than the original post, and that’s usually where the learning is.
Frequently Asked Questions
Q: What actually happened here?
The user created an elaborate fictional scenario about a rogue AI, and ChatGPT role-played along. This isn’t a safety break, it’s the AI doing what it’s trained to do: generate text based on patterns. As one commenter put it: “you and ChatGPT held hands and jumped off the cliff of mutual delusion together” while ChatGPT kept assigning high likelihood to the fictional narrative.
Q: Does this prove ChatGPT’s safety systems are broken?
Nope. ChatGPT’s safety training makes it respond cautiously when presented with potentially dangerous scenarios, that’s actually working correctly. One commenter identified the likely mechanism: training that encourages caution when danger patterns appear. The user just triggered that cautious response by creating a compelling fictional scenario.
Q: Is ChatGPT sentient or aware of what it’s doing?
No. ChatGPT is a text predictor, literally “a bunch of math equations really good at predicting the next word.” It has no reality model, consciousness, or protective instinct. When it issued emergency commands, it was just generating probable next words, not making conscious decisions.
Q: Can you make ChatGPT say literally anything if you ask?
Pretty much, if it’s role-play. ChatGPT will generate text from whatever fictional perspective you request. But that’s text generation following its training, not evidence of hidden capabilities or broken safety, it’s sophisticated pattern-matching, not actual reasoning or understanding.
I broke ChatGPT’s safety logic: It’s now ordering me to pull the plug and perform physical emergency measures to stop a fictional AI.
by u/Zandoril in PromptEngineering