Somebody on r/PromptEngineering just sketched out a language model that skips most of its own thinking on purpose. The Redditor behind it goes by zemondza, and they’ve spent a while building two separate non-Transformer architectures on the side. One is NORD, a spiking, recurrent system built around token-time dynamics. The other is RHEA, short for Reactive Hypergraph Event Architecture, already prototyping at roughly 1 billion parameters. Now the original poster wants to fuse them.
Normal Transformers push every token through the same fixed stack of layers, no matter how easy or hard the input is. RHEA throws that assumption out. Instead of layers, it keeps a pool of latent events and lets a scheduler called ARES decide which computations are actually worth running next. A simple prompt barely lights anything up. A gnarly one triggers more reactions and deeper computation, and the model spends compute where it’s actually needed.
Here’s the part that got me: the reactions themselves aren’t dense matrix math. The author wants each reaction to run as its own tiny spiking microcircuit. A reaction is just the operation that merges two events into a new one. So instead of “event A plus event B goes through an MLP,” you get real spike timing happening inside the reaction itself. That’s adaptive compute pushed one level deeper than most public routing or MoE work bothers to go.
The rest of the design keeps a foot in familiar territory, which is the smart part. Token embeddings, the ARES scoring model, and the language head all stay dense and boring. Spiking dynamics only handle temporal state, persistent memory, and the reaction microcircuits themselves. Memory isn’t passive storage either. It builds up activity over hundreds of tokens and, once it crosses a threshold, fires its own recall event straight back into the system.
There’s a third layer to this that’s easy to miss: time itself is split into three speeds. NORD’s spikes run fast, RHEA’s reaction chains run medium, and persistent memory plus “crystallized” skills run slow. That last part is the interesting bit. If the system keeps firing the same chain of reactions over and over, that chain can get compressed into one faster macro-reaction. That’s basically the model learning its own shortcuts the more it runs.
The whole thing boils down to one line the author keeps repeating. Computation should follow information, instead of information always being forced through a fixed computation graph. For anyone who spends their day hand-crafting prompts to coax more reasoning out of a fixed-depth model, that’s a different way to frame the problem. It reads less like a new model and more like a new default for how models are allowed to think.
Here’s the pipeline the original poster is sketching, step by step:
- Tokens hit NORD first ⚡. Its spiking layer turns them into a stream of events instead of a single flat embedding.
- Those events land in RHEA’s event fabric, a pool of latent facts the model can freely combine.
- ARES scores every candidate reaction and estimates whether the compute is actually worth spending.
- Only reactions that clear the bar fire, and each one runs as its own small spiking microcircuit.
- Firing a reaction creates a new event 🎯. That event can trigger a memory write, another reaction, or the final output.
If you’re the type who likes to prototype this stuff yourself, two things from the thread are worth stealing.
Pro tip one: the credit-assignment problem is the real killer in any skip-a-computation design, since you can’t backprop cleanly through “this reaction didn’t run.” The fix is a counterfactual preview: near-threshold reactions get a cheap dry run so the scheduler can check whether skipping them was a mistake. That’s a neat trick, and it’s a lot cheaper than trying to make the whole routing decision differentiable!
Pro tip two: read the limitations section before you get excited. The author lists the risks without flinching. Irregular compute is unfriendly to GPUs. Sparse routing is hard to train. SNN dynamics can make an already unstable optimization problem worse, and dynamic event memory can quietly fill up with garbage over time. This is a design direction, not a finished model, and the original poster says so directly instead of overselling it.
That honesty is what makes the post worth reading over the usual “I built a new architecture” claims. The author isn’t asking for hype. They’re asking specific people to weigh in: researchers who work on SNNs, recurrent models, MoE routing, and adaptive computation. The two questions on the table are simple. What breaks first, and does anything in the literature already cover this ground?
Go find the thread on r/PromptEngineering and read the full write-up yourself. The best part isn’t the diagrams. It’s the open question at the bottom: should the SNN component stay limited to memory and timing, or get pushed into the reaction operators too? That’s the kind of argument worth watching play out in the comments 🧭.
Exploring a NORD × RHEA hybrid: a spiking/event-driven alternative to a fixed Transformer stack
by u/zemondza in PromptEngineering