Rich Sutton, the Turing Award winner often called the father of reinforcement learning, has a blunt message for anyone building AI that predicts the future: the popular shortcut doesn’t work. In a short essay resurfacing on Hacker News (originally written for X in July 2024), Sutton names the mistake directly. He calls it “the one-step trap,” and he argues it’s baked into a surprising amount of modern AI thinking.
What the one-step trap actually is
The idea sounds reasonable at first. You train an AI to predict just the next step, one small tick forward in time. Then, when you need to know what happens far into the future, you run that one-step model over and over, chaining each prediction into the next. Predict one frame, feed it back in, predict the next, and so on.
Sutton says this is where teams go wrong. The trap is believing that most of an agent’s useful predictions can be one-step ones, with long-term forecasts just stitched together from them on demand.
He admits the appeal, because it holds a grain of truth. If your one-step model is perfectly accurate, then yes, iterating it gives you perfect long-term predictions too. The problem is that no real model is perfect.
Why it breaks down
Sutton lays out two failures that kill the approach in practice:
- Errors compound. Small mistakes in each one-step prediction stack on top of each other. Roll the model forward far enough and those tiny errors snowball into long-term predictions that are simply wrong.
- The math explodes. In a world with any randomness, the future isn’t one clean path. It’s a branching tree of possibilities, each needing its own weight and probability. Computing a long-term prediction this way grows exponentially with how far out you look. Sutton calls it “generally infeasible.”
His verdict is harsh: “one-step models of the world are hopeless, yet extremely appealing.” And they’re everywhere. He points to POMDPs, Bayesian analyses, control theory, and compression-based theories of AI as places the trap keeps showing up.
What stands out here
What makes this worth your attention is who’s saying it and what it targets. A lot of current excitement around “world models” rests on exactly the assumption Sutton is attacking, the idea that a good next-step simulator can be rolled out to imagine long horizons. He’s saying that intuition, borrowed from physics and realistic simulators, quietly falls apart the moment your predictions carry any error at all. Which they always do.
Sutton’s proposed fix is to stop predicting at a single fixed time scale. Instead, he wants AI to learn temporally abstract models, predictions that jump across longer, variable stretches of time rather than one tick at a time. He points to two tools from his own research lineage:
- Options, a framework for temporal abstraction in reinforcement learning (Sutton, Precup, Singh, 1999).
- General Value Functions (GVFs), from the Horde architecture (Sutton et al., 2011) and later work on reward-respecting subtasks (Sutton et al., 2023).
What you can do with this
If you’re building or evaluating agents that plan ahead, treat this as a design warning:
- Be skeptical of any system that predicts far futures by looping a next-step model. Ask how error accumulates over the rollout.
- Watch for compounding drift in long-horizon forecasts and video or dynamics models. If accuracy craters as the horizon grows, you may be living in the trap.
- Look at abstraction. Models that predict over meaningful chunks of time, rather than uniform tiny steps, tend to hold up better over long horizons.
A fair caveat: this is a short opinion piece, not a benchmark study. Sutton presents it as his view, backed by his own foundational papers, not fresh experimental results. The compounding-error problem is well documented in the field, but his preferred solution, options and GVFs, is still an active research direction rather than a settled win.
Still, when one of the people who built reinforcement learning says a whole category of world models is “hopeless,” it’s worth reading closely. You can find Sutton’s full essay and the cited papers at the original source.