The AI industry’s hottest narrative right now is recursive self-improvement: the idea that AI models will soon improve themselves, creating an exponential feedback loop that leads to an intelligence explosion. But a compelling counter-argument is gaining traction. According to Interconnects, what we’ll actually see is “lossy self-improvement”: models becoming central to their own development process, but friction breaking down every core assumption of the exponential story.
Interconnects lays out the three conditions recursive self-improvement (RSI) needs to work: the loop must be closed (models improving themselves to create better models), self-amplifying (each generation yielding bigger gains), and friction-free (no diminishing returns). The argument? None of these conditions will fully hold.
The Complexity Brake
The piece invokes Microsoft co-founder Paul Allen’s “complexity brake” concept: the more progress science makes toward understanding intelligence, the harder additional progress becomes. Patent data backs this up. “The number of patents per thousand peaked in the period from 1850 to 1900, and has been declining since,” as researcher Joseph Tainter documented. Growth in complexity eventually becomes self-limiting.
This isn’t abstract philosophy. It maps directly onto what’s happening in AI labs right now.
Where the Losses Show Up
Interconnects identifies a critical friction point: automatable AI research is too narrow. Yes, models can already optimize localized tasks like lowering test loss. Andrej Karpathy’s recent “autoresearch” project showed AI agents running experiments directly on GPUs. It works for narrow targets: one loss metric, one reward signal.
The problem? There’s a persistent gap between a model that looks better on paper and one that users actually find more productive. Scaling laws tell us loss will keep going down, but they don’t tell us whether that translates into economic value. Pretraining improvements and user-facing quality aren’t the same thing, and the relationship between them remains poorly understood.
This matters because the RSI narrative assumes improvements compound cleanly. In practice, building a leading language model requires deep intuitions, organizational knowledge, and integration across dozens of complex systems. More compute and more agents thrown at a problem means more redundancy, more coordination overhead, and more loss.
What This Means for the Industry
Interconnects still expects “momentous, socially destabilizing changes” from sustained AI progress over the next few years. The disagreement isn’t about whether progress continues; it’s about the shape of the curve. Linear rather than exponential. Significant but not singular.
A few practical takeaways:
- For AI labs: The biggest bottleneck isn’t raw model capability. It’s the translation layer between benchmark improvements and real-world usefulness. Investing in evaluation and user-facing quality will matter more than chasing loss curves.
- For businesses: AI tools are already “extremely good” for plenty of valuable knowledge-work tasks. Waiting for the singularity before adopting them means leaving money on the table. The gains are here now, even if they won’t accelerate exponentially.
- For researchers: Automated research tools will handle narrow optimization tasks increasingly well. The high-value human contribution shifts toward system-level thinking: understanding which problems to solve and how improvements compose across a full training pipeline.
The Oligopoly Factor
One detail worth flagging: Interconnects notes that “two, maybe three, labs are consolidating as an oligopoly with access to the best AI models.” This concentration creates its own friction. Fewer organizations pushing the frontier means fewer independent approaches, less diversity in research direction, and potentially slower aggregate discovery; even as each lab individually gets more capable.
Looking Ahead
The “lossy self-improvement” framing is a useful corrective to the breathless singularity talk dominating AI discourse. Progress will be real and substantial. AI models will increasingly participate in their own development cycle. But the expectation of a clean exponential takeoff ignores the messy realities of how complex systems actually evolve.
What stands out here is the maturity of the argument. It’s not AI skepticism; it’s engineering realism. The full analysis, including deeper discussion of post-training friction and organizational complexity, is worth reading over at Interconnects.