Boris Cherny, head of Claude Code at Anthropic, set an AI agent loose on a hard task 15 days ago. It’s still running. That detail, pulled from his Y Combinator Startup School interview and dissected on Hacker News this week, tells you more about where agentic AI actually stands than any benchmark chart.
Here’s what Cherny did, according to Hacker News. He opened a Claude Tag session (Claude running inside Slack), spun up a macOS runner on GitHub, pointed it at an empty Swift codebase, and gave it one instruction: rewrite Anthropic’s Electron desktop app in native Swift, run both versions in a VM, screenshot them, compare pixel by pixel, and “don’t stop until you’re done.” When interviewer Diana Hu asked how long it had been running, Cherny said two weeks. And counting.
What stands out here isn’t the demo. It’s the thesis underneath it.
Verification is the new prompt engineering
Cherny’s real point had nothing to do with Swift. “The skill nowadays is less about prompt engineering and more about figuring out how do you give Claude a hard task that seems a little bit too hard,” he said. “Then how do you make it possible for Claude to verify its work along the way? The verification is probably the single most important thing that people do not get right.”
That’s the signal worth acting on. The industry spent two years obsessing over how to phrase requests. The frontier has moved. The bottleneck now is giving an agent a way to check its own output and know when it’s wrong. Screenshots, pixel diffs, test suites, running the actual app in a VM. The prompt was one sentence. The engineering was the feedback loop around it.
Why the demo also proves the limits
The Hacker News writer pushes back hard, and the criticism is fair. A pixel-perfect port can only copy what’s already there. If the current Claude app is badly designed, a flawless Swift clone is still a badly designed app, just with better plumbing. “The current Claude app is like a shitty recipe made with shitty ingredients,” the piece argues. “What Cherny asked Claude Code to do is follow the shitty recipe using better ingredients.”
The writer also disputes Cherny’s claim that the app is now “pretty awesome,” reporting a 30-second cold launch with a spinning beach ball. Two perspectives, and both can be true. Cherny is showcasing a verification method. The critic is pointing out that verification against the wrong target still gets you the wrong result. An agent that never stops until the pixels match will happily reproduce your mistakes forever.
What this means for you
If you’re building with coding agents right now, three takeaways:
- Design the check before the task. The hard part isn’t the instruction. It’s the mechanism that lets the agent grade itself. No feedback loop, no reliable long-running work.
- Pick the right target. Verification is only as good as what you compare against. “Match the old version” bakes in every flaw of the old version. Decide whether you want a copy or an improvement.
- Watch the failure mode. A 15-day run with no finish line is a warning, not a flex. “Don’t stop until you’re done” needs a real definition of done, or the agent burns compute chasing a target it can’t reach.
Cherny runs Claude Code, so his read on agent workflows carries weight. But the honest lesson from his own experiment is the one he didn’t frame that way: the technology can grind on a hard problem for two weeks straight, and still not ship, because the goal itself was pointed at the wrong thing.
That gap between “can run forever” and “knows when it’s right” is where the next round of AI tooling gets won. Full interview details and the counterargument are at the original source.