An engineer handed Claude Fable 5 and GPT-5.6 Sol the same unpublished NP-hard optimization problem and let each one grind for 30 minutes. Fable 5 won, and not by a little. According to Hacker News, where the writeup landed with 165 points, Fable produced the best solution overall with a consistency the author says he hasn’t seen from any model on this benchmark.
The second finding is the one that should change how you work: the /goal mode both companies now ship is not a generic “try harder” switch.
The benchmark
The problem is KIRO, a fiber-network design task the author worked on as an engineering student in 2018. Given directed distance matrices for Grenoble, Nice, and Paris, a solver has to connect distribution points and terminals using redundant loops and short branches, respecting structural constraints. Objective: minimize total cable length.
Why this matters as a test bed:
- It’s unpublished, so no model memorized the answer.
- The author spent a week writing C++ to solve it years ago, giving a real human baseline.
- The search space is absurd. Paris alone, restricted to one narrow family of valid solutions, yields roughly 10^1223 configurations.
That’s the kind of space where you can’t brute-force anything. You have to pick a good strategy fast and commit.
The numbers
Six models were swept, then the flagship pair got three matched runs each. Lower is better.
| Model | Plain mean | /goal mean | Mean effect |
|---|---|---|---|
| Fable 5 | 32,386 | 33,145 | 759 worse |
| GPT-5.6 Sol | 34,261 | 35,129 | 868 worse |
Fable’s plain mean beat Sol’s by 1,875 points. Its goal mean beat Sol’s by 1,984. Best clean score of the whole experiment: 31,934, from Fable with /goal.
The variance gap is the part practitioners should stare at. Fable plain stayed inside a 319-point range across three runs. Sol plain spanned 1,958 points. When you’re running an agent unattended on a hard problem, a tight range means you can trust one run. A wide range means you need to run it five times and pick.
Why /goal is a coin flip
Goal won four of six trials. Win rate says useful feature. But both means got worse, because the losses were enormous when they came: one Sol run degraded by 5,790 points, one Fable run by 2,732.
The mechanism explains it. On a normal coding task, progress is legible. Another turn fixes a test. Optimization doesn’t work that way. Once an agent picks a solver, extra time amplifies whatever decision it already made, good or bad. Goal sometimes finds a better basin. Sometimes it gives a bad idea more room to mature.
Same command, two different systems
The article’s deep dive is worth the read on its own. Claude Code and Codex both expose /goal, but the implementations diverge:
- Claude Code runs it as a session-scoped Stop hook. A small evaluator model, Haiku by default, reads the goal and the transcript after each turn and returns yes or no. It cannot use tools or inspect files.
- Codex persists the goal as thread state in SQLite. The working model gets create_goal, get_goal, and update_goal tools, and when the thread goes idle, Codex injects a continuation turn.
The tradeoff is clean: Claude’s evaluator is independent but blind to the filesystem. Codex sees the files but effectively grades its own work.
What to actually do with this
Three takeaways worth carrying into your own agent setups:
- Match the mode to the task shape. Use /goal where progress is verifiable per turn (tests, migrations, builds). Skip it where quality is a continuous score and more time can entrench a bad choice.
- Track variance, not just best score. A model that never blows up beats one that occasionally wins bigger, especially in production pipelines nobody babysits.
- Know which /goal you’re using. Transcript-only evaluation and self-graded persistent goals fail in different directions.
The author acknowledges the setup was intentionally narrow: 30-minute budgets, one problem family, three runs per flagship configuration. Small sample, one domain. All code, prompts, result tables, and exclusions are published for anyone who wants to rerun it.
What stands out is how much of the performance gap came from consistency rather than peak capability. That’s the metric agent builders will be watching next.