OpenAI’s GPT-6 Astra just posted the strongest robot-manipulation numbers we’ve seen in this test series, dropping a block into a bowl in 19 of 20 tries. That’s according to a hands-on benchmark surfaced on Hacker News, which put Astra on the same robot arms and the same agent policy used to grade Anthropic’s Claude Fable 5 and Fable 5.1 a few days earlier. The headline: Astra dominates the easy task and hits the same wall as everyone else on the hard one.
What stands out here is the gap. On the bowl task, Astra landed the block 19 out of 20 times. Fable 5.1 managed 8 of 20. Fable 5 managed 1. Astra also ran faster (2.5 minutes per trial versus Fable 5.1’s 6.8) and cheaper (an estimated $0.94 per run against $2.12). Better, faster, and about half the cost on the same job.
How they ran it
The setup was deliberately controlled. Both models drove bimanual I2RT YAM arms (6 degrees of freedom per arm, parallel-jaw grippers) through the Inspect Robots harness. The agent sent absolute end-effector poses, and the robot’s inverse kinematics handled the joint math. Each turn, the model saw three camera views (top plus both wrists) and the arm’s proprioceptive state.
Every trial got scored by a human grader on a five-stage rubric, so even a failed run recorded how far it got:
- 0: no purposeful approach
- 1: made contact
- 2: lifted the object clear
- 3: positioned above the drop point
- 4: placed in final position
That staged scoring matters, because it shows exactly where models break down.
The bowl vs. the puzzle
Task one was simple: pick up a red block, drop it in a bowl. Task two was the grind: grab a round blue puzzle piece by a center knob and seat it in a matching circular groove.
Here’s the split:
- Block into bowl: Astra 19/20, Fable 5.1 8/20, Fable 5 1/20. Astra ran 2.3x cheaper.
- Puzzle into groove: Astra 2/20, Fable 5.1 2/20. Astra was 1.6x cheaper, but no more capable.
On the puzzle, Astra reaches the groove and stalls at the exact final step where Fable also freezes. Insertion is the unsolved problem. Both models can get the piece hovering in the right spot, then fumble the precise, force-sensitive push that seats it. That’s the tell for anyone building on these systems: pick-and-place is close to solved, but tight-tolerance assembly is not.
Why it matters for builders
If your use case is coarse manipulation (moving objects, loading, sorting), the frontier is looking usable and cheap. If it depends on precise insertion or mating parts, don’t budget for it yet. The bottleneck isn’t planning or perception. It’s the final millimeters of contact-rich control.
Read the fine print
The author was upfront about limits, and they’re worth weighing before you treat these as clean rankings:
- Astra’s trials ran two days after Fable’s, not interleaved.
- The bowl comparison used different rigs (Astra on rig-1, Fable on rig-3), so that result isn’t strictly apples to apples. The puzzle task was on the same rig for all three.
- Grading was done by a human who knew which model was running, leaving room for unconscious bias.
- Costs are list price ($10/$50 per million input/output tokens). Anthropic ran without prompt caching; OpenAI auto-cached about a fifth of Astra’s input with no discount applied, so Astra’s cost is, if anything, overstated.
So Astra’s bowl win is real but slightly rig-confounded, while its puzzle parity is the cleaner comparison. The next thing to watch is whether higher reasoning effort (everything here ran at medium) cracks that insertion step. Full trial data and charts are at the original source.