OpenAI's GPT-6 Astra fully completed 7 out of 100 robotics tasks in a new benchmark called StationeryBench, while Ai2's MolmoAct2 completed none. The median progress score was 46 out of 100 for Astra against 12 for MolmoAct2, according to results published on GitHub. The gap matters because spatial reasoning — the ability to judge where objects are and how to move them — is the part of AI that has lagged furthest behind language, and it is the part that decides whether a model can control a machine in a real room.
What the benchmark measured
StationeryBench was built around five desk-object tasks: uncapping a marker, pouring out paper clips, and passing a ruler between two robot arms, among others. Both models controlled the same dual-arm YAM robots across 200 trials, so the hardware and the task set were identical and the comparison rests on the models rather than on the setup. The scoring is deliberately graded: a model earns partial credit for getting closer to the goal, which is why the median progress figure of 46 for Astra and 12 for MolmoAct2 is reported alongside the count of fully finished tasks. All results, videos, and code are available on GitHub, so the trials can be inspected rather than taken on trust.
The distinction between progress and completion is the core of the result. Astra reached roughly half of the way toward the goal in a typical trial but finished only 7 of 100 runs, which means the model can often start a manipulation correctly and then fails to carry it through. MolmoAct2, at a median of 12, mostly did not get past the opening stage. For anyone evaluating a robot controller, that split is the practical signal: partial progress suggests the perception and planning layers are working, while the low completion count points to the execution layer — grip, force, timing — as the remaining bottleneck.
The result arrives as OpenAI has long-term plans to build its own consumer robots, which reframes the benchmark from an academic exercise into an early test of technology the company may eventually ship. Robotics has been the hardest place to apply large models because errors are physical and a failed grasp cannot be retried in a fraction of a second. A benchmark that separates planning from execution gives OpenAI, and its competitors, a way to measure progress on the manipulation problem before any consumer product exists.
What this means for business
For companies weighing where to apply AI, the immediate consequence is that spatial reasoning is becoming a measurable line item rather than a vague promise. A buyer of warehouse, laboratory, or light-assembly automation can now ask a vendor for completion rates and median progress on a defined task set instead of accepting a demo video. The difference between a small company and a large one is mostly in what they can do with that number: a small operation can pilot a single dual-arm cell and judge it on finished tasks, while a larger firm needs the same metric across many stations before it commits capital.
The limits of the result deserve equal weight. Seven completed tasks out of 100 is a low absolute figure, and the benchmark covers five desk-object tasks with one robot platform, so it says nothing yet about reliability over thousands of cycles, about unfamiliar objects, or about safety around people. The REMAP benchmark cited by Cornell and Google DeepMind researcher Yoav Artzi has not been published, so its accuracy figures cannot be checked independently. Artzi also notes that even Astra does not reach what humans do in other scenarios. Before treating spatial reasoning as solved, a buyer should ask which tasks were tested, how completion was scored, and whether the model was tuned on the same objects it was evaluated on.
Artzi calls Astra a step change in spatial reasoning and suspects OpenAI trained the model on large amounts of 3D data such as Blender scenes, which would line up with Astra's particular improvement on 3D tasks. The marker to watch is whether that suspicion is confirmed and whether the completion rate, not the progress score, moves on the next public benchmark run. If finished tasks climb while the task set stays fixed, spatial reasoning is turning into a component that can be specified in a contract; if only the progress score rises, the technology remains a promising planning layer still waiting for a reliable hand.
