World-model planning on an SO-101: the commanded joints win (0.90 vs 0.60)¶
We wanted to know if a world model could plan on our arm by ranking candidate motions. We asked it to pick the real motion among 8, against a check that reads only the commanded joints. The joints won, scoring up to 0.90 vs 0.60 for the best world model.
Evaluation and judges · 9 of 14
Type: method · Result: Ranking action chunks, the world model's top-1 is 0.60 against 0.68–0.90 for an action-only baseline. · n: 703 val anchors
Not proven: Closed-loop planning was not run.
Compute: External GPU: no for these gates (Mac). The models: W1 ~$6.54 (see the $3.30 post), W2 < $1, probes ~$0.06. · Reproduce: 2026-10-23-paper-vs-hardware-leworldmodel · First published: 2026-10-13 · Updated: 2026-10-13
G1b asked the question that decides whether a latent world model can plan on this arm: given the expert chunk and seven decoys, does imagining each one rank the expert first? Two world models, one bigger pretrained one, and a baseline with no model at all. The baseline won.
G1b: plannable range¶
Bar: top-1 ≥ 0.8 at some target distance.
- W1 (192-d, trained here): best 0.60 at d = 20–30. Fail.
- W2 (frozen DINOv2-S patches + learned dynamics): arm-pose R² 0.92 vs W1's 0.57, so the representation is far better; on G1b, 0.55–0.61. Fail. A decision with W2 takes 1.8 s on the Mac anyway (G2), so a third, bigger model W3 was not run.
- RoboJEPA (pretrained, V-JEPA 2.1 features): the encoder takes 526 ms for two views on the Orin NX and 1 025 ms on the Mac. Off the robot. Zero-shot on our latents, the 1B model is borderline on a partial sample and action conditioning does work on our arm (9–10 % below shuffled actions). Not enough to earn a role.
The diagnosis¶
Top-1 among 8 compounds pairwise errors: W2's pairwise accuracy is 0.92–0.93, and 0.92^7 ≈ 0.56, which is exactly its top-1. A 0.8 top-1 bar needs ~97 % per comparison. Regret replaced top-1 as the metric.
Then the control: score chunks with the commanded joint trajectory alone, no images, no model. Top-1 0.68–0.90, regret ≈ 0. On a position-controlled arm, the actions already say where the arm will be. G1b was measuring kinematics, and the joints are a better kinematics model than any latent trained on 182 episodes.
What it means¶
A world model only has something to add where the actions are blind: object state, contact, failures. Prediction quality (G0 passed) is not decision quality (G1b failed). The next gates test exactly that boundary, and the planning role is gone for good.
Compute. External GPU: no for these gates (Mac). The models: W1 ~$6.54 (see the $3.30 post), W2 < $1, probes ~$0.06.
Reproduce this: 2026-10-23-paper-vs-hardware-leworldmodel
Evaluation and judges, 9 of 14. ← Evaluation and judges 8: Training a JEPA world model on SO-101 data for $3.30 · Evaluation and judges 10: TurboVLA: a policy that ignored its instruction →