LeWorldModel¶
They said a small world model, trained from camera images, learns a robot's motion well enough to plan with. On our arm we checked whether it ranks the motion actually taken first among 8 candidates. It did not hold: it was right 0.60 of the time, below a check that ignores the camera.
Papers tested · 1 of 22
Goal (the paper's): Train a small world model end to end from pixels, with no frozen encoder and no EMA target, kept from collapse by SIGReg alone, and plan with it by sampling actions in its latent space; shown in simulation (Push-T, Two-Room, Reacher, OGBench Cube).
Its hypotheses:
- A 15M-parameter JEPA world model trained end to end with SIGReg learns dynamics stable enough to plan with. (§3.1, Eq. (3); Fig. 6) reproduces
- At a matched planning-compute budget it plans more successfully than DINO-WM and about 48× faster; without that match it leads on Push-T and Reacher only. (Fig. 3 (Push-T at fixed FLOPs 90 % vs 13 %; 0.98 s vs 47 s per plan)) fail
- A linear readout of its latent recovers object state. (Table 1 (Push-T probes)) not tested here
Verdict: fail · Claim tested: A small JEPA world model learns the arm's dynamics well enough to plan with it. · Bar: Dynamics: beat copy-last, every corruption ≥ 1.3 at K = 30. Planning: rank the executed chunk first among 8 in ≥ 0.8 of anchors. · n: 25 val episodes, 703 anchors
Theirs vs ours: Push-T planning 90 % at matched FLOPs vs dynamics 0.31 vs 0.89 copy-last (reproduces); planning top-1 0.60 vs 0.68–0.90 for an action-only baseline (fail) · Not proven: Closed-loop planning on the arm was never run; one training seed.
Paper: arXiv 2603.19312 · Code: lucas-maes/le-wm
Claims:
- reproduces JEPA world model below copy-last baseline on open-loop latent error at K = 30 — theirs Push-T planning 90 % at matched FLOPs · ours 0.31 vs 0.89 copy-last (not stated: the premise of §3–4 (planning needs a usable dynamics model))
- fail latent world-model ranking above action-only baseline on G1b top-1 — theirs Push-T planning 90 % · ours 0.60 vs 0.68–0.90 action-only (paper §4 (planning by ranking imagined outcomes))
- reproduces action-only baseline above latent world-model ranking on G1b top-1 · ours 0.68–0.90 vs 0.60 (our measured comparison)
Compute: External GPU: yes (vast.ai): ~$6.60 incl. $2.96 of failed attempts. · Reproduce: 2026-10-23-paper-vs-hardware-leworldmodel · First published: 2026-10-23 · Updated: 2026-10-23 · Findings: I need to rank candidate action chunks →
One paper, one arm. This series takes each paper I actually tested on a SO-101 + Jetson Orin NX, states what the paper claims and how its authors measured it, puts my number next to theirs, and says what the gap means. Numbers are from journal/EXPERIMENTS.md; gate IDs are the register's.
LeWorldModel (Maes, Le Lidec, Scieur, LeCun, Balestriero; arXiv 2603.19312) is the recipe behind W1, the world model that runs on my arm. Four gates tested it: dynamics (G0), planning (G1b), progress (G6a), outcome readout (G10).
What the paper claims, and how it measures it¶
A 15M-parameter JEPA trained end-to-end from pixels: a ViT-tiny encoder (patch 14, 12 layers, width 192, ~5M) whose CLS token becomes one 192-d latent per frame; a 6-layer predictor with AdaLN zero-init action conditioning (~10M); loss ‖ẑ_{t+1} − z_{t+1}‖² + 0.1 · SIGReg, where SIGReg pushes the batch of latents toward an isotropic Gaussian by testing 1024 random 1-d projections for normality. No EMA target, no stop-gradient; SIGReg alone prevents collapse. Planning is CEM with a terminal cost ‖ẑ_H − z_g‖² inside MPC.
Evaluation is simulation only: Push-T, Two-Room, Reacher, OGBench Cube. The headline is success at matched training FLOPs (Fig. 6): Push-T 90 % vs DINO-WM's 13 % at the same compute, planning 48× faster (0.98 s vs 47 s). Table 1 is a linear-probe table: a linear readout of the latent predicts Push-T block location with r = 0.986 and block angle with r = 0.902. There is no real robot, no non-learned planner baseline, and the authors list "short planning horizons" as the limitation.
What I ran¶
W1 is LeWM's recipe on 182 teleop episodes from two webcams (multi-view variant, ContactGuard-style mean-pool), 100 epochs, $3.30 on vast.ai.
G0, dynamics. Bar: beat copy-last; every action-corruption ratio ≥ 1.3 at K = 30; no collapse. Result: open-loop error at K = 30 0.31 vs copy-last 0.89; timing-shift ratios 1.99 / 1.58; effective rank 24–30 of 192 (the 5-epoch run was 5.8). Pass. The model learns real action-conditioned dynamics.
G1b, planning as ranking. Bar: rank the expert chunk first among 8 candidates at top-1 ≥ 0.8 at some horizon. Result: best 0.60 at d = 20–30. Fail. The diagnosis (G1 diagnosis row) found why: an action-only baseline with no model at all, ranking by commanded-joint distance, scores 0.68–0.90. On a position-controlled arm the commanded joints already determine the outcome of a 7-step chunk; the latent adds noise.
G6a, progress from latent distance. Bar: Spearman ≤ −0.8 between latent distance to goal and time. Result: −0.32 to −0.50. Fail. Latent distance is not progress here (the TD-JEPA paper, 2607.25337, finds the same gap in simulation: Spearman 0.65 for LeWM's Euclidean distance vs step count).
G10, outcome readout at rest. Bar: AUROC ≥ 0.90 on success/failure from the final latent. Result: 1.000. Pass. Table 1's probe result transfers: the latent contains the task outcome.
Their number, my number¶
| Paper (sim) | This arm | |
|---|---|---|
| Dynamics learned | yes (VoE test, qualitative) | yes: 0.31 vs 0.89 copy-last |
| Linear probe of state | r 0.986 (block location) | arm-pose R² 0.57 (W1), outcome AUROC 1.00 |
| Planning success | Push-T 90 % at matched FLOPs | top-1 0.60 vs no-model 0.68–0.90 |
| Baseline without a world model | not run | run; it wins |
What the gap means¶
Not that the paper is wrong. The paper's planner is compared to other learned planners at equal compute; it is never compared to "do nothing clever". On a cheap arm the clever-nothing baseline is strong because the commanded trajectory is the plan, and the latent planner has to beat it, not DINO-WM. The honest role for the model here is the one G10 passed: a verifier at rest. The planning-limits paper (2609.39235) later showed with a real simulator as a perfect predictor that terminal-latent-distance scoring against a distant goal falls from 92 % to 41 %; G1b scored against a goal many chunks away, so it sat in exactly that regime. A subgoal inside the rollout is the fix the paper proves; it has not been tried here yet.
Verdict: dynamics and readout reproduce; planning does not survive the baseline the paper did not run.
What would change it: G7 (object-state ranking where the joints are blind), with a near subgoal. That gate is frozen and waiting for a cube-rollout session.
Compute. External GPU: yes (vast.ai): ~$6.60 incl. $2.96 of failed attempts.
Reproduce this: 2026-10-23-paper-vs-hardware-leworldmodel
Papers tested, 1 of 22. Papers tested 2: DINO-WM →