LeJEPA identifiability¶
They said a world model trained on goal-directed runs recovers the true robot state only partly and unevenly. On our arm we compared its internal numbers with the joint angles the arm logs. It held: it explained 0.565 of the joints' variance on average, 0.91 at the shoulder, 0.27 at the wrist.
Papers tested · 17 of 22
Goal (the paper's): Derive the conditions under which a JEPA trained with SIGReg recovers the true latent state of the system, and show where goal-directed data breaks that recovery.
Its hypotheses:
- On goal-directed trajectory data, recovery of the true state is partial and anisotropic. (§6.2, Table 2 (DMC Reacher trajectories)) reproduces
- Under an isotropic random walk the latent recovers the state up to a linear map. (§5.1, Theorem 1; §6.2, Table 2 (OU pairs R² = 0.95)) not tested here
Verdict: reproduces · Claim tested: A SIGReg JEPA trained on goal-directed data recovers the true latent state only partially and unevenly. · Bar: W1's mean R² from latent to joints < 0.80; anisotropy (max − min per-joint R²) ≥ 0.20; the higher-R² model has the higher G1b top-1. · n: 8 447 validation frames, 25 episodes; 2 models
Theirs vs ours: anisotropic recovery on trajectory data, identifiable under isotropic random walks (simulation) vs W1 mean R² 0.565, anisotropy 0.639 (shoulder lift 0.91, wrist flex 0.27); frozen DINOv2-S 0.867 / 0.301 · Not proven: The random-walk side (gate 1: random-walk data on the arm and a retrain) is not run; B3 compares two models.
Paper: arXiv 2605.26379 · Code: no code link found
Claims:
- reproduces SIGReg JEPA latent below pass mark on latent-to-joints R² — theirs anisotropic recovery on trajectory data (simulation) · ours 0.565 (paper §6.2, Table 2)
- reproduces SIGReg JEPA latent at least pass mark on per-joint R² anisotropy — theirs anisotropic (simulation) · ours 0.639 (paper §6.2, Table 2)
- not tested SIGReg JEPA latent equals pass mark on latent-to-joints R² — theirs identifiable (simulation) (paper §5.1 Theorem 1; §7)
Compute: External GPU: no (Orin NX). Cost: $0. One extraction redone after a red-channel bug. · Reproduce: 2026-10-29-paper-vs-hardware-lejepa · First published: 2026-10-29 · Updated: 2026-10-29 · Findings: A one-timestamp indexing bug, found by a consistency check →
When Does LeJEPA Learn a World Model? (Klindt, LeCun, Balestriero; arXiv 2605.26379) asks when a SIGReg JEPA's latent recovers the true state of the system. A robot arm logs that state for free: the joint positions. So on my SO-101 I can measure directly what the paper could only measure in simulation.
What the paper claims, and how it measures it¶
The theory says a SIGReg-trained JEPA recovers the true latent up to a linear map when the data moves through state space like an isotropic random walk. When the data is goal-directed, as in demonstrations, recovery becomes anisotropic: the directions the task moves along a lot are recovered well, the rest poorly. The paper checks this in simulation (DMC Reacher), where the true state is known.
What I ran¶
W1 is my multi-view LeWM, trained with SIGReg on teleoperated bowls demonstrations: goal-directed data, the case the paper predicts goes anisotropic. I fitted a ridge regression from W1's latent to the six logged joints on 17,724 training frames and scored R² on 8,447 validation frames from 25 held-out episodes. The comparison is a frozen DINOv2-S encoder (W2′), which was never trained on this data.
Three bars, frozen before any latent was computed: B1 W1's mean R² stays below 0.80 (recovery is partial); B2 W1's anisotropy, the gap between its best and worst joint, is at least 0.20; B3 the model with the higher R² also ranks action chunks better (G1b top-1).
My first extraction fed both encoders the red channel only (a one-timestamp indexing bug). I found it on a later run, fixed it and re-extracted. The verdict didn't change; the numbers below are the corrected ones.
Their number, my number¶
| W1 (SIGReg LeWM, teleop) | W2′ (frozen DINOv2-S) | |
|---|---|---|
| mean R², latent → joints | 0.565 | 0.867 |
| per joint: pan / lift / elbow / wrist flex / wrist roll / gripper | 0.58 / 0.91 / 0.88 / 0.27 / 0.30 / 0.45 | 0.92 / 0.98 / 0.97 / 0.74 / 0.68 / 0.91 |
| anisotropy (best − worst joint) | 0.639 | 0.301 |
| G1b ranking top-1 | 0.605 | 0.646 |
B1, B2 and B3 are all met. W1 recovers the joints the task moves most (shoulder lift 0.91, elbow 0.88) and loses the ones it barely varies (wrist flex 0.27, wrist roll 0.30): the anisotropic recovery the paper predicts for trajectory data, on a real arm, with the true state logged.
What the gap means¶
The direction of B3 holds, but it's two models, and both sit well below the action-only baseline on G1b (0.68–0.90). A 0.30 R² gap moved ranking by 0.04, so identifiability isn't what limits ranking here. The paper's other half, that an isotropic random walk would make W1's recovery even, needs new data on the arm and a retrain. That's gate 1, and it hasn't run.
Verdict: the anisotropic recovery reproduces on a real arm; the random-walk side is untested.
What would change it: a random-walk dataset of the same size on the arm, the same recipe retrained, and the same measurement.
Compute. External GPU: no (Orin NX). Cost: $0. One extraction redone after the red-channel bug.
Reproduce this: 2026-10-29-paper-vs-hardware-lejepa
Papers tested, 17 of 22. ← Papers tested 16: HWM (hierarchical latent world models) · Papers tested 18: SURVEYOR →