VLA-JEPA¶
They said a robot policy pretrained on human videos without actions trains cheaply and runs well. We trained it on our data for about $10 and ran it on the arm 10 times. It held: 9 of 10 runs succeeded, at 355 ms per chunk on the robot's computer.
Papers tested · 13 of 22
Goal (the paper's): Pretrain a VLA on human videos without actions through a latent world model, so that its latent action tokens carry state-transition structure without leaking future frames; evaluated on LIBERO, SimplerEnv, LIBERO-Plus and a real Franka.
Its hypotheses:
- Pretraining on action-free human video through a leakage-free latent world model improves robustness to perturbations. (Table 3 (LIBERO-Plus 79.5 vs 62.9 without human video)) not tested here
- The model reaches state-of-the-art success on LIBERO and transfers to a real arm with 100 demonstrations. (Table 1; Fig. 4) reproduces
Verdict: reproduces · Claim tested: VLA-JEPA trains cheaply on SO-101 data and runs on the robot. · Bar: 10-trial operator-judged gate (no numeric pass mark registered); for reranking (H0): oracle / mean error ≤ ~0.7 and N = 8 ≤ 1.5× the latency. · n: 10 trials; H0 on the teleop val split
Theirs vs ours: +1.1 over no-human-video on LIBERO (Table 1); no latency reported vs 9/10 for ~$9.7; 355 ms per chunk on the Orin NX; reranking no-go (0.76, 1.84×) · Not proven: Perturbation robustness (their main claim) not tested.
Paper: arXiv 2602.10098 · Code: ginwind/VLA-JEPA
Claims:
- reproduces VLA-JEPA at least pass mark on success on the arm — theirs real Franka: ahead of π0 and π0.5 in-distribution and with object-layout changes, second on unseen tasks (Fig. 4) · ours 9/10 (paper §4.4, Fig. 4 (real Franka))
- fail best-of-N reranking at most pass mark on oracle-to-mean error ratio · ours 0.76, 1.84× latency (our measured comparison)
Compute: External GPU: yes (vast.ai): ~$9.7 for the fine-tune. Everything else on the Orin NX, $0. · Reproduce: 2026-10-23-paper-vs-hardware-vla-jepa · First published: 2026-10-23 · Updated: 2026-10-23
VLA-JEPA (Sun, Zhang, Qi et al.; arXiv 2602.10098) pretrains a VLA on human video through a latent world model. I trained it for $10, ran it on the arm at 9/10, and tested whether its eight samples were worth choosing between. The paper's honest ablation row says what to expect: robustness, not accuracy.
What the paper claims, and how it measures it¶
A Qwen3-VL backbone with a flow-matching head. Latent-action tokens are read out of the VLM given the initial frame only plus the instruction — no future frame ever enters the VLM, which is the leakage-free principle — and an autoregressive latent world model predicts frozen V-JEPA 2 state chunks from them; L_WM is the teacher-forced prediction error. Robot data adds action tokens after the latent ones and a flow-matching loss; the joint objective is L_FM + β L_WM. Pretrained on Something-Something-v2 (220k human videos) and DROID; fine-tuned per benchmark.
Evaluation: LIBERO at 500 episodes per suite, 97.2 vs π0.5 96.9 — and 96.1 without human video (+1.1). SimplerEnv: Google Robot 65.2 with human video, 78.4 without (the authors attribute the reversal to the real-to-sim gap). LIBERO-Plus, seven perturbation axes: 79.5 vs OpenVLA-OFT 69.6, and 62.9 without human video (+16.6, the number the paper is about). Real Franka, 100 demos, three tasks: 0.70 vs π0 0.57 in distribution. No intervals; ±1 on LIBERO is inside binomial noise at this n.
What I ran¶
Training. Qwen3-VL-2B + DiT-B, world model off at inference, ~$9.7 on vast.ai over two days.
The bowls gate. 9/10 on the Orin NX at 355 ms per chunk (PyTorch); in the twelve-VLA latency benchmark through vla.cpp, 177 ms minimum.
H0 (2026-10-08). The question the policy's design invites: do its 8 flow samples vary enough that picking one with a world model would help? Bars: oracle/mean error ≤ 0.7, latency at N = 8 ≤ 1.5×, baseline success < 85 %. Result: 0.76 (null 0.92); 1.84× (643 vs 349 ms); baseline 90 %. No-go. Averaging the eight chunks got 45 % of the oracle's gain with no model at all. Reranking was dropped the same day, and with it the first architecture version.
Their number, my number¶
| Paper | This arm | |
|---|---|---|
| In-distribution success | +1.1 over no-human-video (LIBERO) | 9/10, same as ACT and SmolVLA (ceiling) |
| Perturbation robustness | +16.6 (LIBERO-Plus) | not tested |
| Sample diversity useful? | not asked | oracle/mean 0.76; mean-of-8 gets 45 % of it free |
| Latency | 108.7 ms (4090) | 355 ms PyTorch / 177 ms vla.cpp on Orin NX |
What the gap means¶
The paper's own ablation rows say the human-video pretraining buys robustness to perturbation, not nominal accuracy, and sometimes costs accuracy (SimplerEnv). A nominal gate at ceiling, which is all I ran, cannot see that and did not. If I ever run a perturbation gate — lighting, background, camera — this paper predicts which policy survives and why; a nominal gate showing no difference would confirm it, not refute it. The H0 result is mine, not theirs: the samples do not differ enough to be worth a chooser, and the cheapest chooser (the mean) is half as good as an oracle for free. That one afternoon ended a design.
Verdict: trains and runs as advertised; the robustness claim is untested here; sample reranking is a no-go.
What would change it: a perturbation gate against SmolVLA on the same layouts, after the camera question (SEVO) is settled.
Compute. External GPU: yes (vast.ai): ~$9.7 for the fine-tune. Everything else on the Orin NX, $0.
Reproduce this: 2026-10-23-paper-vs-hardware-vla-jepa
Papers tested, 13 of 22. ← Papers tested 12: TurboVLA · Papers tested 14: REKI →