Skip to content

TurboVLA: a policy that ignored its instruction

We wanted to know why a policy trained on two bowl tasks did the same motion for both instructions. We traced it to the data, then reran the original authors' trainer on the same data. Only about 6 % of frames differ between the two tasks, so ignoring the instruction still fits the rest.

TurboVLA task swap, degrees of pan: ours (path A) 0.020°; LIBERO control 5.525°; bar 5°ours (path A)0.020°LIBERO control5.525°bar 5°

Evaluation and judges · 10 of 14

Type: correction · Result: The policy ignored its instruction on the two-task bowls data (0/3 on the last port); the claim that our port caused it was withdrawn on 2026-10-23. · n: 3 arm trials on the last port; 25 val episodes

Not proven: Only two tasks; the instruction is redundant on 94 % of frames.

Compute: External GPU: yes (vast.ai). The port-era trainings behind this post: cost not recorded; the later path-A rerun: ~$2.8 incl. ~$0.8 of failed starts. · Reproduce: 2026-10-23-paper-vs-hardware-turbovla · First published: 2026-10-08 · Updated: 2026-10-23 (port claim withdrawn)

Ten days of work, several variants, a published PyPI package, and a closed project. TurboVLA fine-tuned on the two-task bowls data learned to perform whichever task it had seen last, whatever the instruction said. The failure taught me two things about data that the later datasets were built around.

Corrected 2026-10-23

The conclusion below about the model was wrong. My port differed from the paper in four places, and the paper's own trainer, run on the same data with the paper's initialisation, also ignores the instruction (task swap 0.020° vs 0.105 for the same model on LIBERO-Goal). The data finding (fact 7) is the whole story. Full account: TurboVLA.

What happened

  • Checkpoint conversion from the LIBERO warm start worked (verified 2026-09-08).
  • Trained on bowls, it rolled out at 0/3 on the last variant. On inspection it did the same motion for "blue in pink" and "pink in blue": a copycat.
  • An upstream-port spec was written for the maintainers, and the honest estimate was about one chance in three of passing a 50-episode benchmark. I stopped.

Why

  1. Instruction-dependent behaviour needs data where the instruction matters. In the bowls set, the frames where the two tasks differ (which bowl to grasp first) are about 6 % of all frames. The rest is identical reach, lift, place. A policy can fit 94 % of the data while ignoring the instruction entirely.
  2. Idle frames teach freezing. The earlier recordings started with the arm at rest for a second or two. TurboVLA learned "stay" as a valid action. Every dataset since has its idle starts trimmed with the same script.

What survived

lerobot-policy-turbovla-so101 stays on PyPI as a working plugin policy; the bowls dataset was re-cut; the two lessons went into the recording guide. Not every project needs to end in a merge to be worth the time.

Compute. External GPU: yes (vast.ai). The port-era trainings behind this post: cost not recorded; the later path-A rerun: ~$2.8 incl. ~$0.8 of failed starts.
Reproduce this: 2026-10-23-paper-vs-hardware-turbovla

Evaluation and judges, 10 of 14. ← Evaluation and judges 9: World-model planning on an SO-101: the commanded joints win (0.90 vs 0.60) · Evaluation and judges 11: VLA-JEPA on the Orin for $9.7, and why best-of-N reranking is a no-go →