SmolVLA on a Jetson Orin NX: 974 → 238 ms per chunk with vla.cpp¶
We wanted to know if SmolVLA could run fast enough on the Jetson to stop stuttering. We served the same trained policy through a lighter engine, vla.cpp, and ran 10 trials per engine. One motion chunk took 238 ms instead of 974 ms, and 8 or 9 of 10 trials succeeded.
Engineering on the Orin · 3 of 8
Type: engineering · Result: SmolVLA 974 → 238 ms per chunk on the Orin NX with vla.cpp; 9/10 and 8/10 on the arm. · n: 10 trials per engine
Not proven: Success differences are within noise at n = 10.
Compute: External GPU: no (Orin NX, 20 trials on the arm). Cost: $0; SmolVLA training cost not recorded. · Reproduce: 2026-10-23-paper-vs-hardware-smolvla · First published: 2026-10-01 · Updated: 2026-10-01 · Findings: I need lower latency on a Jetson →, SmolVLA chunk inference on the Orin →
974 ms per chunk made SmolVLA stutter on the Orin NX. Serving the same checkpoint through vla.cpp brought it to 238 ms with no loss in task success.
Numbers, same bowls task, 10 trials¶
| Engine | Success | Chunk inference |
|---|---|---|
| SmolVLA, PyTorch | 9/10 | 974 ms |
| SmolVLA, vla.cpp | 9/10 | 390 ms |
| SmolVLA, vla.cpp + flash attention | 8/10 | 238 ms |
The 8/10 on the flash-attention build is one failed trial out of ten; the gate is too small to say whether that is a regression or noise. Inference went from blocking the control loop most of the time to 14 % of the time at the 26.6 Hz tick.
What it took¶
- A cppzmq workaround for Ubuntu 22.04 / JetPack 6, now documented upstream (vla.cpp
#47). - Real SO-101 latency numbers posted on the project's benchmark issue (
#31), which were the first on this hardware. - An
--arch actclient preset so the same client drives ACT and SmolVLA.
Why it matters¶
On GPU-poor hardware the engine is the lever, not the model. Everything that follows runs the policy through vla.cpp, and two weeks later the same engine is where the chunk-boundary fix lands at a cost of 0.4 ms.
Compute. External GPU: no (Orin NX, 20 trials on the arm). Cost: $0; SmolVLA training cost not recorded.
Reproduce this: 2026-10-23-paper-vs-hardware-smolvla
Engineering on the Orin, 3 of 8. ← Engineering on the Orin 2: ACT and SmolVLA on an SO-101: a 10-trial evaluation harness on a €350 arm kit · Engineering on the Orin 4: Twelve VLAs on a Jetson Orin NX: latency and memory with vla.cpp →