Skip to content

Porting ACT to vla.cpp, so a Jetson can run it without PyTorch

We wanted to know if the most common policy on this arm could run on a Jetson without PyTorch. We ported it into the vla.cpp engine and checked it against PyTorch on fixed inputs and on the arm. The port merged upstream; speed was never the point, since PyTorch already ran it at 59 ms per chunk.

Engineering on the Orin · 5 of 8

Type: engineering · Result: ACT added to vla.cpp (merged upstream as #38); the PyTorch checkpoint scores 9/10 on the Orin NX gate. · n: 10 trials

Not proven: The vla.cpp ACT path was not gated on the arm.

Compute: External GPU: no (Orin NX + Mac). Cost: $0; ACT training cost not recorded. · Reproduce: 2026-10-12-act-vlacpp-port · First published: 2026-10-12 · Updated: 2026-10-12 · Findings: ACT in vla.cpp →

ACT is the most common SO-101 policy and vla.cpp did not have it. Merged as vla.cpp #38 after a scoping issue (#37) the maintainer approved in a day. The pitch was never speed: PyTorch ACT already runs at 59 ms per 100-action chunk on the Orin. The pitch is no PyTorch at inference: no 5.5-hour source build, no 12–24 GB image, and CPU-only boards can run it.

What a port involves

CONTRIBUTING.md lists the seven mechanical sites for a new architecture. Most of ACT's pieces already existed in-tree:

  • An ACT-style transformer decoder with learned queries, from the TurboVLA implementation.
  • Conv + group norm via im2col, from Octo. ACT's ResNet-18 uses frozen batch norm, which folds into the conv weights at conversion time.
  • No flow-matching steps: ACT goes on the num_steps refusal list. The VAE encoder is training-only, so the latent is zeros at inference.

The converter and act.cpp read state and action dims, camera count and resolution from the checkpoint config rather than hard-coding the SO-101, because the maintainer asked for a parity check on lerobot/act_aloha_sim_transfer_cube_human as well: bimanual, 14-dim, one 480×640 camera.

How it was tested

  • vla_predict_check against PyTorch on fixed inputs for both checkpoints (ours and the ALOHA one).
  • The same gate harness as every other policy here, with an --engine vlacpp path for ACT, on the real arm. The PyTorch checkpoint scores 9/10 on the Orin NX.
  • An --arch act client preset in the LeRobot-side client fork, so the same client drives ACT and SmolVLA (no tokenizer, ImageNet normalization).

Why a hobby lab should do ports

A port is the one contribution nobody can casually redo without your hardware, and it forces you to read the model as code rather than as a paper. The same reading is what later turned up a silently wrong objective in another policy's training code (LeRobot #4381) and the chunk-boundary fix that went into this engine a week after (G13b).

Compute. External GPU: no (Orin NX + Mac). Cost: $0; ACT training cost not recorded.
Reproduce this: 2026-10-12-act-vlacpp-port

Engineering on the Orin, 5 of 8. ← Engineering on the Orin 4: Twelve VLAs on a Jetson Orin NX: latency and memory with vla.cpp · Engineering on the Orin 6: Why your VLA's actions jump at every chunk boundary on real hardware, and a 0.4 ms fix →