ACT and SmolVLA on an SO-101: a 10-trial evaluation harness on a €350 arm kit¶
We wanted to know if a learned policy could run reliably on our arm, measured the same way each time. We built a harness that runs fixed trials and logs every motion, then tested two policies. Both worked, 9/10 or better, but one took 974 ms per motion chunk on the Jetson.
Engineering on the Orin · 2 of 8
Type: engineering · Result: ACT 10/10 (M1) and 9/10 (Orin NX), SmolVLA 9/10 on both; SmolVLA takes 974 ms per chunk on the Orin. · n: 10 trials per policy and machine
Not proven: The task is at ceiling for every policy; success is operator-judged.
Compute: External GPU: none for the gates (Orin NX, M1 Mac, 40 trials of 25 s on the arm). Checkpoint training: cost not recorded. · Reproduce: 2026-09-29-three-policies-so101 · First published: 2026-09-29 · Updated: 2026-10-27 · Findings: The $200 arm executes what it is told →
Correction, 2026-10-29
This post first said "$300 arm", then (2026-10-27) "$230 arm", the list price of the follower and leader. What I actually paid: €350 for the SO-101 pack (follower, leader and the wrist camera). The title now says €350 arm kit and the URL is unchanged. Every part with its price source is on the rig page.
Before any research question, the arm had to run a learned policy reliably and I had to be able to measure it the same way every time. This post is the baseline everything else is compared against.
Setup¶
- SO-101 follower arm (6 DoF, STS3215 servos), SO-101 leader for teleop, Logitech C920 top camera, a wrist camera.
- Jetson Orin NX 16 GB as the robot computer, headless, in a Docker image I published (the Dockerfile is merged upstream in LeRobot,
#4599). - A MacBook M1 for a second machine, and vast.ai for anything that needs a real GPU.
- Task: "put the blue bowl in the pink bowl", and the reverse. 182 teleop episodes, idle starts trimmed, public on the Hub as
ravediamond/so101_bowls_all_trimmed.
The harness¶
monitor_rollout.py runs a fixed number of trials, logs a tick file per trial (joint commands, chunk latency, timestamps), records video from both cameras, and takes one key press from the operator at the end of each trial: success or failure. Every trial since this day sits in one catalogue, 85 of them with video by mid-October. The rule that mattered later: freeze the pass bar before the run.
Results, 10 trials each¶
| Policy | Machine | Success | Chunk inference |
|---|---|---|---|
| ACT | M1 Mac | 10/10 | – |
| SmolVLA 450M (PyTorch) | M1 Mac | 9/10 | – |
| ACT | Orin NX | 9/10 | 59 ms |
| SmolVLA 450M (PyTorch) | Orin NX | 9/10 | 974 ms |
Both policies work. SmolVLA at 974 ms per 50-action chunk is a problem on the Orin: the arm visibly pauses at every chunk boundary. That number is the reason the next post exists.
What I learned¶
The first thing a hobby setup needs is not a better model, it is a harness that produces the same number twice. Operator judgment is fine for 10 trials; it will not be fine for 100, which is a judging problem I only understood two weeks later.
Compute. External GPU: none for the gates (Orin NX, M1 Mac, 40 trials of 25 s on the arm). Checkpoint training: cost not recorded.
Reproduce this: 2026-09-29-three-policies-so101
Engineering on the Orin, 2 of 8. ← Engineering on the Orin 1: LeRobot on a Jetson: why there is no pip install torch, and what it took · Engineering on the Orin 3: SmolVLA on a Jetson Orin NX: 974 → 238 ms per chunk with vla.cpp →