ContactGuard¶
They said a world model can predict a failed grasp before contact by imagining the next moments. On our arm we built the same monitor and compared it with the current camera view. It did not hold: imagining scored 0.70, the plain current view 0.82.
Papers tested · 6 of 22
Goal (the paper's): Detect grasp failures before contact for a chunked policy, by rolling an action-conditioned latent world model forward under the chunk the policy already emitted and reading the imagined latent with a small probe; shown on a dual-arm AgileX Piper with ACT.
Its hypotheses:
- A probe on the imagined latent after the grasp predicts grasp failure better than a probe on the current latent. (Table 2 (0.98 vs 0.84)) fail
- The monitor can abort a grasp before contact, without access to the policy. (§3.5 (online pre-contact monitor), §1) not tested here
Verdict: fail · Claim tested: An event-anchored probe on imagined latents predicts grasp failure better than the current latent. · Bar: AUROC ≥ 0.85 and ≥ 0.05 above the current latent. · n: 65 scored rollouts, 21 failures
Theirs vs ours: AUROC 0.98–0.99 imagined vs 0.84–0.92 current over four grasp tasks (Cup 0.98 vs 0.84) vs 0.70 imagined vs 0.82 current · Not proven: Mixed policies and two views against their single policy and three views; corruption control pending.
Paper: arXiv 2608.13438 · Code: no code link found
Claims:
- fail imagined-latent probe above current-latent probe on failure AUROC — theirs AUROC 0.98–0.99 vs 0.84–0.92 over four grasp tasks (Cup 0.98 vs 0.84) · ours 0.70 vs 0.82 (paper Table 2 (ROC AUC, Cup .982 vs .840))
- reproduces current-latent probe above imagined-latent probe on failure AUROC · ours 0.82 vs 0.70 (our measured comparison)
Compute: External GPU: no (Orin NX). Cost: $0. · Reproduce: 2026-10-23-paper-vs-hardware-contactguard · First published: 2026-10-23 · Updated: 2026-10-23 · Findings: I need a probe on a world model's latents →
ContactGuard (Zheng, Johnson-Roberson, Zhi; arXiv 2608.13438) is the closest paper to what I run: a LeWM-style world model used as a pre-contact failure monitor on a real arm. Their monitor works. Mine, built to the same protocol, does not, with the opposite sign.
What the paper claims, and how it measures it¶
Shared ViT-tiny per camera, mean-pool three views then a linear projection → one latent; predictor of four AdaLN-zero blocks; LeWM loss trained teacher-forced on unlabelled rollouts and teleop; then frozen. The probe: anchor 15 frames (0.5 s) before the first gripper-closure command, roll out K = 30 steps under the policy's actual chunk, standardise the imagined latent, fit a class-balanced ℓ2 logistic regression on ~250 labelled grasp attempts per task, freeze one threshold per task on validation. Online: trigger on the first open→close in the upcoming chunk, abort before contact if P(fail) > τ.
Evaluation on an AgileX Piper with an ACT policy, four grasp settings, test episodes excluded from everything. Closed loop: 50 monitored live rollouts per setting with a 5-s pause-and-resume so the would-be outcome is recorded; AUC 0.99 / 0.95 / 0.90 / 0.92 vs a single-view LeWM and a no-rollout probe. Offline pools (n = 101–144, five seeded 5-fold CV): ContactGuard 0.98 vs current latent 0.84 — same encoder, labels, probe, pools, only the imagined future replaced by the present one — with paired bootstrap CIs excluding zero on every task. Action corruption (Table 3): shuffled chunks from other episodes collapse the full model from 0.965 to 0.535; zero and mean actions to ~0.84. So the signal is the specific action's imagined consequence. A counterfactual action swap raises P(fail) by +0.25 to +0.65. Proprioception added to the probe hurts on all four (small-data shortcut). Runtime 19 ms on a 5090.
What I ran¶
W1-C1 (2026-10-19). The protocol as written: anchor 15 frames before the first close, 30-step imagination under the executed chunk, logistic probe, current latent as the matched baseline, 5-fold × 5 seeds on the 69 published rollouts (65 scored; two stalls and two others never close). Bar: AUROC ≥ 0.85 and ≥ 0.05 above the current latent. Result: imagined 0.70 (folds 0.72 ± 0.13), current 0.82 (0.83 ± 0.11). Imagination is 0.12 worse. The fold threshold gives 27 % held-out false aborts.
W1 itself follows the multi-view ContactGuard recipe (mean-pooled views), which is why the comparison is as direct as it gets.
Their number, my number¶
| Paper (Piper, ACT) | This arm (SO-101, mixed policies) | |
|---|---|---|
| Imagined-latent probe | 0.98 | 0.70 |
| Current-latent probe | 0.84 | 0.82 |
| Difference | +0.14, CI excludes 0 | −0.12 |
| Labelled attempts | ~250 per task | 65 |
| Views | 3 | 2 |
| World-model training data | rollouts + teleop | teleop only |
| Arm | torque-controlled | position-controlled servos |
What the gap means¶
The current-latent probes agree (0.84 vs 0.82): the encoder reads "this state looks risky" about as well on both arms. The imagined-latent probes do not: theirs adds information, mine subtracts it. The candidates, none proven: their world model saw the deployed policy's own rollouts in training and mine saw only teleop, so my imagination is off-distribution exactly at the states the monitor is for; 65 labels vs 250; two views vs three; and a torque-controlled approach that carries contact information a position-controlled one does not. The paper gives the control that would say which: Table 3's action corruption. If shuffled chunks do not push my imagined-latent AUROC below 0.70, the rollout carries no action information on my data and the encoder, not the probe, is the gap. That test is offline and free and is now in the amendments.
Verdict: does not reproduce; the matched baseline wins on my arm; mechanism unknown.
What would change it: the corruption test, then W1 retrained with policy rollouts in the mix.
Compute. External GPU: no (Orin NX). Cost: $0.
Reproduce this: 2026-10-23-paper-vs-hardware-contactguard
Papers tested, 6 of 22. ← Papers tested 5: D-JEPA · Papers tested 7: FAIL-Detect and SAFE →