FAIL-Detect and SAFE¶
They said a statistical band fitted on successful runs keeps false alarms at a chosen rate. On our arm we fitted it to a world model's surprise and set that rate to 5 %. It did not hold: it flagged 22.7 % of successful runs and caught 46 % of failures.
Papers tested · 7 of 22
Goal (the paper's): Detect policy failures at run time from scores built without failure data (FAIL-Detect) or from a probe on a VLA's own features trained on both outcomes (SAFE), with a conformal band that fixes the false-alarm rate.
Its hypotheses:
- A conformal band calibrated on successful rollouts keeps false alarms at or below the chosen rate α on in-distribution successes. (FAIL-Detect §IV-B; SAFE §4.3) fail
- Learned scores flag failures early enough to matter, on simulated and real robots. (FAIL-Detect Fig. 4–5; SAFE Table 1, Fig. 4 and 6) fail
Verdict: fail · Claim tested: A conformal band on a learned failure score keeps false alarms at α while catching failures. · Bar: Successes flagged ≤ 1/20; failures flagged ≥ 50 %. · n: 22 + 22 successes, 25 failures, 20 splits
Theirs vs ours: false alarms ≤ α on in-distribution successes (conformal coverage bound) vs 22.7 % of successes flagged at α = 5 %; 46 % of failures caught · Not proven: One world model's surprise as the score; their own scores were not reimplemented.
Paper: arXiv 2503.08558, arXiv 2506.09937 · Code: CXU-TRI/FAIL-Detect, vla-safe/SAFE
Claims:
- fail conformal failure band equals conformal level α on successes flagged — theirs false alarms ≤ α · ours 22.7 % at α = 5 % (FAIL-Detect §IV-B; SAFE §4.3 (coverage ≥ 1 − α))
- fail conformal failure band at least pass mark on failures caught — theirs high recall at the target α · ours 46 % (FAIL-Detect Fig. 4–5; SAFE Table 1)
Compute: External GPU: no (Orin NX). Cost: $0. · Reproduce: 2026-10-23-paper-vs-hardware-contactguard · First published: 2026-10-23 · Updated: 2026-10-23
Two papers, one experiment: both calibrate a time-varying conformal band on successful rollouts and flag a run that leaves it. FAIL-Detect (Xu et al., TRI; arXiv 2503.08558) does it on learned OOD scores with no failure data; SAFE (Gu et al., UofT/TRI; arXiv 2506.09937) on a probe of the VLA's own features trained with failures. I ran the band on W1's surprise (W1-C2). It failed, and the way it failed is the way FAIL-Detect's own hardware section describes.
What the papers claim, and how they measure it¶
FAIL-Detect: a scalar score per step from the policy's observation embedding — logpZO (a flow from the embedding distribution to N(0, I); the score is the log-density), NatPN, RND — plus post-hoc baselines (STAC, which needs 256 policy samples per step; PCA-kmeans; smoothness). A band over time is fitted to the success distribution; a flag is raised when the score exceeds it. Sixteen simulated settings (four robomimic tasks × two policies × ID/OOD); summary is top-1 count across settings: logpZO 10/16. Hardware: Franka bimanual, two tasks, fewer rollouts; logpZO top-1 on 8/12. The caveat is in §VI: with ID-only bands, OOD-but-successful rollouts are flagged, "at times yielding low or close to zero TNR"; the fix needs successes from every new setting. Detection time for STAC exceeds the average success duration.
SAFE: the VLA's last-layer features already separate success from failure across tasks (a shared "failure zone" in t-SNE). A tiny MLP or LSTM on them, trained on seen tasks with trajectory-level labels, generalises to unseen tasks. Threshold by functional conformal prediction on successful calibration rollouts, FPR ≤ α at any time step. Three random seen/unseen splits; ROC-AUC on the max-so-far score (a flag anywhere on a success is a false positive); 14 adapted baselines. Sim unseen AUROC ~0.78 vs best baseline ~0.74; real Franka 30 + 30 per task and WidowX 532 rollouts; < 1 ms overhead.
What I ran¶
W1-C2 (2026-10-19). A time-varying functional CP band on W1's long surprise, fitted on 22 successes, tested on the other 22 successes and the 8 failures, 20 seeds. Bar (median): successes flagged ≤ 1/20, failures ≥ 50 %. Result: successes 22.7 % (IQR 13.6–27.3), failures 46 %, stalls 34 %. Fail. The band as registered fits μ, σ and the radius on the same successes: on synthetic exchangeable data its false-alarm rate is 0.14 at 20 anchors — about α only with separate μ/σ data. So the band was not conformal across time as written, and even a correct band sits on a surprise distribution that widens on fresh data (W1-A: the live reflex fitted on one session flagged 42/42 successes on the next).
C1 probe (2026-10-11), the SAFE-shaped question — does the imagined latent predict failure better than simpler signals? — found policy identity alone at 0.76 beating every probe (current 0.68, imagined 0.72), because the failure set was dominated by one policy.
Their number, my number¶
| Papers | This arm | |
|---|---|---|
| False alarms at the registered α | ≈ α by construction | 22.7 % (α = 5 %) |
| Failures caught | logpZO top-1 in 10/16 settings; SAFE unseen AUROC 0.78 | 46 % |
| Transfer across sessions | "close to zero TNR" without new-setting successes (FAIL-Detect §VI) | 42/42 successes flagged on the next session |
| Policy-identity baseline | not run | 0.76, beats every probe |
What the gap means¶
The unsupervised band fails on hardware the same way in Palo Alto and in Paris: the success distribution shifts with the setting and the band does not know. The supervised probe (SAFE's design) is the one that works on hardware in every paper I read, and the C1 result says my version of it is blocked on a confound the papers do not check — the failure labels came mostly from one policy. The register now requires a policy-identity and episode-length baseline under every monitor result, which neither paper runs.
Verdict: FAIL-Detect's band fails here as its authors warn it can; SAFE's probe is not yet tested cleanly.
What would change it: SAFE-MLP on SmolVLA's action-expert input features exported from vla.cpp, three random task splits, with episode length beside it.
Compute. External GPU: no (Orin NX). Cost: $0.
Reproduce this: 2026-10-23-paper-vs-hardware-contactguard
Papers tested, 7 of 22. ← Papers tested 6: ContactGuard · Papers tested 8: ReAP →