A preprint posted on arXiv on 9 October by researchers affiliated with Université Paris-Saclay, CEA, Inria and CNRS examines the limits of detecting manipulation failures from image similarity alone. Robot-WATCHDOG represents objects in a scene and their relationships over time as a graph. The team tests one score that measures geometric and relational deviation from expert demonstrations, and a second that predicts future object states with a graph-attention network and Transformer.

The authors tested both approaches on four compatible tasks from an earlier robot-failure benchmark and on two tasks they created: table setting and waste sorting on a moving conveyor. They excluded interactions such as liquids and deformable objects that do not fit the method’s assumption of persistent object identity. The paper reports that both graph-based scores outperformed image-based detectors particularly on failures involving object interactions and task structure; the non-training geometric-relational score had the strongest overall results and lowest inference cost. The findings are limited to the datasets and tasks selected in the paper and do not establish a safety system validated for every robot or failure type. The work is a preprint that has not yet undergone peer review.