Story details

A preprint posted on arXiv on 9 October by Georgia Tech researchers introduces Splat2Act, a real-to-simulation-to-real workflow for adapting a general-purpose robot-arm policy to a particular indoor setting such as a home or office. A user scans the space with a handheld phone and supplies images of their objects. The system builds a building-scale digital twin using 3D Gaussian Splatting and collision geometry, identifies receptacles and pickup tasks, and generates successful demonstrations in simulation. The authors applied the pipeline to scans of 101 campus environments, where it generated 852 receptacles and more than 68,000 robot demonstrations.

The researchers report that their strongest adapted policy reached 62.5% aggregate pickup success in a target scene. On a real Stretch 3 robot, it achieved 63.9% success on a 12-task subset, compared with 66.7% in simulation on those tasks. They also report that the unadapted π₀.₅ policy achieved no pickups in the captured deployment scenes. The findings are limited to the selected campus scenes, objects and robot setup; they do not establish performance in independent homes or for general commercial use. The work is a preprint that has not yet undergone peer review.