Physically plausible scene estimation for manipulation in clutter

Perceiving object poses in a cluttered scene is a challenging problem because of the partial observations available to an embodied robot, where cluttered scenes are especially problematic. In addition to occlusions, cluttered scenes have various cases of uncertainty due to physical object interactions, such as touching, stacking and partial support. In this paper, we discuss these cases of physics-based uncertainty one by one and propose methods for physically-viable scene estimation. Specifically, we use Newtonian physical simulation to validate the plausibility of hypotheses within a generative probabilistic inference framework for: particle filtering, MCMC and an MCMC variant on particle filtering. Assuming that object geometries are known, we estimate the scene as a collection of object poses, and infer a distribution over the state space of scenes as well as the maximum likelihood estimate. We compare with ICP based approaches and present our results for scene estimation in isolated cases of physical object interaction as well as multi-object scenes such that manipulation of graspable objects can be performed with a PR2 robot.