Abstract
Adversarial object rearrangement in the real world (e.g., previously unseen or oversized items in kitchens and stores) could benefit from understanding task scenes, which inherently entail heterogeneous components such as current objects, goal objects, and environmental constraints. The semantic relationships among these components are distinct from each other and crucial for multi-skilled robots to perform efficiently in everyday scenarios. We propose a hierarchical robotic manipulation system that learns the underlying relationships and maximizes the collaborative power of its diverse skills (e.g., PICK-PLACE, PUSH) for rearranging adversarial objects in constrained environments. The high-level coordinator employs a heterogeneous graph neural network (HetGNN), which reasons about the current objects, goal objects, and environmental constraints; the low-level 3D Convolutional Neural Network-based actors execute the action primitives. Our approach is trained entirely in simulation, and achieved an average success rate of 87.88% and a planning cost of 12.82 in real-world experiments, surpassing all baseline methods. Supplementary material is available at https://sites.google.com/umn.edu/versatile-rearrangement.
Original language | English (US) |
---|---|
Title of host publication | 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 1008-1015 |
Number of pages | 8 |
ISBN (Electronic) | 9781665491907 |
DOIs | |
State | Published - 2023 |
Event | 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023 - Detroit, United States Duration: Oct 1 2023 → Oct 5 2023 |
Publication series
Name | IEEE International Conference on Intelligent Robots and Systems |
---|---|
ISSN (Print) | 2153-0858 |
ISSN (Electronic) | 2153-0866 |
Conference
Conference | 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023 |
---|---|
Country/Territory | United States |
City | Detroit |
Period | 10/1/23 → 10/5/23 |
Bibliographical note
Publisher Copyright:© 2023 IEEE.
Keywords
- Deep Learning in Grasping and Manipulation
- Perception for Grasping and Manipulation