Attribute-Based Robotic Grasping with One-Grasp Adaptation

Yang Yang, Yuanhao Liu, Hengyue Liang, Xibai Lou, Changhyun Choi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Scopus citations


Robotic grasping is one of the most fundamental robotic manipulation tasks and has been actively studied. However, how to quickly teach a robot to grasp a novel target object in clutter remains challenging. This paper attempts to tackle the challenge by leveraging object attributes that facilitate recognition, grasping, and quick adaptation. In this work, we introduce an end-to-end learning method of attribute-based robotic grasping with one-grasp adaptation capability. Our approach fuses the embeddings of a workspace image and a query text using a gated-attention mechanism and learns to predict instance grasping affordances. Besides, we utilize object persistence before and after grasping to learn a joint metric space of visual and textual attributes. Our model is self-supervised in a simulation that only uses basic objects of various colors and shapes but generalizes to novel objects and real-world scenes. We further demonstrate that our model is capable of adapting to novel objects with only one grasp data and improving instance grasping performance significantly. Experimental results in both simulation and the real world demonstrate that our approach achieves over 80% instance grasping success rate on unknown objects, which outperforms several baselines by large margins. Supplementary material is available at

Original languageEnglish (US)
Title of host publication2021 IEEE International Conference on Robotics and Automation, ICRA 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages7
ISBN (Electronic)9781728190778
StatePublished - 2021
Event2021 IEEE International Conference on Robotics and Automation, ICRA 2021 - Xi'an, China
Duration: May 30 2021Jun 5 2021

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)1050-4729


Conference2021 IEEE International Conference on Robotics and Automation, ICRA 2021

Bibliographical note

Funding Information:
*This work was supported by UMII MnDRIVE Ph.D. Graduate Assistantship and MnDRIVE Initiative on Robotics, Sensors, and Advanced Manufacturing.

Publisher Copyright:
© 2021 IEEE


  • Deep learning in grasping and manipulation
  • Grasping
  • Perception for grasping and manipulation


Dive into the research topics of 'Attribute-Based Robotic Grasping with One-Grasp Adaptation'. Together they form a unique fingerprint.

Cite this