Please use this identifier to cite or link to this item:
Title: ClueDepth Grasp: Leveraging positional clues of depth for completing depth of transparent objects
Authors: Yuanlin, Hong
CHEN, Junhong 
Cheng, Yu
Han, Yishi
Liu, Wenyin
Issue Date: 2022
Publisher: Frontiers
Source: Frontiers in Neurorobotics, 16 (Art N° 1041702)
Abstract: Obtaining accurate depth information is key to robot grasping tasks. However, for transparent objects, RGB-D cameras have di culty perceiving them owing to the objects’ refraction and reflection properties. This property makes it di cult for humanoid robots to perceive and grasp everyday transparent objects. To remedy this, existing studies usually remove transparent object areas using a model that learns patterns from the remaining opaque areas so that depth estimations can be completed. Notably, this frequently leads to deviations from the ground truth. In this study, we propose a new depth completion method [i.e., ClueDepth Grasp (CDGrasp)] that works more eectively with transparent objects in RGB-D images. Specifically, we propose a ClueDepth module, which leverages the geometry method to filter-out refractive and reflective points while preserving the correct depths, consequently providing crucial positional clues for object location. To acquire su cient features to complete the depth map, we design a DenseFormer network that integrates DenseNet to extract local features and swin-transformer blocks to obtain the required global information. Furthermore, to fully utilize the information obtained from multi-modal visual maps, we devise a Multi-Modal U-Net Module to capture multiscale features. Extensive experiments conducted on the ClearGrasp dataset show that our method achieves state-of-the-art performance in terms of accuracy and generalization of depth completion for transparent objects, and the successful employment of a humanoid robot grasping capability verifies the e cacy of our proposed method.
Keywords: depth completion;transparent objects;grasping;deep learning;robot
Document URI:
ISSN: 1662-5218
e-ISSN: 1662-5218
DOI: 10.3389/fnbot.2022.1041702
ISI #: 000889800400001
Rights: © 2022 Hong, Chen, Cheng, Han, Reeth, Claesen and Liu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Category: A1
Type: Journal Contribution
Appears in Collections:Research publications

Files in This Item:
File Description SizeFormat 
fnbot-16-1041702.pdfPublished version2.71 MBAdobe PDFView/Open
Show full item record

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.