Please use this identifier to cite or link to this item:
Title: Detecting Tool Keypoints with Synthetic Training Data
Authors: VANHERLE, Bram 
PUT, Jeroen 
Issue Date: 2022
Publisher: Springer
Source: Galambos, Péter; Kayacan, Erdal; Madani, Kurosh (Ed.). Robotics, Computer Vision and Intelligent Systems, Springer, p. 190 -207
Series/Report: Communications in Computer and Information Science
Series/Report no.: 1667
Abstract: In this paper an end-to-end technique is presented to create a deep learning model to detect 2D keypoint locations from RGB images. This approach is specifically applied to tools, but can be used on other objects as well. First, 3D models of similar objects are sourced form the internet to avoid the need for exact textured models of the target objects. It is shown in this paper that these exact 3D models are not needed. To avoid the high cost of manually creating a data set, an image generation technique is introduced that is specifically created to generate synthetic training data for deep learning models. Special care was taken when designing this method to ensure that models trained on this data generalize well to unseen, real world data. A neural network architecture, Intermediate Heatmap Model (IHM), is presented that can generate probability heatmaps to predict keypoint locations. This network is equipped with a type of intermediate supervision to improve the results on real world data, when trained on synthetic data. A number of other tricks are employed to ensure generalisation towards real world images. A dataset of real tool images is created to validate this approach. Validation shows that the proposed method works well on real world images. Comparison to two other techniques shows that this method outperforms them. Additionally, it is investigated which deep learning techniques, such as transfer learning and data augmentation, help towards generalization on real data.
Keywords: Object keypoint detection;Deep learning;Synthetic data generation
Document URI:
ISBN: 9783031196492
DOI: 10.1007/978-3-031-19650-8_10
Category: B2
Type: Book Section
Appears in Collections:Research publications

Files in This Item:
File Description SizeFormat 
  Restricted Access
Published version2.2 MBAdobe PDFView/Open    Request a copy
Show full item record

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.