Peter K. Allen: Multi-Modal Geometric Learning for Grasping | CMU RI Seminar

01 December 2018

share this:

Link to video on YouTube

Abstract: “In this talk, we will describe methods to enable robots to grasp novel objects using multi-modal data and machine. The starting point is an architecture to enable robotic grasp planning via shape completion using a single occluded depth view of objects. Shape completion is accomplished through the use of a 3D CNN. The network is trained on our open source dataset of over 440,000 3D exemplars captured from varying viewpoints. At runtime, a pointcloud captured from a single point of view is fed into the CNN, which fills in the occluded regions of the scene, allowing grasps to be planned and executed on the completed object, which extends to novel objects as well. We have extended this network to incorporate both depth and tactile information. Offline, the network is provided with both simulated depth and tactile information and trained to predict the object’s geometry, thus filling in regions of occlusion. At runtime, the network is provided a partial view of an object and exploratory tactile information is acquired to augment the captured depth information. We demonstrate that even small amounts of additional tactile information can be incredibly helpful in reasoning about object geometry. We also provide experimental results comparing grasping success using our method.”

John Payne

Related posts :

Interview with Dautzenberg Roman: #IROS2023 Best Paper Award on Mobile Manipulation sponsored by OMRON Sinic X Corp.

The award-winning author describe their work on an aerial robot which can exert large forces onto walls.
19 November 2023, by

Robot Talk Episode 62 – Jorvon Moss

In the latest episode of the Robot Talk podcast, Claire chatted to Jorvon (Odd-Jayy) Moss from Digikey about making robots at home, and robot design and aesthetics.
17 November 2023, by

California is the robotics capital of the world

In California, robotics technology is a small fish in a much bigger technology pond, and that tends to conceal how important Californian companies are to the robotics revolution.
12 November 2023, by

Robot Talk Episode 61 – Masoumeh Mansouri

In the latest episode of the Robot Talk podcast, Claire chatted to Masoumeh (Iran) Mansouri from the University of Birmingham about culturally sensitive robots and planning in complex environments.
10 November 2023, by

The 5 levels of Sustainable Robotics

Robots can solve the UN SDGs and not just via the application area.
08 November 2023, by

Using language to give robots a better grasp of an open-ended world

By blending 2D images with foundation models to build 3D feature fields, a new MIT method helps robots understand and manipulate nearby objects with open-ended language prompts.
06 November 2023, by

©2021 - ROBOTS Association


©2021 - ROBOTS Association