Robohub.org
 

Robots that feel by seeing

by
06 March 2021



share this:

Humanoid robot hand. Futuristic cyborg concept.

While modern cameras provide machines with a very well-developed sense of vision, robots still lack such a comprehensive solution for their sense of touch. At ETH Zurich, in the group led by Prof. Raffaello D’Andrea at the Institute for Dynamic Systems and Control, we have developed a tactile sensing principle that allows robots to retrieve rich contact feedback from their interactions with the environment. I recently described our approach in a TEDx talk at the last TEDxZurich. The talk features a tech demo that introduces the novel tactile sensing technology targeting the next generation of soft robotic skins.

Design

The sensing technique is based on a camera that tracks fluorescent particles, which are densely and randomly distributed within a soft, deformable gel. The randomness of the patterns simplifies production of the gel and their density provides strain information at each pixel of the resulting image. In addition, the technique does not make any assumption about the shape of the sensing surface, which can exhibit an arbitrary geometry.

Data Processing

While the images capturing the particles’ motion are intuitive and to some extent visually interpretable, the extraction of accurate physical quantities is challenging. In order to overcome the complexity of modeling the behavior of soft materials in real-time, the information extracted from the images is mapped to the distribution of the applied (shear and pressure) contact forces in a data-driven fashion. Specifically, a neural network is employed, which is trained entirely via accurate finite-element simulations to extract the aforementioned mapping.

Applications

This technology has the potential to impact several application fields, the obvious one being robotic manipulation. In a recent proof-of-concept, we have demonstrated how highly dynamic manipulation tasks can be achieved via the sole use of tactile sensing, as shown in the video below.

In addition, the versatility of this approach makes it suitable for various products beyond the robotics domain. In fact, an artificial sense of touch may find applications in smart prosthetic systems, with the potential to restore tactile sensations to people who have lost limbs.



tags: , , ,


Carlo Sferrazza is a PhD candidate at the Institute for Dynamic Systems and Control, ETH Zurich, under the supervision of Prof. Dr. Raffaello D’Andrea. His current research interests include the design and the development of vision-based, data-driven tactile sensors, and learning-based model predictive control.
Carlo Sferrazza is a PhD candidate at the Institute for Dynamic Systems and Control, ETH Zurich, under the supervision of Prof. Dr. Raffaello D’Andrea. His current research interests include the design and the development of vision-based, data-driven tactile sensors, and learning-based model predictive control.





Related posts :



Soft robotic tool provides new ‘eyes’ in endovascular surgery

The magnetic device can help visualise and navigate complex and narrow spaces.

‘Brainless’ robot can navigate complex obstacles

Researchers who created a soft robot that could navigate simple mazes without human or computer direction have now built on that work, creating a “brainless” soft robot that can navigate more complex and dynamic environments.
21 September 2023, by

Battery-free origami microfliers from UW researchers offer a new bio-inspired future of flying machines

Researchers at the University of Washington present battery-free microfliers that can change shape in mid-air to vary their dispersal distance.

Virtual-reality tech is fast becoming more real

Touch sensations are improving to help sectors like healthcare and manufacturing, while other advances are being driven by the gaming industry.
16 September 2023, by

High-tech microscope with ML software for detecting malaria in returning travellers

Method not as accurate as human experts, but shows promise.
14 September 2023, by and

How drones are used during earthquakes

Drones are being used by responders in the terrible Morocco earthquake.
13 September 2023, by and





©2021 - ROBOTS Association


 












©2021 - ROBOTS Association