Robohub.org
 

Robots that feel by seeing


by
06 March 2021



share this:

Humanoid robot hand. Futuristic cyborg concept.

While modern cameras provide machines with a very well-developed sense of vision, robots still lack such a comprehensive solution for their sense of touch. At ETH Zurich, in the group led by Prof. Raffaello D’Andrea at the Institute for Dynamic Systems and Control, we have developed a tactile sensing principle that allows robots to retrieve rich contact feedback from their interactions with the environment. I recently described our approach in a TEDx talk at the last TEDxZurich. The talk features a tech demo that introduces the novel tactile sensing technology targeting the next generation of soft robotic skins.

Design

The sensing technique is based on a camera that tracks fluorescent particles, which are densely and randomly distributed within a soft, deformable gel. The randomness of the patterns simplifies production of the gel and their density provides strain information at each pixel of the resulting image. In addition, the technique does not make any assumption about the shape of the sensing surface, which can exhibit an arbitrary geometry.

Data Processing

While the images capturing the particles’ motion are intuitive and to some extent visually interpretable, the extraction of accurate physical quantities is challenging. In order to overcome the complexity of modeling the behavior of soft materials in real-time, the information extracted from the images is mapped to the distribution of the applied (shear and pressure) contact forces in a data-driven fashion. Specifically, a neural network is employed, which is trained entirely via accurate finite-element simulations to extract the aforementioned mapping.

Applications

This technology has the potential to impact several application fields, the obvious one being robotic manipulation. In a recent proof-of-concept, we have demonstrated how highly dynamic manipulation tasks can be achieved via the sole use of tactile sensing, as shown in the video below.

In addition, the versatility of this approach makes it suitable for various products beyond the robotics domain. In fact, an artificial sense of touch may find applications in smart prosthetic systems, with the potential to restore tactile sensations to people who have lost limbs.



tags: , , ,


Carlo Sferrazza is a PhD candidate at the Institute for Dynamic Systems and Control, ETH Zurich, under the supervision of Prof. Dr. Raffaello D’Andrea. His current research interests include the design and the development of vision-based, data-driven tactile sensors, and learning-based model predictive control.
Carlo Sferrazza is a PhD candidate at the Institute for Dynamic Systems and Control, ETH Zurich, under the supervision of Prof. Dr. Raffaello D’Andrea. His current research interests include the design and the development of vision-based, data-driven tactile sensors, and learning-based model predictive control.





Related posts :



Robot Talk Episode 119 – Robotics for small manufacturers, with Will Kinghorn

  02 May 2025
In the latest episode of the Robot Talk podcast, Claire chatted to Will Kinghorn from Made Smarter about how to increase adoption of new tech by small manufacturers.

Multi-agent path finding in continuous environments

  01 May 2025
How can a group of agents minimise their journey length whilst avoiding collisions?

Interview with Yuki Mitsufuji: Improving AI image generation

  29 Apr 2025
Find out about two pieces of research tackling different aspects of image generation.

Robot Talk Episode 118 – Soft robotics and electronic skin, with Miranda Lowther

  25 Apr 2025
In the latest episode of the Robot Talk podcast, Claire chatted to Miranda Lowther from the University of Bristol about soft, sensitive electronic skin for prosthetic limbs.

Interview with Amina Mević: Machine learning applied to semiconductor manufacturing

  17 Apr 2025
Find out how Amina is using machine learning to develop an explainable multi-output virtual metrology system.

Robot Talk Episode 117 – Robots in orbit, with Jeremy Hadall

  11 Apr 2025
In the latest episode of the Robot Talk podcast, Claire chatted to Jeremy Hadall from the Satellite Applications Catapult about robotic systems for in-orbit servicing, assembly, and manufacturing.

Robot Talk Episode 116 – Evolved behaviour for robot teams, with Tanja Kaiser

  04 Apr 2025
In the latest episode of the Robot Talk podcast, Claire chatted to Tanja Katharina Kaiser from the University of Technology Nuremberg about how applying evolutionary principles can help robot teams make better decisions.

AI can be a powerful tool for scientists. But it can also fuel research misconduct

  31 Mar 2025
While AI is allowing scientists to make technological breakthroughs, there’s also a darker side to the use of AI in science: scientific misconduct is on the rise.



 

Robohub is supported by:




Would you like to learn how to tell impactful stories about your robot or AI system?


scicomm
training the next generation of science communicators in robotics & AI


©2025.05 - Association for the Understanding of Artificial Intelligence


 












©2025.05 - Association for the Understanding of Artificial Intelligence