Robohub.org
 

The importance of research reproducibility in robotics

by
20 September 2017



share this:

As highlighted in a previous post, despite the fact that robotics is increasingly regarded as a ‘Science’, as shown by the launch of new journals such as Science Robotics, reproducibility of experiments is still difficult or entirely lacking.

This is quite unfortunate as the possibility of reproducing experimental results is a cornerstone of the scientific method. This situation pushes serious discussions (What’s ‘soft robotics’? Is it needed? What has to be ’soft’?) and paradigm clashes (Good Old Fashioned Artificial Intelligence vs. Deep Learning vs. Embodied Cognition) into the realm of literary controversy or even worse religious territory fights, with quite little experimental evidence supporting the claims of the different parties. Not even wrong, as they say (following Peter Woit’s arguments on String Theory)?

The robotics community has been aware of these issues for a long time and more and more researchers in recent years have published datasets, code and other valuable information to allow others to reproduce their results. We are heading in the right direction, but we probably need to do more.

I think we should therefore welcome the fact that for the first time ever, IEEE R&A Mag. will start accepting R-articles (i.e., papers that report experiments aiming to be fully reproducible) beginning this September. Actually, they will also accept short articles reporting on the replication of R-Article results, and author replies are solicited and will be published after peer-review. The result will be a two-stage high-quality review process. The first stage will be the ordinary rigorous review process of a top-tier publishing venue. The second stage will be the replication of the experiments by the community (which is the core of the scientific method).

This seems like a historical improvement, doesn’t it?

There is more information on this in the column I wrote in the September issue of IEEE Robotics and Automation.



tags:


Fabio Bonsignorio is a professor in the BioRobotics Institute at the Scuola Superiore Sant'Anna (Pisa, Italy).
Fabio Bonsignorio is a professor in the BioRobotics Institute at the Scuola Superiore Sant'Anna (Pisa, Italy).





Related posts :



Tesla’s Optimus robot isn’t very impressive – but it may be a sign of better things to come

Musk has now unveiled a prototype of the robot, called Optimus, which he hopes to mass-produce and sell for less than US$20,000 (A$31,000).
04 October 2022, by

Bipedal robot achieves Guinness World Record in 100 metres

Cassie the robot, developed at Oregon State University, records the fastest 100 metres by a bipedal robot.
03 October 2022, by and

Breaking through the mucus barrier

A capsule that tunnels through mucus in the GI tract could be used to orally administer large protein drugs such as insulin.
02 October 2022, by

Women in Tech leadership resources from IMTS 2022

There’ve been quite a few events recently focusing on Women in Robotics, Women in Manufacturing, Women in 3D Printing, in Engineering, and in Tech Leadership. One of the largest tradeshows in the US is IMTS 2022. Here I bring you some resources shared in the curated technical content and leadership sessions.
29 September 2022, by and

MIT engineers build a battery-free, wireless underwater camera

The device could help scientists explore unknown regions of the ocean, track pollution, or monitor the effects of climate change.
27 September 2022, by

How do we control robots on the moon?

In the future, we imagine that teams of robots will explore and develop the surface of nearby planets, moons and asteroids - taking samples, building structures, deploying instruments.
25 September 2022, by , and





©2021 - ROBOTS Association


 












©2021 - ROBOTS Association