Robohub.org
 

The importance of research reproducibility in robotics


by
20 September 2017



share this:

As highlighted in a previous post, despite the fact that robotics is increasingly regarded as a ‘Science’, as shown by the launch of new journals such as Science Robotics, reproducibility of experiments is still difficult or entirely lacking.

This is quite unfortunate as the possibility of reproducing experimental results is a cornerstone of the scientific method. This situation pushes serious discussions (What’s ‘soft robotics’? Is it needed? What has to be ’soft’?) and paradigm clashes (Good Old Fashioned Artificial Intelligence vs. Deep Learning vs. Embodied Cognition) into the realm of literary controversy or even worse religious territory fights, with quite little experimental evidence supporting the claims of the different parties. Not even wrong, as they say (following Peter Woit’s arguments on String Theory)?

The robotics community has been aware of these issues for a long time and more and more researchers in recent years have published datasets, code and other valuable information to allow others to reproduce their results. We are heading in the right direction, but we probably need to do more.

I think we should therefore welcome the fact that for the first time ever, IEEE R&A Mag. will start accepting R-articles (i.e., papers that report experiments aiming to be fully reproducible) beginning this September. Actually, they will also accept short articles reporting on the replication of R-Article results, and author replies are solicited and will be published after peer-review. The result will be a two-stage high-quality review process. The first stage will be the ordinary rigorous review process of a top-tier publishing venue. The second stage will be the replication of the experiments by the community (which is the core of the scientific method).

This seems like a historical improvement, doesn’t it?

There is more information on this in the column I wrote in the September issue of IEEE Robotics and Automation.



tags:


Fabio Bonsignorio is a professor in the BioRobotics Institute at the Scuola Superiore Sant'Anna (Pisa, Italy).
Fabio Bonsignorio is a professor in the BioRobotics Institute at the Scuola Superiore Sant'Anna (Pisa, Italy).





Related posts :



Livestream of RoboCup2025

  18 Jul 2025
Watch the competition live from Salvador!

Tackling the 3D Simulation League: an interview with Klaus Dorer and Stefan Glaser

and   15 Jul 2025
With RoboCup2025 starting today, we found out more about the 3D simulation league, and the new simulator they have in the works.

An interview with Nicolai Ommer: the RoboCupSoccer Small Size League

and   01 Jul 2025
We caught up with Nicolai to find out more about the Small Size League, how the auto referees work, and how teams use AI.

RoboCupRescue: an interview with Adam Jacoff

and   25 Jun 2025
Find out what's new in the RoboCupRescue League this year.

Robot Talk Episode 126 – Why are we building humanoid robots?

  20 Jun 2025
In this special live recording at Imperial College London, Claire chatted to Ben Russell, Maryam Banitalebi Dehkordi, and Petar Kormushev about humanoid robotics.

Gearing up for RoboCupJunior: Interview with Ana Patrícia Magalhães

and   18 Jun 2025
We hear from the organiser of RoboCupJunior 2025 and find out how the preparations are going for the event.

Robot Talk Episode 125 – Chatting with robots, with Gabriel Skantze

  13 Jun 2025
In the latest episode of the Robot Talk podcast, Claire chatted to Gabriel Skantze from KTH Royal Institute of Technology about having natural face-to-face conversations with robots.

Preparing for kick-off at RoboCup2025: an interview with General Chair Marco Simões

and   12 Jun 2025
We caught up with Marco to find out what exciting events are in store at this year's RoboCup.



 

Robohub is supported by:




Would you like to learn how to tell impactful stories about your robot or AI system?


scicomm
training the next generation of science communicators in robotics & AI


©2025.05 - Association for the Understanding of Artificial Intelligence


 












©2025.05 - Association for the Understanding of Artificial Intelligence