Lucy Smith – Robohub https://robohub.org Connecting the robotics community to the world Mon, 05 Oct 2026 01:19:56 +0000 en-US hourly 1 https://wordpress.org/?v=7.1.3 An open source approach to physical AI from Intrinsic https://robohub.org/an-open-source-approach-to-physical-ai-from-intrinsic/ Fri, 25 Sep 2026 08:41:15 +0000 https://robohub.org/?p=221427 A repeating pattern of a photograph of a silicon chip, recoloured so that it is multi-coloured, in the style of pop art.Deborah Lupton / Pop Chips / Licenced by CC-BY 4.0

Intrinsic (an AI robotics group at Google) announced that the company is making parts of its platform open source. Intrinsic Core™ is a set of ROS-compatible capabilities for building sophisticated robotic applications.

The capabilities included in the open source package are as follow:

  • Intrinsic Control. A hardware-agnostic, real-time control framework that delivers sensor-based control. It allows the robot to dynamically adapt its behavior mid-trajectory based on changes in the environment. It is designed to deliver efficient, collision-free paths. The idea is that users can swap out different robot arms, grippers, sensors, and more without needing to rewrite drivers.
  • Pose estimation. Provides an accurate six degrees of freedom pose estimation of 3D parts.
  • Motion planning. Logic auto-generates efficient paths while avoiding collisions.
  • Grasp planning. Designed to enable various grippers to automatically and dynamically adapt to the location and orientation of a part and grasp it with a higher probability of success.
  • Simulation services. These aim to provide a straightforward way to test and troubleshoot robotic solutions that update as you build.
  • Camera calibration. Automates how a robot’s camera can be calibrated and physically aligned
  • Intrinsic-ROS drivers. Pre-configured, ROS-compatible drivers for supported robots, grippers, and 3D cameras.

Alongside this package, Intrinsic has also released Open Machine Tending Solution (OMTS). This is a ready-made reference design for a real-world use case. The intention is to give developers a starting point to customize their own solution, without needing to build from scratch.

You can download Intrinsic Core on GitHub here.

]]> What’s coming up at #IROS2026? https://robohub.org/whats-coming-up-at-iros2026/ Thu, 24 Sep 2026 09:50:46 +0000 https://robohub.org/?p=221324

The 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026) will be held from 27 September – 1 October in Pittsburgh, USA. The programme includes plenary and keynote talks, workshops, tutorials, forums, competitions, and a debate.

Keynote talks

The keynote talks will be held from 28 – 30 September:

  • Elena De Momi – Open Questions on the Road to Autonomous Surgery
  • Naomi Fitter – Socially-Informed Context Awareness for Robots in Human-Populated Settings
  • Greg Chirikjian – From Hyper-Redundancy to Robot Imagination and Affordance Learning
  • Genta Suzuki – Fujitsu Kozuchi Physical OS: A Research Strategy for Scalable Physical AI
  • Gaurav Bansal – Title to be confirmed
  • Al Rizzi – Developing Advanced Robot Capability at the RAI Institute
  • Yekai Sun – Tokens of the Physical World: Accelerating Intelligence Emergence via Multimodal Embodied Data
  • He Wang – Towards the AlphaGo and ChatGPT Moments of Embodied AI
  • Jen Jen Chung – All data are equal, but some data are more equal than others
  • Yigit Menguc – Soft Robotics, Tensegrity, and the People Who Love Them
  • Robin Murphy – Rethinking Disaster Robotics: The Open Questions That Matter Now
  • Gina Olson – Embracing Design Complexity: A Biomimetic Approach to (Soft?) Snake Robots
  • Tim Bretl – How you can be part of a revolution in composite manufacturing
  • Lael Odhner – Big bets in manipulation systems: where is the frontier today?
  • David Held – Robots That Learn After Deployment: Designing Policies for Fast Adaptation
  • Ankur Mehta – Robotics problems that robotics can’t solve
  • Brittany Duncan – Patterns from the Field: how contextual lessons can inform robot design
  • Ralph Hollis – Lorentz Levitation Technology for Fine Motion Robotics, Haptic Interaction, and Vibration Isolation
  • Genki Yamauchi – From Big Sites to Every Site: Extending Construction Robotics to the Work Done by Hand
  • Nancy Pollard – Title to be confirmed

Forums

Community discussion sessions open to all attendees, these forums will be held on Monday 28 and Tuesday 29 September.

  • Monday 28 (afternoon): US Government Funding
  • Tuesday 29 (morning): Robotics Education: Experiential Learning
  • Tuesday 29 (afternoon): Is Robotics Underinvesting in Its Own Foundations?

Plenary panels

Three plenary panels will be held:

  • Monday 28: Start-ups and entrepreneurship
  • Tuesday 29: Space
  • Wednesday 30: Should robots be generalists or specialists?

Workshops and tutorials

There are 44 workshops and tutorials planned for Sunday 27 September, with a further 42 scheduled for Thursday 1 October. You can find out more about these here.

Introduction to science communication

Join Sabine Hauert (University of Bristol and Robohub), Ella Scallan (Robohub), and Evan Ackerman (IEEE Spectrum) for a science communication crash course. This will take place at 11:00 on Tuesday 29 September. Find out more here.

Find out more

]]> Watch the ICRA keynote and plenary talks https://robohub.org/watch-the-icra-keynote-and-plenary-talks/ Wed, 16 Sep 2026 08:47:13 +0000 https://robohub.org/?p=221222

The 2026 IEEE International Conference on Robotics & Automation (ICRA) was held in Vienna from the 1 – 5 June. The event brought together researchers and industry professionals with expertise spanning many different aspects of robotics.

IEEE have made the keynote and plenary talk recordings available to watch. You can also catch the panel discussions and keynote tutorials.

Watch here.

Available to watch are the following:

  • Plenary talks
    • Ken Goldberg – A Tale of Two Cultures: Can Agentic Coding Close the Gap?
    • Barbara Mazzolai – Natural Revolution: Biological Principles for Frugal and Sustainable Robotics
    • Roland Siegwart – Aerial Robots – From Omnidirectional Flight to Physical Interaction at Height
  • Keynote sessions
    • Autonomous Vehicles & Navigation
    • Medical & Healthcare Robotics
    • Robot Perception & Spatial AI
    • Manipulation, Humanoids, Embodied Design
    • Robot Learning, Planning & Foundation Models
    • Human-Robot Interaction
  • Industry keynote sessions
  • Awards
  • Keynote tutorials
  • Panels
    • Panel 1 – From Humanoid Robotics Research to Startup Creation: The Role of Public Funding
    • Panel 2 – Advancing Sustainability in Robotics: From Green Design to Real-World Impact
    • Panel 3 – Building Sustainable and Trustworthy AI for Automation
    • Panel 4 – Publish or Perish: Surviving the Paper Deluge – Is AI the solution? If not, what is the Solution?
    • Panel 5 – “Robots for All” in a Fragmented World: Competing Global Visions and Shared Futures from Europe, Asia, and the United States
    • Panel 6 – Impact and Financial Return on Humanoid Investment
]]> First 11 vs 11 humanoid soccer game played at RoboCup 2026 https://robohub.org/first-11-vs-11-humanoid-soccer-game-played-at-robocup-2026/ Fri, 28 Aug 2026 06:25:01 +0000 https://aihub.org/?p=21143 Action from the 11 vs 11 humanoid match at RoboCup 2026. Photo credit: RoboCup Federation.

RoboCup 2026 saw history made, as two teams of 11 humanoids took to the soccer field, the first time a full complement of robots has competed. The game saw B-Human (Bremen, Germany) take on HTWK Robots (Leipzig, Germany), with both sides using machines designed by Booster Robotics.

You can watch highlights of the match below:

Back in 1997, RoboCup’s founders set the lofty goal of developing a team of autonomous robots that could beat the human World Cup champions by 2050. There has been significant progress since those early days, and this match saw another step towards that ambition.

“This match shows how far humanoid robotics has come,” said Ubbo Visser, President of the RoboCup Federation. “We have seen increased teamwork and advanced skills over the years already, but the new humanoid hardware paired with the new level of intelligence provided by AI puts humanoid robot soccer on another level.”

Find out more:

]]> #AAMAS2026 blue sky award winner: Foundation world models for agents in changing environments https://robohub.org/aamas2026-blue-sky-award-winner-foundation-world-models-for-agents-in-changing-environments/ Fri, 21 Aug 2026 08:23:06 +0000 https://aihub.org/?p=20949

Florent Delgrange won the Best Blue Sky Paper Award at AAMAS 2026 for his work Foundation World Models for Agents that Learn, Verify, and Adapt Reliably Beyond Static Environments. We caught up with him to find out more about his vision for agent learning.

What is the topic of your Blue Sky Ideas paper and why is it an interesting area for study?

My Blue Sky Ideas paper asks a simple but difficult question: how can an autonomous agent keep learning as its world changes without quietly losing the guarantees that made its behavior trustworthy?

Reinforcement learning and formal methods address complementary parts of this problem. Reinforcement learning allows an agent to learn by trial and error and can scale to environments for which we could never write down every rule. However, the agent is usually asked to maximize a reward. A poorly specified reward can be exploited, and a high reward does not by itself tell us that a safety or coordination requirement has been satisfied. Reactive synthesis starts from the other end: given a model of the environment and a logical description of the intended behavior, it can construct a policy that is correct by design. The difficulty is that it normally needs an explicit, fixed model, which is precisely what an agent lacks in an open and changing world.

The paper proposes a research agenda that brings these traditions into one loop. A foundation world model would be learned from experience, but structured so that a verifier can reason about it. As the agent learns a policy, it would update its model, measure the reliability of its abstraction, and check whether the policy still satisfies its specification. The verifier’s feedback could reject an unsafe update, request data from an uncertain region, or trigger a revision of the model.

This matters because real environments do not politely remain as they were during training. Goals evolve, conditions change, and, in a multi-agent system, every adapting agent changes the environment perceived by the others. Reliability therefore cannot be a certificate obtained once at deployment. It has to be maintained as the agent continues to learn.

What is your vision for foundation world models?

To me, foundation refers first to reuse, not simply to size. I do not envision a larger video predictor trained on more trajectories. I envision a persistent and structured model that an agent can carry across tasks, policies, and changing populations of agents.

One way to think about it is as a map that records more than roads. It should also tell the agent which areas have been surveyed, which conclusions depend on those areas, and when a change in the world has made an old route unreliable. A useful foundation world model should do the same for decision-making: predict what may happen, expose the structure needed for reasoning, and quantify when those predictions are trustworthy enough to support a guarantee.

I’d want such a model to have three key properties. First, it should be calibrated: every learned abstraction should come with a measure of its error or coverage, so that the agent knows where formal conclusions remain valid. Second, it should be compositional: verified local dynamics, behaviors, and certificates should be reusable when a new task is assembled. Third, it should be semantically queryable: a formal requirement or high-level instruction should help the agent derive a suitable reward model, task-specific abstraction, or policy prior with little additional experience.

The durable idea is that the world model becomes a common substrate for learning, planning, and verification. It should help an agent act, but also identify the limits of its competence, gather evidence where those limits matter, and explain why a particular behavior can or cannot currently be certified.

How does this vision differ from current models?

Most current model-based approaches focus on learning for a particular task, environment, policy, or training distribution. Their main model learning objective is predictive accuracy: reconstruct an observation, forecast the next (latent) state, or generate imagined trajectories for planning. Those are important capabilities, but average predictive metric is not the same as fitness for a particular guarantee.

Consider a model for a scenario involving an agent interacting in a warehouse, where the world model predicts almost every transition correctly but misses a rare, dangerous interaction with a forklift. Its average error may be excellent while that transition is exactly the one that determines whether a collision-avoidance claim is valid. The reverse is also possible: a compact model may ignore colors and textures yet preserve everything needed to reason about routes and collisions. For verification, the relevant question is therefore not only “How accurate is the model?” but “Which conclusions does this accuracy justify, for this policy and this requirement?”

Foundation world models would make that connection an explicit design objective. Their abstractions would carry reliability information tied to the behavior being analyzed, and this information would be revised online as the data distribution changes. Previously verified components could be reused and composed, while specifications expressed in logic or language could guide which representation and policy the agent needs for a new task.

The main difference is therefore a change in role. Current models are primarily prediction tools. The model I envisage is a persistent, analyzable basis for learning, adaptation, and formal reasoning, with language models providing a complementary semantic interface. They could translate high-level instructions into candidate specifications or propose structured model updates, while the world model grounds these proposals in experience and the verifier checks their validity. Foundation world models may benefit from scale through broader task and environment coverage, but scale alone does not provide the structure or certificates required for reliability.

Could you give an example of how such a model might work?

Consider a delivery robot in a busy warehouse. Its task could be stated as: “eventually deliver the package while always avoiding collisions.” Instead of manually combining many bonuses and penalties, the system would translate that requirement into a reward model. Learning and verification would then start from the same description of the intended behavior.

As the robot receives observations, it would learn a compact representation of the warehouse and a policy that acts on that representation. A verifier would ask two related questions. Does the policy satisfy the delivery and collision-avoidance requirement in the learned world model? And is the learned abstraction accurate enough, along the routes that matter, for that conclusion to be trusted? A certificate is meaningful only when both answers are supported.

Now suppose a new forklift begins using a shortcut that was rarely visited during training, cluttering the way. Rather than treating an old prediction as a guarantee, the model should lower its confidence in that region. The verifier would detect this, withdraw the affected certificate, reject a risky policy update, direct exploration toward the shortcut, and reinstate a guarantee only after the world model has been recalibrated.

The paper also considers a more ambitious test-time loop. A language model could propose one or more small formal program describing the new dynamics. A model checker would test them, learn to compose with them, and return a counterexample or structural inconsistency when they are wrong. The language model could revise its hypothesis, the robot could collect targeted experience, and the cycle would repeat. In this division of labor, the language model proposes and the formal verifier checks.

We already have pieces of the loop, including formal reward translations, verifiable abstractions, safe policy-improvement methods, and program generation. Building an efficient end-to-end agent that keeps all of these pieces calibrated while it learns remains the research challenge.

What do you think the impact of such a framework on multi-agent systems could be?

Multi-agent systems make the problem both more urgent and more difficult. Each learning agent is a moving part of every other agent’s environment. Even when the physical world is unchanged, the effective dynamics evolve as agents update their policies, join or leave the system, share information, or pursue new objectives.

A foundation world model could retain reusable descriptions of these interaction patterns, while formal specifications state what must hold for the group. A fleet of warehouse robots, for example, may need to avoid collisions and complete deliveries while respecting shared capacity constraints. The model could connect each local policy to the assumptions on which its certificate depends. If one robot changes its route, the system could identify which assumptions and guarantees are affected, collect new data where needed, and revise only the relevant components rather than relearning and re-verifying the entire fleet.

Composition is the main source of potential leverage. Verified local dynamics or coordination behaviors could serve as building blocks for new teams and tasks, and previous agents could provide useful priors for new participants. This could support faster adaptation while making failures easier to diagnose: the system should report which interaction invalidated a certificate and provide a counterexample, rather than only revealing that the joint reward has fallen.

The central obstacle for multi-agent systems is scalability. The joint state space grows very quickly with the number of agents, and local guarantees do not automatically compose into a global one. We will need principled ways to expose dependencies, preserve soundness under composition, and run verification quickly enough to influence learning online. If we can solve those problems, the field could move from learning coordination strategies and checking them afterwards to learning new strategies while continuously tracking which global properties remain guaranteed.

About Florent

Florent Delgrange is a postdoctoral researcher in computer science at the Artificial Intelligence Lab of Vrije Universiteit Brussel (VUB). His research lies at the intersection of reinforcement learning, world models, and formal verification. He develops methods for agents that can learn and adapt while justifying and certifying the behavior they adopt. He completed a joint PhD at VUB and the University of Antwerp in 2024 on the formal verification of deep reinforcement learning policies. His paper Foundation World Models for Agents that Learn, Verify, and Adapt Reliably Beyond Static Environments received the Best Blue Sky Paper Award at AAMAS 2026.
]]> #RoboCup2026 social media round-up https://robohub.org/robocup2026-social-media-round-up/ Wed, 08 Jul 2026 06:56:38 +0000 https://aihub.org/?p=20640

This year, RoboCup took place in Incheon, South Korea, from 2-6 July. The event saw teams take part in competitions, training sessions, and a symposium. Take a look at what the participants got up to in our round up from social media.

Little Booster K1, trying to get up after falling mid football game!
#robocup2026

[image or embed]

— Raghav Arora (@ra-aurora.bsky.social) 6 July 2026 at 05:05

Straight from RoboCup 2026, at Incheon, Korea.
We added an external lidar, and a literal backpack to the booster T1 to achieve accurate indoor localisation and navigation.
@texasrobotics.bsky.social #Robocup2026

[image or embed]

— Raghav Arora (@ra-aurora.bsky.social) 6 July 2026 at 05:01

]]> #RoboCup2026 – humanoid league knockout stages https://robohub.org/robocup2026-humanoid-league-knockout-stages/ Mon, 06 Jul 2026 10:15:49 +0000 https://aihub.org/?p=20629

This weekend saw the finale of the league competitions at RoboCup 2026 in Incheon, South Korea, with the winners in the small, middle, and large humanoid divisions decided. Congratulations to the following teams, who finished in the top three positions in each size class:

Small division

  1. Invic, Wuhan University, China
  2. Hamburg Bit-Bots, Universität Hamburg, Germany
  3. GeoHBots, School of Artificial Intelligence, China University of Geosciences, China

Middle division

  1. B-Human, Universität Bremen and German Research Center for Artificial Intelligence (DFKI), Germany
  2. HTWK Robots, Leipzig University of Applied Sciences, Germany
  3. Rhoban, University of Bordeaux, France

Large division

  1. Tsinghua Hephaestus, Tsinghua University, China
  2. CAU Mountain&Sea, China Agricultural University, China
  3. Water, Beijing Information Science & Technology University, China

You can watch the action from one of the semi-finals in the middle division, which saw HTWK take on Rhoban.

In the final of the middle division, HTWK took on B-Human:

In addition to the main competitions, there were five league-wide awards:

  • Best Customized Humanoid Award: HERoEHS (ALICE 4th version)
  • Best Humanoid Software Award: B-Human (Game Controller)
  • Open Research Challenge: Ruhrbot Devils (AI Camera Platform for Embedded 2D/3D Game Analysis in RoboCup HSL)
  • Best Innovation Award: Bahia Robotics Team
  • Best Referee Award: Anastasia Prisacaru (Berlin United)

Hear from Team Hephaestus of Tsinghua University, who won the large division:

Although the competitions have drawn to a close, RoboCup 2026 continues today with a symposium, which brings together researchers and practitioners from around the world to present and discuss innovative research in robotics and artificial intelligence. You can find out more here.

]]> #RoboCup2026 – humanoid league day 2 https://robohub.org/robocup2026-humanoid-league-day-2/ Fri, 03 Jul 2026 18:36:55 +0000 https://aihub.org/?p=20591

The second day’s play at RoboCup 2026 has drawn to a close with another bumper set of matches. Teams have come from far and wide to take part in the humanoid soccer competition this year, with 17 different countries represented. China is the most represented country, boasting 15 teams across the three divisions. Other countries taking part are geographically widespread, ranging from Colombia to Malaysia, from Germany to Australia.

In advance of the competition, all applying teams provided a video, team description paper, and information about the robots and software that they use. You can see the complete set of these here. As a taster, here is the qualification video from team CAU Mountain&Sea (from China Agricultural University) who are currently leading the small division competition, and are in fourth place in the large division.

Yesterday’s play saw the first issuing of a red card. The robot received two yellow cards for unsafe challenges and was removed from play on safety grounds. You can see a clip of the second foul in question here.

In terms of the vital statistics of the robots, the heaviest and tallest robot is HERoEHS’s ALICE 4 robot, weighing in at 48kg and measuring 160cm tall. There is a big jump down in weight to the second heaviest bot, which is team BigHeroX’s Z4 bipedal humanoid, at 37kg. A number of teams are using the 35kg Unitree G1. At the other end of the spectrum, the lightest and shortest robot is ITAndroids’ Chape, at just 3.8kg and 53cm tall.

With the seeding rounds almost complete (the knockout rounds start tomorrow) the competitions are hotting up nicely. After four competed rounds in the small division, CAU Mountain&Sea is the only team to win all four matches, and sits atop the table with 12 points, and impressively only conceding one goal. Hamburg Bit-Bots and GeoHBots are in second and third place respectively, having both won three games and lost one.

The middle division has fast established itself as one of the most compelling competitions. B-Human has taken a commanding lead, winning all four matches so far with a +35 goal difference. Behind them, there are four teams on nine points: HTWK Robots, Rhoban, whIRLwind Amsterdam, and THMOS.

Over in the large division, teams have completed three rounds of seeding, with just one further round to come tomorrow. At this stage Tsinghua Hephaestus is the only perfect team, sitting on nine points. Behind them, two teams have each won two matches and drawn the third: PCMS-HRG and Robo-Erectus.

Once the seeding rounds have been completed, 12 teams from each division will make it forward to the knockout stages, with the quarter finals taking place in the evening.

Many thanks to JT Genter for providing the photos, videos, and facts for this article.

]]> #RoboCup2026 – humanoid league day 1 https://robohub.org/robocup2026-humanoid-league-day-1/ Thu, 02 Jul 2026 14:14:34 +0000 https://aihub.org/?p=20573 Image credit: RoboCup Federation.

RoboCup 2026 kicked off today in Incheon, South Korea, with the league competitions running until 5 July. It’s an exciting time for RoboCup, as there have been some updates to the leagues and competition format. Most prominently, the soccer leagues will have a primary focus on humanoid robots. In a series of daily updates, we’ll be bringing you the latest results, videos, and news from the humanoid soccer league.

This year, the humanoid league is split into three sizes: large division, middle division, and small division. There are 18 teams participating the small division, 16 in the middle, and an impressive 22 competing in the large division.

The first two days of competition are be devoted to the seeding round. Teams play using the Swiss-system of ranking to decide who gets through to the knockout stages.

Livestreams from the different fields of play can be found here.

At the end of the first day of competition, teams in the small and middle divisions have played two games each, with the large division ending the day part way through the second round. Early leaders in the small division, with the full six points from two games are: GeoHBots, CAU Mountain&Sea, and Hamburg Bit-Bots. There are also three teams in the middle division who have claimed the six point haul: B-Human, RoboRoos, and HTWK Robots.

You can watch a short summary of the first day, including some of the robots in action, from KBS News:

This short video from RoboCup gives a flavour of the day’s happenings, which also featured the opening ceremony.

We will be back tomorrow with further updates on competition results, some highlights from the day, and insights into how the event is progressing.

Useful links

]]> What’s coming up at #RoboCup2026? https://robohub.org/whats-coming-up-at-robocup2026/ Mon, 29 Jun 2026 08:03:25 +0000 https://robohub.org/?p=220072

This year, RoboCup will be held in Incheon, South Korea, from 2-6 July. The event will see teams take part in competitions, training sessions, and a symposium.

It’s an exciting time for RoboCup, as there have been some updates to the leagues and competition format. Most prominently, the soccer leagues will have a primary focus on humanoid robots.

The leagues and their competitions

You can find out more about the different leagues and the competition schedules and details at these links:

WEROB

A workshop focused on sharing projects, experiences, and innovations in educational robotics. This session is geared towards students, mentors, and educators. Find out more here.

Symposium

The RoboCup symposium will take place on 6 July. More information can be found here.

There will be two keynote talks:

  • Hyun Myung, Spatial Intelligence for Autonomous Robot Navigation in the Wild
  • Gentiane Venture, From function to meaning: Making robots that understand and belong

Find out more at the event website.

]]> Congratulations to the #AAMAS2026 best paper award winners https://robohub.org/congratulations-to-the-aamas2026-best-paper-award-winners/ Mon, 08 Jun 2026 10:02:11 +0000 https://robohub.org/?p=220089

The AAMAS 2026 best paper awards were presented at the 25th International Conference on Autonomous Agents and Multiagent Systems, which took place from 25-29 May 2025 in Paphos, Cyprus. The winners and nominees in the three categories (best paper, best student paper, best blue sky paper) are as follows:

Best Paper Award

Winner

  • Developing Guidelines for Human-LLM Agent Teams: A Multi-Stakeholder Lens, Mireia Yurrita, Davide Dell’Anna, Pradeep K. Murukannaiah, Catholijn M Jonker, and Pinar Yolum

Nominees

  • UNCAP: Uncertainty-Guided Neurosymbolic Planning Using Natural Language Communication for Cooperative Autonomous Vehicles, Neel P. Bhatt, Po-han Li, Kushagra Gupta, Rohan Siva, Daniel Milan, Alexander Todd Hogue, Sandeep P. Chinchali, David Fridovich-Keil, Zhangyang Wang, and Ufuk Topcu
  • Ratio-Based Signaling for Source-Victim Separation in Swarm Fault Detection, Longyin Cui
  • The Impossibility of Strategyproof Rank Aggregation, Manuel Eberl and Patrick Lederer
  • Generalized Per-Agent Advantage Estimation for Multi-Agent Policy Optimization, Seongmin Kim, Giseung Park, Woojun Kim, Jiwon Jeon, Seungyul Han, and Youngchul Sung
  • R-Debater: Retrieval-Augmented Debate Generation through Argumentative Memory, Maoyuan Li, Zhongsheng Wang, Haoyuan Li, and Jiamou Liu
  • Defection at First Sight: Learning Partner Selection in Optional Social Dilemmas without Prior Information, Benedict Russell, Chin-wing Leung, and Paolo Turrini
  • Characterizing Consensuses in Belief Flow Networks, Nicolas Schwind, Gauvain Bourgne, and Katsumi Inoue
  • Grassroots Federation: Fair Democratic Governance at Scale, Nimrod Talmon and Ehud Shapiro
  • Health Facility Location in Ethiopia: Leveraging LLMs to Integrate Expert Knowledge into Algorithmic Planning, Yohai Trabelsi, Guojun Xiong, Fentabil Getnet, Stéphane Verguet, and Milind Tambe
  • MeCo: Enhancing LLM-Empowered Multi-Robot Collaboration via Similar Task Memoization, Baiqing Wang, Helei Cui, Bo Zhang, Xiaolong Zheng, Bin Guo, and Zhiwen Yu

Pragnesh Jay Modi Best Student Paper Award

Winner

  • Planning Ahead with RSA: Efficient Signalling in Dynamic Environments by Projecting User Awareness across Future Timesteps, Anwesha Das, John Duff, Jörg Hoffmann, and Vera Demberg

Nominees

  • Efficiently Computing Equilibria in Budget-Aggregation Games, Patrick Becker, Alexander Fries, Matthias Greger, and Erel Segal-Halevi
  • Building Large-Scale Drone Defenses from Small-Team Strategies, Grant Douglas, Stephen Franklin, Claudia Szabo, and Mingyu Guo
  • Robust Counterfactual Inference in Markov Decision Processes, Jessica Lally, Milad Kazemi, and Nicola Paoletti
  • Flow-Based Task Assignment for Large-Scale Online Multi-Agent Pickup and Delivery, Yue Zhang, Zhe Chen, Daniel Harabor, Pierre Le Bodic, and Peter J. Stuckey
  • Reputation as a Solution to Cooperation Collapse in LLM-based MASs, Siyue Ren, Wanli Fu, Xinkun Zou, Chen Shen, Yi Cai, Chu Chen, Zhen Wang, and Shuyue Hu

Blue Sky Ideas Award

Winner

  • Foundation World Models for Agents that Learn, Verify, and Adapt Reliably Beyond Static Environments, Florent Delgrange

Nominees

  • Guiding Sociotechnical Systems toward Value-Norm Equilibrium, Nirav Ajmeri, Marina De Vos, Davide Dell’Anna, Pradeep K. Murukannaiah, Vivek Nallur, Luis Gustavo Nardin, and Munindar P. Singh
  • The Dynamic Turn in Strategy Logics, Rustam Galimullin, Maksim Gladyshev, Munyque Mittelmann, and Nima Motamed
]]> Global robotics technology roadmap https://robohub.org/global-robotics-technology-roadmap/ Wed, 03 Jun 2026 09:13:54 +0000 https://robohub.org/?p=220029 A repeating pattern of a photograph of a silicon chip, recoloured so that it is multi-coloured, in the style of pop art.Deborah Lupton / Pop Chips / Licenced by CC-BY 4.0.

Henrik I Christensen, Professor of Computer Science & Engineering at University of California San Diego, has recently released a global robotics technology roadmap. This position paper focuses on Asia, Europe, and America and outlines the current state-of-the-art in robotics, and highlights the main opportunities.

The roadmap draws on robotics research and industry data to identify a global technology trajectory for the decade 2025–2035. It integrates findings from leading robotics conferences (such as ICRA, IROS, RSS, CoRL), machine-learning venues (including NeurIPS, ICML), and journal publications, combined with market intelligence from trade organizations and regional government strategies. The document is structured for use by policymakers, technology strategists, research agencies, and industrial research and development leaders. It is based on a review of present research, industry statistics and numerous visits by Henrik to research labs across three continents.

Key headline findings of this roadmap are:

  • The global robotics market reached $53.2B in 2024 and is on a trajectory to $178.7B by 2033.
  • Asia dominates industrial deployment (74% of global installations in 2024; China alone 54%), while Europe leads in safety-critical regulation and collaborative cobots, and the United States leads in AI-powered autonomy and defense robotics.
  • Vision-Language-Action (VLA) models are the most consequential algorithmic development of the current period, enabling cross-embodiment generalization for the first time.
  • Soft robotics and compliant mechanisms, enabled by liquid crystal elastomers (LCEs), electroactive polymers (EAPs), and self-healing hydrogels, are bridging the gap between rigid industrial systems and bio-compatible medical devices.
  • The humanoid robot segment, currently $370M in 2025, is projected to reach $6.5B by 2030 , with Chinese original equipment manufacturers (OEMs) and US technology companies racing to scale production.
  • Regulatory asymmetry is a critical geopolitical variable: the EU AI Act, the first comprehensive legal framework for high-risk AI systems, is reshaping humanoid robot design globally.

The 52-page comprehensive document covers the following sub-topics:

  • Introduction and scope. Motivation and methodology.
  • Global market baseline.
  • State of the art: academic research landscape. Embodied AI, foundation models, reinforcement learning, navigation, manipulation and sensing, bio-inspired locomotion, multi-robot systems, and human-robot collaboration.
  • Enabling technologies: cross-cutting advances. Materials science and soft robotics, computing infrastructure, perception and sensing.
  • Regional technology strategies. Europe, Asia, USA.
  • Technology roadmap 2025–2035. Algorithms and AI, hardware and actuation, materials and manufacturing, and systems, safety and deployment.
  • Sector-specific analysis, observations, and recommendations. Manufacturing, logistics, healthcare, agriculture, mining, construction, service robots.
  • Cross-cutting strategic themes. The humanoid convergence race, sustainability, workforce and societal impacts, geopolitical technology risks.
  • Recommended research priorities by region. Covering Europe, USA and Asia.

You can read the roadmap in full here.

]]> Robotics Café brings together autonomous robot practitioners https://robohub.org/robotics-cafe-brings-together-autonomous-robot-practitioners/ Wed, 20 May 2026 13:03:38 +0000 https://robohub.org/?p=219832

The recently launched Robotics Café is a weekly online seminar series to bring together researchers, students and industry practitioners working in the field of autonomous robotics. One of the key aims of the initiative is to provide a dedicated platform for students to present and disseminate their work, enabling broader visibility and impact across academia and industry.

Organised by P.B. Sujit (IISER Bhopal), Sandeep Manjanna (Plaksha University) and Aditya Paranjape (Monash University), the talks take place every Thursday from 17:00-18:00 Indian Standard Time. The link to watch the presentations live via Google Meet is here. Alternatively, you can catch the recordings on the Robotics Café YouTube channel.

The series kicked off with a talk from Professor Debasish Ghose (Indian Institute of Science) entitled: AERObotics: The Art of Catching Objects in the Air. The presentation explored the fascinating intersection of guidance theory and aerial robotics.

In the second lecture, Professor Arun Kumar Singh (University of Tartu) spoke about leveraging predictive uncertainty for model-based planning and control.

Find out more:

]]> A history of RoboCup with Manuela Veloso https://robohub.org/a-history-of-robocup-with-manuela-veloso/ Tue, 24 Mar 2026 12:43:44 +0000 https://aihub.org/?p=19758

RoboCup is an international competition that promotes and advances robotics and AI through the challenges presented by its various leagues. We got the chance to sit down with Professor Manuela Veloso, one of RoboCup’s founders, to find out more about how it all started, how the community has grown over the years, and the vision for the future.

I think it would be very interesting to go right back to the beginning and hear how RoboCup got started. What was the initial idea, and how did it get set up?

So we are talking about the mid-90s. In terms of the research in those days, it was the beginning of the internet and many AI and computer science researchers were focused on the internet, first on sophisticated search algorithms, on natural language understanding, on information retrieval, and then on software agents and machine learning applied to digital information. From what I recall, there was a smaller group of researchers who were interested in actual, physical robots, and in particular in AI and robotics. I myself was specifically interested in the problem of creating autonomous robots with perception (get information from the world), cognition (select action to achieve goals), and then act (execute the planned actions). This combination of perception, cognition, and action is a very good framework for autonomous robots, because they have to get their information from their sensors, they have to reason about actions to achieve their goals, and then execute them. So, during the 90s, I was at Carnegie Mellon with this AI research goal of integrating perception, cognition, and action in autonomous robots.

Over in Canada there was Alan Mackworth, who jointly with his wonderful student, Michael Sahota, built a one-on-one little autonomous robot soccer set-up. Two robots ran on a small field that had a camera overhead, and aimed at scoring each in one of the two little goals. This work showed that this task of kicking a ball and defending and aiming at a goal could be done autonomously. So it was a tremendous demonstration that a robot soccer world could exist. Mostly at the same time, in Japan, Minoru Asada was showing that a big robot could learn with reinforcement learning how to push a ball into a goal. So you have these one-on-one, fully autonomous little robot cars that were pushing balls around in Canada, and then there was this effort of learning to score with a larger robot in Japan. The learning robot didn’t have a team, it was not a real game, but it was showing that reinforcement learning could learn the skill of aiming into a goal. And then there was also Hiroaki Kitano at Sony who was very interested in little humanoids.

So this is very beautiful because all these things came into play – all of us had different interests.

Alan Mackworth did not get involved with RoboCup, but he gave a demonstration of these one-on-one robots at AAAI in 1994. And in those days, I had a PhD student who had just joined – Peter Stone. And Peter was a serious and passionate soccer player. He saw this little game and he came to me and said, “this is what I want to do for my thesis research, robot soccer!” And for me, I was trying to find a research environment where autonomy was needed in the robot world. I had already a student, Karen Haigh, who was working with autonomous office robots, and learning to plan and execute. But with these soccer robots and Peter Stone’s interest everything came to play, and we started robot soccer research in my lab.

In 1996, there was also a robot soccer effort in South Korea, called MIROSOT, and that’s the first competition we participated in. So Peter Stone, myself, and the team we built at Carnegie Mellon – Sorin Achim and Kwun Han – went to South Korea to participate. From South Korea, we flew to a robot soccer workshop in Japan organized by Minoru Asada and Hiroaki Kitano. Also in attendance were Dominique Duhaut, Itsuki Noda, Silvia Coradeschi, and Enrico Pagello. And that’s where RoboCup really started – we decided to do a competition. And the good thing was that Kitano was the chair of IJCAI, which was going to happen in Osaka in the summer of 1997. So we are there in Osaka and literally we came up with this idea of having a robot soccer competition, RoboCup. It was a big moment for us as researchers. We had to come up with the rules of this competition so that people would be able to participate seven months later. We came up with the three leagues that we were interested in and had expertise in.

The small size league, building upon our Carnegie Mellon interests, would have a field with a camera overhead connected to a computer and then the computer would remotely control the robots through radio.

Then Minoru Asada had these bigger robots with wheels, and we created a league that we call the middle-size league to include the robotics research of Minoru Asada and others. And then Itsuki Noda was interested in creating a simulation environment. We thought that this would help get more people participating in this task of robot soccer.

So that’s how the three leagues started: the small size, the middle size, and the simulation. Hiroaki Kitano, Dominique Duhaut and I were in charge of the small-size league, Minoru Asada was in charge of the middle-size league, and Itsuki Noda ran the simulation league.

One of the challenges was to come up with the rules and define the robots and playing fields. I remember my own pragmatism in suggesting that we play on a ping-pong table for the small-size, as a ping-pong table is something that exists in the whole world. That meant that we would have a playing surface, with defined size and texture, anywhere in the world. We decided one ping-pong table for the small-size league, and nine ping-pong tables for the middle-size league.

In the summer of 1997, at IJCAI, when we all went to the actual first RoboCup competition, the space was gorgeous. Hiroaki Kitano had made these beautiful fields and white bleachers around the fields. It was a very beautiful space with an area with computers for the simulation league. There were 80 teams that had joined the simulation online. We were, I believe, five teams for the small size league and about eight to ten for the middle size league.

That’s how it all started. And 1997 it was in Osaka, in 1998 Dominique Duhaut organized RoboCup in Paris, at a modern Science Center, La Villette. And then in 1999, RoboCup was organized by Silvia Coradeschi, in Stockholm, again co-located with the IJCAI conference. In these three years 1997, ’98, and ’99, there were only these three leagues, small size, middle size, and simulation. It was the foundation of everything. RoboCup grew consistently every year in terms of the number of teams, the number of participants, and the number of participating countries.

Some of the pitches in the main soccer arena at RoboCup 2024, held in Eindhoven, The Netherlands.

So how did RoboCup expand and could you talk about the decisions to add extra leagues?

Well, nothing was necessarily planned, it was more related to the research interests that people had. So when we were in Melbourne in 2000, there were two things that were added. One was RoboCupJunior. There was a professor, Elizabeth Sklar, who had a tremendous interest in robotics education for children. She proposed the RoboCupJunior competitions for children K-12. The goal of RoboCupJunior was to train all these young people to do research in robotics. It was and it still is extremely successful. In Melbourne there were probably the same number of children as there were people competing in what we now call the major leagues. As well as the soccer leagues, Elizabeth also ran a dance competition. This was very impressive – the children would dance on stage with their designed, built, and programmed mobile robots.

I believe that the rescue league was also introduced in 2000. The reason why we came up with rescue was because in those days, there was a lot of research on robots in disaster environments. So many people in our groups had an interest in developing robots that were able to handle disaster environments. So, we included that interest. And later on, @home was also introduced because people had an interest in service robots.

The logistics league came later to cover robots inside factories. The reason for introducing new leagues was always to include the community. And we didn’t want anyone to feel they could not come to RoboCup because they did different research with their robots. And so it was a very intellectually inclusive environment for AI and robotics researchers that could perform tasks autonomously – with perception, cognition, and action mostly in teams for particular tasks.

Rescue league arena at RoboCup 2024.

Could you talk about the changes that were announced last year, and the decision behind those?

Back in 1997, Hiroaki Kitano came up with this goal of a robot soccer team being able to beat the human World Cup winners by 2025. (I later tried, in the early 2000s, to rephrase that goal so that it was not about the robots beating the humans, but instead having robots playing alongside human players by 2050.) However, for this to happen, we have started the humanoid league so that the robots would have legs, not wheels. Our roadmap hence moves towards humanoid robots. It also happens that currently there are several robot companies who are producing humanoid robots that are easy to acquire, so researchers do not have the design and build their own humanoid robots, which was a difficult task that was pursued only by a limited group of researchers. In parallel, events started aiming at including humanoid robot soccer. We believed that RoboCup should be visible and a reference for humanoid robot soccer. We then proposed and changed that, in the international RoboCup competitions, robot soccer will focus solely on humanoid robots. That doesn’t mean that the other RoboCup leagues aren’t valuable. But if they continue at a regional level, there will still be venues to foster these other types of interests. The RoboCup international competitions would be solely focused on humanoid robot soccer.

Action from the humanoid soccer league at RoboCup 2025. Image courtesy of Alessandra Rossi.

RoboCup was tremendously successful in terms of including a variety of different autonomous robot research interests from the community. But I also believe that now we should move into just humanoid soccer at the international level. This will help with visibility of the competition and also for consolidation. We can now buy humanoid robots that we didn’t have before. Previously, you had to build them in your universities, and that was not a research direction for many people. But there is not that excuse anymore. You can buy humanoid robots, and increasingly, you can talk with these humanoid robots using GenAI. So it became a different challenge, a much more accessible challenge than before. The challenge will be: can these platforms play soccer in the presence of another team? I think it’s a fascinating new direction to try to focus on this challenge of the game of soccer – multiple players, two teams in a large space, eventually coordinating, collaborating, or playing with and against humans. So that’s the rationale.

When people don’t know the story, they tend to wonder why we have all these other leagues, such as rescue and logistics. It was always driven by the interests of the community and a big heart for a research approach. And we always thought that because we were organizing an event with a venue, that we could include these people that have interests outside of soccer. That has always been our drive. However, it’s true that it diluted a little bit the goal of 2050 – the humanoid soccer robots. Now I think it’s time to go back to soccer as a main focus, and still keep supporting the other interests that we have fostered, more at a regional level.

There are still a lot of details to be sorted out. And, of course, people may be upset by leagues terminating at the international level. I have always participated in the small-size league myself, 20 years of participating, and I am very attached to it. I was always the trustee representing the small-size league, and I am not sure I am happy with the small-size not being in the international event. But I need to think beyond what my personal interests are and try to understand what would have a bigger impact for AI, robotics, and RoboCup. I think that that’s demonstrating a group of humanoids playing soccer. And if we don’t do it at RoboCup, maybe someone else will.

The way I think about it is that we have made this decision and we can reevaluate it in five or ten years. But not making a change and never having the courage to deviate from what we have been pursuing is less exciting. Things have stalled a bit, with the same number of teams, the same people, the same rules, the same type of intellectual and research accomplishments. We need to excite people about something major again. So I think that the community will greatly embrace this new RoboCup international. And we are so well organized locally, that we can support other leagues at the local events.

I do think that, from a scientific and research point of view, it’s the right moment to target the soccer humanoid robots, because of their availability and our RoboCup ultimate goal. I think it would be amazing if people knew RoboCup international as the humanoid robot soccer competition. Can you imagine in 2027 having 100 teams all playing soccer with humanoids? We’ll see.

About Manuela Veloso

  • Herbert A. Simon University Professor Emerita, School of Computer Science, Carnegie Mellon University
  • 2018-2026 Founder and Head, JPMorganChase AI Research, Managing Director

For the last eight years, Manuela Veloso has been the founder and Head of JPMorganChase AI Research and Herbert A. Simon University Professor Emerita at Carnegie Mellon University, where she was faculty in the Computer Science Department and then Head of the Machine Learning Department.

At JPMorganChase, she built a team of 100 top talented members with graduate education (PhD and Masters) in AI and related disciplines. The team focused on pillar areas of AI in finance, including data-driven optimization, planning and search, document analysis, trustworthy AI, AI and mathematical reasoning, continual learning, and multiagent systems. The team published their research in academic venues and addressed and contributed to business needs and vision.

Veloso has a licenciatura degree in Electrical Engineering and an M.Sc. in Electrical and Computer Engineering from Instituto Superior Técnico, Lisbon, an M.A. in Computer Science from Boston University, and a Ph.D. in Computer Science from Carnegie Mellon University. Veloso has Doctorate Honoris Causa degrees from the Örebro University, Sweden, the Instituto Universitário de Lisboa (ISCTE), Portugal, the Université de Bordeaux, France, and the Universidade Católica of Portugal.

She served as president of the Association for the Advancement of Artificial Intelligence (AAAI), and she is co-founder and a Past President of the RoboCup Federation. She is a fellow of main professional organizations in her area, namely AAAI, IEEE, AAAS, and ACM. She is the recipient of the ACM/SIGART Autonomous Agents Research Award, the Einstein Chair of the Chinese Academy of Sciences, an NSF Career Award, and the Allen Newell Medal for Excellence in Research. Veloso is a member of the National Academy of Engineering with a citation “for contributions to artificial intelligence and its applications in robotics and the financial services industry.” She is also a member of the Academy of Sciences of Portugal.

Her research interests are in AI, including Multiagent Systems, Autonomous Robots, Continual Learning Agents, and AI in Finance. For further details, see Manuela’s webpage.

]]> A multi-armed robot for assisting with agricultural tasks https://robohub.org/a-multi-armed-robot-for-assisting-with-agricultural-tasks/ Wed, 18 Mar 2026 12:47:01 +0000 https://robohub.org/?p=219244 Humans often use one hand to grasp the branch for better accessibility, while the other hand is used to perform primary tasks like (a) branch pruning and (b) hand pollination of the flower. (c) An overview of the approach used by Madhav and colleagues, where one robot manipulates the branch to move the flower to the field of view of another robot by planning a force-aware path. Figure from Force Aware Branch Manipulation To Assist Agricultural Tasks.

In their paper Force Aware Branch Manipulation To Assist Agricultural Tasks, which was presented at IROS 2025, Madhav Rijal, Rashik Shrestha, Trevor Smith, and Yu Gu proposed a methodology to safely manipulate branches to aid various agricultural tasks. We interviewed Madhav to find out more.

Could you give us an overview of the problem you were addressing in the paper?

Madhav Rijal (MR): Our work is motivated by StickBug [1], a multi-armed robotic system for precision pollination in greenhouse environments. One of the main challenges StickBug faces is that many flowers are partially or fully hidden within the plant canopy, making them difficult to detect and reach directly for pollination. This challenge also arises in other agricultural tasks, such as fruit harvesting, where target fruits may be occluded by surrounding branches and foliage.

To address this, we study how one robot arm can safely manipulate branches so that these occluded flowers can be brought into the field of view or reachable workspace of another robot arm. This is a challenging manipulation problem because plant branches are deformable, fragile, and vary significantly from one branch to another. In addition, unlike pick-and-place tasks, where objects move freely in space, branches remain attached to the plant, which imposes additional motion constraints during manipulation. If the robot moves a branch without accounting for these constraints and safety limits, it can apply excessive force and damage the branch.

So, the core problem we addressed in this paper is: how can a robot safely manipulate branches to reveal hidden flowers while remaining aware of interaction forces and minimizing damage?

How did your approach go about tackling the problem?

MR: Our approach [2] combines motion planning that accounts for branch constraints with real-time force feedback.

First, we generate a feasible manipulation path using an RRT* (rapidly exploring random tree) algorithm-based planner in the workspace. The planner respects the geometric constraints of the branch and the task requirements. We model branches as deformable linear objects and use a geometric heuristic to identify configurations that are safer to manipulate.

Then, during execution, we monitor the interaction force using a force sensor mounted on the manipulator. If the measured force exceeds a predefined safe threshold, the system does not continue along the same path. Instead, it re-plans the motion online and searches for an alternative path or goal configuration that can reduce branch stress while still achieving the task.

So, the key idea is that the robot does not plan only for reachability. It also adapts its motion based on the physical response of the branch during manipulation.

Madhav with the multi-armed pollination robot, StickBug.

What are the main contributions of your work?

MR: The main contributions of our work are:

  1. A geometric heuristic model for branch manipulation that does not require branch-specific parameter tuning or physical probing.
  2. A motion planning strategy for branch manipulation that respects both workspace and branch constraints, using the geometric heuristic to guide RRT* and incorporating online replanning based on force feedback.
  3. An experimental demonstration showing that force feedback-based motion planning can protect branches from excessive force during manipulation.
  4. Generalization across different branch types, since the method relies primarily on branch geometry and can adapt online to compensate for model inaccuracies.

Could you talk about the experiments that you carried out to test the approach?

MR: We evaluated the proposed method through a set of branch manipulation experiments using five different starting poses, all targeting a common goal region. Each configuration was tested 10 times, resulting in a total of 50 trials. A trial was considered successful if the robot brought the grasp point to within 5 cm of the goal point. For all trials, the planning time limit was set to 400 seconds, and the allowable interaction force range was −40 N to 40 N. Across the 50 trials, 39 were successful and 11 failed, corresponding to a success rate of about 78%. The average number of replanning attempts across all scenarios was 20.

In terms of force reduction, the results show a clear progression in safety. Constraint-aware planning reduced the manipulation force from above 100 N to below 60 N. Building on this, online force-aware replanning further reduced the force from about 60 N to below the desired 40 N threshold. This indicates that safety awareness through geometric heuristics, which model branches as deformable linear objects, together with force-aware online replanning, can effectively lower interaction forces during manipulation.

Overall, the experiments demonstrate that the proposed framework enables safer branch manipulation while maintaining task feasibility. By combining branch-constraint-aware planning with real-time force feedback, the robot can adapt its motion to reduce excessive force and minimize the risk of branch damage. These findings highlight the value of force-aware planning for practical robotic manipulation in agricultural environments.

Do you have plans to further extend this work?

MR: Yes, there are several directions for extending this work.

One current limitation is the need to define a safe force threshold in advance. In practice, different types of branches require different force limits for safe manipulation. A key direction for future work is to learn or estimate safe force thresholds automatically from branch geometry or visual cues.

Another extension is to improve grasp-point selection. Instead of only replanning after grasping, the system could also reason about the most suitable grasp point beforehand so that the required manipulation force is reduced from the start.

We are also interested in designing a compliant gripper with integrated force sensing that is better suited for manipulating delicate branches. In the longer term, we plan to integrate this method into a multi-arm agricultural robot, where one arm manipulates the branch and another performs pollination, pruning, or harvesting.

Overall, this work advances the development of agricultural robots that can actively manipulate branches to support tasks such as harvesting, pruning, and pollination. By exposing fruits, cut points, and hidden flowers within the canopy, this capability can help overcome key barriers to the broader adoption of robot-assisted agricultural technologies.

References

[1] Smith, Trevor, Madhav Rijal, Christopher Tatsch, R. Michael Butts, Jared Beard, R. Tyler Cook, Andy Chu, Jason Gross, and Yu Gu. Design of Stickbug: a six-armed precision pollination robot. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 69-75. IEEE, 2024.
[2] Rijal, Madhav, Rashik Shrestha, Trevor Smith, and Yu Gu, Force Aware Branch Manipulation To Assist Agricultural Tasks. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1217-1222. IEEE, 2025.

About Madhav

Madhav Rijal is a Ph.D. candidate in Mechanical Engineering at West Virginia University working in agricultural robotics. His research combines motion planning, optimization, multi-agent collaboration and distributed decision making to develop robotic systems for precision pollination and other plant-interaction tasks. His current work focuses on branch manipulation and safe robot operation in agricultural environments.
]]> How can robots acquire skills through interactions with the physical world? An interview with Jiaheng Hu https://robohub.org/how-can-robots-acquire-skills-through-interactions-with-the-physical-world-an-interview-with-jiaheng-hu/ Thu, 12 Feb 2026 13:54:12 +0000 https://aihub.org/?p=19450

One of the key challenges in building robots for household or industrial settings is the need to master the control of high-degree-of-freedom systems such as mobile manipulators. Reinforcement learning has been a promising avenue for acquiring robot control policies, however, scaling to complex systems has proved tricky. In their work SLAC: Simulation-Pretrained Latent Action Space for Whole-Body Real-World RL, Jiaheng Hu, Peter Stone and Roberto Martín-Martín introduce a method that renders real-world reinforcement learning feasible for complex embodiments. We caught up with Jiaheng to find out more.

What is the topic of the research in your paper and why is it an interesting area for study?

This paper is about how robots (in particular, household robots like mobile manipulators) can autonomously acquire skills via interacting with the physical world (i.e. real-world reinforcement learning). Reinforcement learning (RL) is a general learning framework for learning from trial-and-error interaction with an environment, and has huge potential in allowing robots to learn tasks without humans hand-engineering the solution. RL for robotics is a very exciting field, as it can open possibilities for robots to self-improve in a scalable way, towards the creation of general-purpose household robots that can assist people in our everyday lives.

What were some of the issues with previous methods that your paper was trying to address?

Previously, most of the successful applications of RL to robotics were done by training entirely in simulation, then deploying the policy in the real-world directly (i.e. zero-shot sim2real). However, such a method has big limitations: on one hand, it is not very scalable, as you need to create task-specific, high-fidelity simulation environments that highly match the real-world environment that you want to deploy the robot in, and this can often take days or months for each and every task. On the other hand, some tasks are actually very hard to simulate, as they involve deformable objects and contact-rich interactions (for example, pouring water, folding clothes, wiping whiteboard). For these tasks, the simulation is often quite different from the real world. This is where real-world RL comes into play: if we can allow a robot to learn by directly interacting with the physical world, we don’t need a simulator anymore. However, while several attempts have been made towards realizing real-world RL, it is actually a very hard problem since: 1. Sample-inefficiency: RL requires a lot of samples (i.e. interaction with the environment) to learn good behavior, which is often impossible to collect in large quantities in the real-world. 2. Safety Issues: RL requires exploration, and random exploration in the real-world is often very very dangerous. The robot can break itself and will never be able to recover from that.

Could you tell us about the method (SLAC) that you’ve introduced?

So, creating high-fidelity simulations is very hard, and directly learning in the real-world is also really hard. What should we do? The key idea of SLAC is that we can use a low-fidelity simulation environment to assist subsequent real-world RL. Specifically, SLAC implements this idea in a two-step process: in the first step, SLAC learns a latent action space in simulation via unsupervised reinforcement learning. Unsupervised RL is a technique that allows the robot to explore a given environment and learn task-agnostic behaviors. In SLAC, we design a special unsupervised RL objective that encourages these behaviors to be safe and structured.

In the second step, we treat these learned behaviors as the new action space of the robot, where the robot does real-world RL for downstream tasks such as wiping whiteboards by making decisions in this new action space. Importantly, this method allow us to circumvent the two biggest problem of real-world RL: we don’t have to worry about safety issues since the new action space is pretrained to be always safe; and we can learn in a sample-efficient way because our new action space is trained to be very structured.

The robot carrying out the task of wiping a whiteboard.

How did you go about testing and evaluating your method, and what were some of the key results?

We test our methods on a real Tiago robot – a high degrees-of-freedom, bi-manual mobile manipulation, on a series of very challenging real-world tasks, including wiping a large whiteboard, cleaning a table, and sweeping trash into a bag. These tasks are challenging from three aspects: 1. They are visuo-motor tasks that require processing of high-dimensional image information. 2. They require the whole-body motion of the robot (i.e. controlling many degrees-of-freedom at the same time), and 3. They are contact-rich, which makes it hard to simulate accurately. On all of these tasks, our method allows us to learn high-performance policies (>80% success rate) within an hour of real-world interactions. By comparison, previous methods simply cannot solve the task, and often risk breaking the robot. So to summarize, previously it was simply not possible to solve these tasks via real-world RL, and our method has made it possible.

What are your plans for future work?

I think there is still a lot more to do at the intersection of RL and robotics. My eventual goal is to create truly self-improving robots that can learn entirely by themselves without any human involvement. More recently, I’ve been interested in how we can leverage foundation models such as vision-language models (VLMs) and vision-language-action models (VLAs) to further automate the self-improvement loop.

About Jiaheng

Jiaheng Hu is a 4th-year PhD student at UT-Austin, co-advised by Prof. Peter Stone and Prof. Roberto Martín-Martín. His research interest is in Robot Learning and Reinforcement Learning, with the long-term goal of developing self-improving robots that can learn and adapt autonomously in unstructured environments. Jiaheng’s work has been published at top-tier Robotics and ML venues, including CoRL, NeurIPS, RSS, and ICRA, and has earned multiple best paper nominations and awards. During his PhD, he interned at Google DeepMind and Ai2, and is a recipient of the Two Sigma PhD Fellowship.

Read the work in full

SLAC: Simulation-Pretrained Latent Action Space for Whole-Body Real-World RL, Jiaheng Hu, Peter Stone, Roberto Martín-Martín.

]]> Taking humanoid soccer to the next level: An interview with RoboCup trustee Alessandra Rossi https://robohub.org/taking-humanoid-soccer-to-the-next-level-an-interview-with-robocup-trustee-alessandra-rossi/ Wed, 14 Jan 2026 09:07:58 +0000 https://aihub.org/?p=19175

A core objective of RoboCup is to promote and advance robotics and AI research through the challenges offered by its various leagues. The ultimate goal of the soccer competition is that, by 2050, a team of fully autonomous humanoid robots will defeat the most recent winner of the FIFA World Cup. To bring this vision closer to reality, the RoboCup Federation has announced several changes to the leagues. We spoke with Alessandra Rossi, a trustee who has been involved in the humanoid soccer league for many years, to learn more.

Could you start by introducing yourself and tell us how you’ve been involved in RoboCup throughout the years, because you’ve been involved in so many aspects of the competition!

I am Alessandra Rossi from the University of Naples “Federico II”, where I am an Assistant Professor of Computer Science. I began working with and collaborating in RoboCup in 2016, when I started my PhD at the University of Hertfordshire in the UK. I am still affiliated with the University of Hertfordshire, as I remain a member of the humanoid KidSize team Bold Hearts, the longest continuously active team in the UK. After a few years, I became the team leader of Bold Hearts.

In 2019, I became a member of both the Technical Committee and the Organizing Committee of the Humanoid League. After serving on the Technical Committee for two years, I was elected to the Executive Committee of the Humanoid League. In 2025, I was elected to the Board of Trustees for the first time.

Over the years, I have steadily increased my involvement and commitment to RoboCup. I have always sought to actively engage the RoboCup community, both during competitions and outside of competition periods. I also work to encourage engagement between the major and junior leagues and to participate in regional RoboCup events.

While working at the University of Hertfordshire as a Visiting Lecturer, we launched an online module that uses RoboCup as a benchmark for teaching robotics to undergraduate students. The module is still running. I initially served as the module leader, and this role has since been taken over by our Bold Hearts teammate, Bente Riegler.

Last year, Maike Paetzel-Prüsmann, Merel Keijsers, and I (as lead authors), in collaboration with several trustees and many members from different leagues, published a paper on the current and future challenges in humanoid robotics. The paper was published in Autonomous Robots and is, to the best of my knowledge, the first to involve such a large and diverse group of contributors from across the RoboCup leagues. It discusses research within RoboCup and the collaboration and synergies between the leagues.

Group photo of the humanoid league teams at RoboCup 2025.

I understand that there are some changes planned for the leagues. Could say something about that, and specifically the changes that affect the soccer and the humanoid side.

The 2050 goal of the RoboCup Federation, as many people are probably aware, is for a team of humanoid robots to play against the winners of the FIFA World Cup. To achieve this, it is necessary to push further in that direction. One of the key changes, therefore, will be a stronger focus on humanoid robots.

Another major change will be the merger of the Standard Platform League (SPL) and the KidSize Humanoid League. This merged league will have the freedom to define its exact format and to develop a new roadmap that aligns the entire league toward a shared objective. While the 2050 goal itself remains unchanged, the path toward achieving it will need to be adjusted.

It is crucial to continue fostering the engagement of teams in the leagues that will be affected by these changes. At the same time, we must recognize that technology is advancing rapidly. Over the past year, in particular, we have seen significant progress in both hardware platforms and large language models. As RoboCup serves as a global benchmark for robotics research, we should continuously strive to advance technology and research—while still having fun.

Soccer is the complex task and behavior we are studying, and it is complex in many dimensions: from physical control and robot motion, to communication and strategy, and even human-like interactions. These include responding to the referee’s whistle, verbal and non-verbal communication among team members, interactions with the coach, and communication with the referee. All of these aspects will ultimately need to be incorporated into the humanoid league.

The RoboCup Federation has agreed some new partnerships with Unitree, Fourier and Booster. What impact will this have on the humanoid league? Will there be a standard platform element with teams using a specific humanoid robot?

I believe we will see a mix of different robots. With the three companies currently sponsoring RoboCup, we have already seen that their robots can achieve a wide range of behaviors, and there have been significant improvements in robot control. Some of these robots can walk very quickly—almost to the point of running.

Initially, there may be the possibility of multiple teams using the same platform. However, we must keep in mind that both hardware and software can become obsolete very quickly, so we need to remain open to multiple options. A robot that is state of the art today may no longer be so in a year or two. As a result, committing to a single standard platform could limit future progress.

For this reason, the current idea is to remain open to multiple platforms. Many teams already have excellent custom-built robots, and further improvements to these platforms should be encouraged. That said, the exact structure has not yet been decided, and these decisions will be made in consultation with the teams. It is important to give the RoboCup community the time it needs to adapt and move forward.

There have been some big advances in the humanoid adult-size league in the past couple of years. What improvements stood out to you at RoboCup2025 in Brazil?

One major change is that we have added extra robots to each team. Previously, teams played with just two robots per side, but matches are now played three versus three.

Another important improvement is the reduced presence of humans on the field. There is no longer a handler assigned to each robot. In the past, a team member had to walk behind the robots in case they fell and risked being damaged.

I have actually played in a match against the winning humanoid team. Naturally, the human team won, but it was an enjoyable and very interesting game, as the robots were surprisingly fast.


Action from the human vs humanoid match at RoboCup 2025.


Further action from the human vs humanoid match at RoboCup 2025.

What has been the general reaction from the RoboCup community to the changes? I guess it depends on which league you’re in as to how much it affects you.

Yes, it depends on which league you are part of. The reactions have been a mix of excitement and passion. Of course, everyone is keen to see improvements, and participants have always been prepared for changes to the rules and the league structures. However, there are still some open questions, and teams are waiting to see how things will evolve. Tomorrow, there will be a meeting with the President and several trustees to address questions raised by the leagues.

The overall direction of RoboCup, guided by the 2050 goal has not changed. Each league has been extremely valuable and has contributed in different ways toward achieving that goal. RoboCup has also been immensely valuable for robotics research more broadly. Beyond being fun, the challenges involved in making robots play soccer are extraordinarily complex. The research and solutions developed within RoboCup can be applied to many other fields and applications.

About Alessandra Rossi

Alessandra is Assistant Professor at the University of Naples Federico II, Italy. Her PhD thesis was part of the Marie Sklodowska-Curie Research ETN SECURE project at the University of Hertfordshire (UK). Her research interests include Human–(Multi) Robot Interaction, social robotics, trust, XAI, multi-agent systems and user profiling. She is Project Manager and co-supervisor of the MSCA PERSEO (955778), TRAIL (101072488) and SWEET (101168792). She is also co-PI of the project ERROR (FA8655-23-1-7060), and part of several national and international projects. Alessandra is also trustee member of RoboCup Federation, and member of the Humanoid League team called Bold Hearts. She is Chair of the IEEE P3108™ “Study Design”, and a member of the “Appendix” groups, she is Program Chair of IEEE RO-MAN 2027, she has been Robotic Challenge Chair at ICSR 2025, Special Session Chair of IEEE RO-MAN 2024, Publicity Chair of IEEE RO-MAN 2022 and 2023, Organising Chair of the 26th RoboCup International Symposium 2023, and she is on the program committee of several international conferences on human–robot interaction and artificial intelligence.

]]> Robohub highlights 2025 https://robohub.org/robohub-highlights-2025/ Mon, 29 Dec 2025 11:19:26 +0000 https://robohub.org/?p=218611

Over the course of the year, we’ve had the pleasure of working with many talented researchers from across the globe. As 2025 draws to a close, we take a look back at some of the excellent blog posts, interviews and podcasts from our contributors.

Teaching robot policies without new demonstrations: interview with Jiahui Zhang and Jesse Zhang

Jiahui Zhang and Jesse Zhang to tell us about their framework for learning robot manipulation tasks solely from language instructions without per-task demonstrations.

CoRL2025 – RobustDexGrasp: dexterous robot hand grasping of nearly any object

Hui Zhang writes about work presented at CoRL2025 on RobustDexGrasp, a novel framework that tackles different grasping challenges with targeted solutions.

Robot Talk Episode 133 – Creating sociable robot collaborators, with Heather Knight

Robot Talk host Claire Asher chatted to Heather Knight from Oregon State University about applying methods from the performing arts to robotics.

Generations in Dialogue: Human-robot interactions and social robotics with Professor Marynel Vasquez

In this podcast from AAAI, host Ella Lan asked Professor Marynel Vázquez about what inspired her research direction, how her perspective on human-robot interactions has changed over time, robots navigating the social world, and more.

Learning robust controllers that work across many partially observable environments

In this blog post, Maris Galesloot summarizes work presented at IJCAI 2025, which explores designing controllers that perform reliably even when the environment may not be precisely known.

Robot Talk Episode 130 – Robots learning from humans, with Chad Jenkins

Claire Asher chatted to Chad Jenkins from University of Michigan about how robots can learn from people and assist us in our daily lives.

Interview with Zahra Ghorrati: developing frameworks for human activity recognition using wearable sensors

Zahra Ghorrati is pursuing her PhD at Purdue University, where her dissertation focuses on developing scalable and adaptive deep learning frameworks for human activity recognition (HAR) using wearable sensors.

Self-supervised learning for soccer ball detection and beyond: interview with winners of the RoboCup 2025 best paper award

We caught up with some of the authors of the RoboCup 2025 best paper award to find out more about the work, how their method can be transferred to applications beyond RoboCup, and their future plans for the competition.

#IJCAI2025 distinguished paper: Combining MORL with restraining bolts to learn normative behaviour

Agata Ciabattoni and Emery Neufeld introduce a framework for guiding reinforcement learning agents to comply with social, legal, and ethical norms.

Robot Talk Episode 114 – Reducing waste with robotics, with Josie Gotz

Claire Asher chatted to Josie Gotz from the Manufacturing Technology Centre about robotics for material recovery, reuse and recycling.

Multi-agent path finding in continuous environments

Kristýna Janovská and Pavel Surynek write about how can a group of agents minimise their journey length whilst avoiding collisions.

RoboCupRescue: an interview with Adam Jacoff

Find out what’s new in the RoboCupRescue League this year.

An interview with Nicolai Ommer: the RoboCup Soccer Small Size League

We caught up with Nicolai to find out more about the Small Size League, how the auto referees work, and how teams use AI.

Interview with Kate Candon: Leveraging explicit and implicit feedback in human-robot interactions

Hear from PhD student Kate about her work on human-robot interactions.

AIhub coffee corner: Agentic AI

The AIhub coffee corner captures the musings of AI experts over a short conversation.

Generations in Dialogue: Multi-agent systems and human-AI interaction with Professor Manuela Veloso

Host Ella Lan chats to Professor Manuela Veloso about her research journey and path into AI, the history and evolution of AI research, inter-generational collaborations, and more.

Preparing for kick-off at RoboCup2025: an interview with General Chair Marco Simões

We spoke to Marco Simões, one of the General Chairs of RoboCup 2025 and President of RoboCup Brazil.

Gearing up for RoboCupJunior: Interview with Ana Patrícia Magalhães

RoboCup Junior Rescue @ WK RoboCup 2024. Photo: RoboCup/Bart van Overbeeke


We heard from the organiser of RoboCupJunior 2025 and find out more about the event.

]]> Teaching robot policies without new demonstrations: interview with Jiahui Zhang and Jesse Zhang https://robohub.org/teaching-robot-policies-without-new-demonstrations-interview-with-jiahui-zhang-and-jesse-zhang/ Thu, 04 Dec 2025 10:47:14 +0000 https://robohub.org/?p=218362 The ReWiND method, which consists of three phases: learning a reward function, pre-training, and using the reward function and pre-trained policy to learn a new language-specified task online.

In their paper ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrations, which was presented at CoRL 2025, Jiahui Zhang, Yusen Luo, Abrar Anwar, Sumedh A. Sontakke, Joseph J. Lim, Jesse Thomason, Erdem Bıyık and Jesse Zhang introduce a framework for learning robot manipulation tasks solely from language instructions without per-task demonstrations. We asked Jiahui Zhang and Jesse Zhang to tell us more.

What is the topic of the research in your paper, and what problem were you aiming to solve?

Our research addresses the problem of enabling robot manipulation policies to solve novel, language-conditioned tasks without collecting new demonstrations for each task. We begin with a small set of demonstrations in the deployment environment, train a language-conditioned reward model on them, and then use that learned reward function to fine-tune the policy on unseen tasks, with no additional demonstrations required.

Tell us about ReWiND – what are the main features and contributions of this framework?

ReWiND is a simple and effective three-stage framework designed to adapt robot policies to new, language-conditioned tasks without collecting new demonstrations. Its main features and contributions are:

  1. Reward function learning in the deployment environment
    We first learn a reward function using only five demonstrations per task from the deployment environment.

    • The reward model takes a sequence of images and a language instruction, and predicts per-frame progress from 0 to 1, giving us a dense reward signal instead of sparse success/failure.
    • To expose the model to both successful and failed behaviors without having to collect failed behavior demonstrations, we introduce a video rewind augmentation: For a video segmentation V(1:t), we choose an intermediate point t1. We reverse the segment V(t1:t) to create V(t:t1) and append it back to the original sequence. This generates a synthetic sequence that resembles “making progress then undoing progress,” effectively simulating failed attempts.
    • This allows the reward model to learn a smoother and more accurate dense reward signal, improving generalization and stability during policy learning.
  2. Policy pre-training with offline RL
    Once we have the learned reward function, we use it to relabel the small demonstration dataset with dense progress rewards. We then train a policy offline using these relabeled trajectories.
  3. Policy fine-tuning in the deployment environment
    Finally, we adapt the pre-trained policy to new, unseen tasks in the deployment environment. We freeze the reward function and use it as the feedback for online reinforcement learning. After each episode, the newly collected trajectory is relabeled with dense rewards from the reward model and added to the replay buffer. This iterative loop allows the policy to continually improve and adapt to new tasks without requiring any additional demonstrations.

Could you talk about the experiments you carried out to test the framework?

We evaluate ReWiND in both the MetaWorld simulation environment and the Koch real-world setup. Our analysis focuses on two aspects: the generalization ability of the reward model and the effectiveness of policy learning. We also compare how well different policies adapt to new tasks under our framework, demonstrating significant improvements over state-of-the-art methods.

(Q1) Reward generalization – MetaWorld analysis
We collect a metaworld dataset in 20 training tasks, each task include 5 demos, and 17 related but unseen tasks for evaluation. We train the reward function with the metaworld dataset and a subset of the OpenX dataset.

We compare ReWiND to LIV[1], LIV-FT, RoboCLIP[2], VLC[3], and GVL[4]. For generalization to unseen tasks, we use video–language confusion matrices. We feed the reward model video sequences paired with different language instructions and expect the correctly matched video–instruction pairs to receive the highest predicted rewards. In the confusion matrix, this corresponds to the diagonal entries having the strongest (darkest) values, indicating that the reward function reliably identifies the correct task description even for unseen tasks.

Video-language reward confusion matrix. See the paper for more information.

For demo alignment, we measure the correlation between the reward model’s predicted progress and the actual time steps in successful trajectories using Pearson r and Spearman ρ. For policy rollout ranking, we evaluate whether the reward function correctly ranks failed, near-success, and successful rollouts. Across these metrics, ReWiND significantly outperforms all baselines—for example, it achieves 30% higher Pearson correlation and 27% higher Spearman correlation than VLC on demo alignment, and delivers about 74% relative improvement in reward separation between success categories compared with the strongest baseline LIV-FT.

(Q2) Policy learning in simulation (MetaWorld)
We pre-train on the same 20 tasks and then evaluate RL on 8 unseen MetaWorld tasks for 100k environment steps.

Using ReWiND rewards, the policy achieves an interquartile mean (IQM) success rate of approximately 79%, representing a ~97.5% improvement over the best baseline. It also demonstrates substantially better sample efficiency, achieving higher success rates much earlier in training.

(Q3) Policy learning in real robot (Koch bimanual arms)
Setup: a real-world tabletop bimanual Koch v1.1 system with five tasks, including in-distribution, visually cluttered, and spatial-language generalization tasks.
We use 5 demos for the reward model and 10 demos for the policy in this more challenging setting. With about 1 hour of real-world RL (~50k env steps), ReWiND improves average success from 12% → 68% (≈5× improvement), while VLC only goes from 8% → 10%.

Are you planning future work to further improve the ReWiND framework?

Yes, we plan to extend ReWiND to larger models and further improve the accuracy and generalization of the reward function across a broader range of tasks. In fact, we already have a workshop paper extending ReWiND to larger-scale models.

In addition, we aim to make the reward model capable of directly predicting success or failure, without relying on the environment’s success signal during policy fine-tuning. Currently, even though ReWiND provides dense rewards, we still rely on the environment to indicate whether an episode has been successful. Our goal is to develop a fully generalizable reward model that can provide both accurate dense rewards and reliable success detection on its own.

References

[1] Yecheng Jason Ma et al. “Liv: Language-image representations and rewards for robotic control.” International Conference on Machine Learning. PMLR, 2023.
[2] Sumedh Sontakke et al. “Roboclip: One demonstration is enough to learn robot policies.” Advances in Neural Information Processing Systems 36 (2023): 55681-55693.
[3] Minttu Alakuijala et al. “Video-language critic: Transferable reward functions for language-conditioned robotics.” arXiv:2405.19988 (2024).
[4] Yecheng Jason Ma et al. “Vision language models are in-context value learners.” The Thirteenth International Conference on Learning Representations. 2024.

About the authors

Jiahui Zhang is a Ph.D. student in Computer Science at the University of Texas at Dallas, advised by Prof. Yu Xiang. He received his M.S. degree from the University of Southern California, where he worked with Prof. Joseph Lim and Prof. Erdem Bıyık.

Jesse Zhang is a postdoctoral researcher at the University of Washington, advised by Prof. Dieter Fox and Prof. Abhishek Gupta. He completed his Ph.D. at the University of Southern California, advised by Prof. Jesse Thomason and Prof. Erdem Bıyık at USC, and Prof. Joseph J. Lim at KAIST.
]]> Human-robot interaction design retreat https://robohub.org/human-robot-interaction-design-retreat/ Tue, 25 Nov 2025 11:29:32 +0000 https://robohub.org/?p=218341 Seventeen multicoloured post-it notes are roughly positioned in a strip shape on a white board. Each one of them has a hand drawn sketch in pen on them, answering the prompt on one of the post-it notes "AI is...." The sketches are all very different, some are patterns representing data, some are cartoons, some show drawings of things like data centres, or stick figure drawings of the people involved.Rick Payne and team / Ai is… Banner / Licenced by CC-BY 4.0.

Earlier this year, the HRI Design Retreat brought together experts from academia and industry in the field of design for human-robot interaction (HRI). During the two-day event, which featured hands-on interactive activities, participants explored the future of design for HRI, how this could be shaped, and worked on a roadmap for the next five-ten years.

The retreat was organised by Patrícia Alves-Oliveira and Anastasia Kouvaras Ostrowski, and you can see a short documentary about it below:

Find out more about the retreat here.

]]> Social media round-up from #IROS2025 https://robohub.org/social-media-round-up-from-iros2025/ Mon, 27 Oct 2025 11:24:44 +0000 https://robohub.org/?p=218118

The 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025) took place from October 19 to 25, 2025 in Hangzhou, China. The programme included plenary and keynote talks, workshops, tutorials, forums, competitions, and a debate. There was also an exhibition where companies and institutions were able to showcase their latest hardware and software.

We cast an eye over the social media platforms to see what participants got up to during the week.

📢 This week, we are participating in the IEEE/RSJ International Conference on Intelligent Robots and Systems #IROS2025 in Hangzhou #China

📸 (IRI researchers right-left): @juliaborrassol.bsky.social, David Blanco-Mulero and Anais Garrell

#IRI #IROSHangzho

[image or embed]

— IRI-Institut de Robòtica i Informàtica Industrial (@iri-robotics.bsky.social) 24 October 2025 at 01:33

Truly enjoyed discussing the consolidation of specialist and generalist approaches to physical AI at #IROS2025.

Hoping to visit Hangzhou in physical rather than digital form myself in the not too distant future – second IROS AC dinner missed in a row.

#Robotics #physicalAI

[image or embed]

— Markus Wulfmeier (@mwulfmeier.bsky.social) 20 October 2025 at 11:25

At #IROS2025 General Chair Professor Hesheng Wang and Program Chair Professor Yi Guo share what makes this year’s conference unique, from the inspiring location to the latest research shaping the future of intelligent robotics. youtu.be/_JzGoH7wilU

[image or embed]

— WebsEdge Science (@websedgescience.bsky.social) 23 October 2025 at 21:32

From Hangzhou, #IROS2025 unites the brightest minds in #Robotics, #AI & intelligent systems to explore the Human–Robotics Frontier. Watch IROS TV for highlights, interviews, and a behind-the-scenes look at the labs shaping our robotic future. youtu.be/SojyPncpH1g

[image or embed]

— WebsEdge Science (@websedgescience.bsky.social) 23 October 2025 at 21:25

Impressive live demonstration by @unitreerobotics.bsky.social #G1 at the #IROS2025 conference! It really works!

[image or embed]

— Davide Scaramuzza (@davidescaramuzza.bsky.social) 23 October 2025 at 15:26

]]> What’s coming up at #IROS2025? https://robohub.org/whats-coming-up-at-iros2025/ Wed, 15 Oct 2025 11:38:49 +0000 https://aihub.org/?p=18631

The 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025) will be held from 19-25 October in Hangzhou, China. The programme includes plenary and keynote talks, workshops, tutorials, forums, competitions, and a debate.

Plenary talks

There are three plenary talks on the programme this year, with one per day on Tuesday 21, Wednesday 22, and Thursday 23 October.

  • Marco Hutter – The New Era of Mobility: Humanoids and Quadrupeds Enter the Real World
  • Hyoun Jin Kim – Autonomous Aerial Manipulation: Toward Physically Intelligent Robots in Flight
  • Song-Chun Zhu – TongBrain: Bridging Physical Robots and AGI Agents

Keynote talks

The keynotes this year fall under eleven umbrella topics:

  • Rehabilitation & Physically Assistive Systems
    • Patrick Wensing – From Controlled Tests to Open Worlds: Advancing Legged Robots and Lower-Limb Prostheses
    • Hao Su – AI-Powered Wearable and Surgical Robots for Human Augmentation
    • Lorenzo Masia – Wearable Robots and AI for Rehabilitation and Human Augmentation
    • Shingo Shimoda – Science of Awareness: Toward a New Paradigm for Brain-Generated Disorders
  • Bio-inspired Robotics
    • Kevin Chen – Agile and robust micro-aerial-robots driven by soft artificial muscles
    • Josie Hughes – Bioinspired Robots: Building Embodied Intelligence
    • Jee-Hwan Ryu – Soft Growing Robots: From Disaster Response to Colonoscopy
    • Lei Ren – Layagrity robotics: inspiration from the human musculoskeletal system
  • Soft Robotics
    • Bram Vanderborght – Self healing materials for sustainable soft robots”
    • Cecilia Laschi – From AI Scaling to Embodied Control: Toward Energy-Frugal Soft Robotics
    • Kyu-Jin Cho – Soft Wearable Robots: Navigating the Challenges of Building Technology for the Human Body
    • Li Wen – Multimodal Soft Robots: Elevating Interaction in Complex and Diverse Environments
  • Al and Robot Learning
    • Fei Miao – From Uncertainty to Action: Robust and Safe Multi-Agent Reinforcement Learning for Embodied AI
    • Xifeng Yan – Adaptive Inference in Transformers
    • Long Cheng – Learning from Demonstrations by the Dynamical System Approach
    • Karinne Ramírez-Amaro – Transparent Robot Decision-Making with Interpretable & Explainable Methods
  • Perception and Sensors
    • Davide Scaramuzza – Low-latency Robotics with Event Cameras
    • Kris Dorsey – Sensor design for soft robotic proprioception
    • Perla Maiolino – Shaping Intelligence: Soft Bodies, Sensors, and Experience
    • Roberto Calandra – Digitizing Touch and its Importance in Robotics
  • Human Robot Interaction
    • Javier Alonso-Mora – Multi-Agent Autonomy: from Interaction-Aware Navigation to Coordinated Mobile Manipulation
    • Jing Xiao – Robotic Manipulation in Unknown and Uncertain Environments
    • Dongheui Lee – From Passive Learner to Pro-Active and Inter-Active Learner with Reasoning Capabilities
    • Ya-Jun Pan – Intelligent Adaptive Robot Interacting with Unknown Environment and Human
  • Embodied Intelligence
    • Fumiya Iida – Informatizing Soft Robots for Super Embodied Intelligence
    • Nidhi Seethapathi – Predictive Principles of Locomotion
    • Cewu Lu – Digital Gene: An Analytical Universal Embodied Manipulation Ideology
    • Long Cheng – Learning from Demonstrations by the Dynamical System Approach
  • Medical Robots
    • Kenji Suzuki – Small-data Deep Learning for AI Doctor and Smart Medical Imaging
    • Li Zhang – Magnetic Microrobots for Translational Biomedicine: From Individual and Modular Designs to Microswarms
    • Kanako Harada – Co-evolution of Human and AI-Robots to Expand Science Frontiers
    • Loredana Zollo – Towards Synergistic Human–Machine Interaction in Assistive and Rehabilitation Robotics: Multimodal Interfaces, Sensory Feedback, and Future Perspectives
  • Field Robotics
    • Matteo Matteucci – Robotics Meets Agriculture: SLAM and Perception for Crop Monitoring and Precision Farming
    • Brendan Englot – Situational Awareness and Decision-Making Under Uncertainty for Marine Robots
    • Abhinav Valada – Open World Embodied Intelligence: Learning from Perception to Action in the Wild
    • Timothy H. Chung – Catalyzing the Future of Human, Robot, and AI Agent Teams in the Physical World
  • Humanoid Robot Systems
    • Kei Okada – Transforming Humanoid Robot Intelligence: From Reconfigurable Hardware to Human-Centric Applications
    • Xingxing Wang – A New Era of Global Collaboration in Intelligent Robotics
    • Wei Zhang – Towards Physical Intelligence in Humanoid Robotics
    • Dennis Hong – Staging the Machine: Not Built for Work, Built for Wonder
  • Mechanisms and Controls
    • Kenjiro Tadakuma – Topological Robotic Mechanisms
    • Angela P. Schoellig – AI-Powered Robotics: From Semantic Understanding to Safe Autonomy
    • Lu Liu – Safety-Aware Multi-Agent Self-Deployment: Integrating Cybersecurity and Constrained Coordination
    • Fuchun Sun – Knowledge-Guided Tactile VLA: Bridging the Sim-to-Real Gap with Physics and Geometry Awareness

Debate

On Wednesday, a debate will be held on the following topic: “Humanoids Will Soon Replace Most Human Workers: True or False?” The participants will be: XingXing Wang (Unitree Robotics), Jun-Oh Ho (Samsung and Rainbow Robotics), Hong Qiao (Chinese Academy of Sciences), Andra Keay, (Silicon Valley Robotics), Yu Sun (EiC, IEEE Trans on Automation Science and Engineering), Tamim Asfour (Professor of Humanoid Robotics, Karlsruhe Institute of Technology), Ken Goldberg (UC Berkeley, Moderator).

Tutorials

There are three tutorials planned, taking place on Monday 20 and Friday 24 October.

Workshops

You can find a list of the workshops here. These will take place on Monday 20 and Friday 24 October.There are 83 to choose from this year.

Find out more

]]> Interview with Zahra Ghorrati: developing frameworks for human activity recognition using wearable sensors https://robohub.org/interview-with-zahra-ghorrati-developing-frameworks-for-human-activity-recognition-using-wearable-sensors/ Wed, 08 Oct 2025 08:02:12 +0000 https://aihub.org/?p=18421
In this interview series, we’re meeting some of the AAAI/SIGAI Doctoral Consortium participants to find out more about their research. Zahra Ghorrati is developing frameworks for human activity recognition using wearable sensors. We caught up with Zahra to find out more about this research, the aspects she has found most interesting, and her advice for prospective PhD students.

Tell us a bit about your PhD – where are you studying, and what is the topic of your research?

I am pursuing my PhD at Purdue University, where my dissertation focuses on developing scalable and adaptive deep learning frameworks for human activity recognition (HAR) using wearable sensors. I was drawn to this topic because wearables have the potential to transform fields like healthcare, elderly care, and long-term activity tracking. Unlike video-based recognition, which can raise privacy concerns and requires fixed camera setups, wearables are portable, non-intrusive, and capable of continuous monitoring, making them ideal for capturing activity data in natural, real-world settings.

The central challenge my dissertation addresses is that wearable data is often noisy, inconsistent, and uncertain, depending on sensor placement, movement artifacts, and device limitations. My goal is to design deep learning models that are not only computationally efficient and interpretable but also robust to the variability of real-world data. In doing so, I aim to ensure that wearable HAR systems are both practical and trustworthy for deployment outside controlled lab environments.

This research has been supported by the Polytechnic Summer Research Grant at Purdue. Beyond my dissertation work, I contribute to the research community as a reviewer for conferences such as CoDIT, CTDIAC, and IRC, and I have been invited to review for AAAI 2026. I was also involved in community building, serving as Local Organizer and Safety Chair for the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2025), and continuing as Safety Chair for AAMAS 2026.

Could you give us an overview of the research you’ve carried out so far during your PhD?

So far, my research has focused on developing a hierarchical fuzzy deep neural network that can adapt to diverse human activity recognition datasets. In my initial work, I explored a hierarchical recognition approach, where simpler activities are detected at earlier levels of the model and more complex activities are recognized at higher levels. To enhance both robustness and interpretability, I integrated fuzzy logic principles into deep learning, allowing the model to better handle uncertainty in real-world sensor data.

A key strength of this model is its simplicity and low computational cost, which makes it particularly well suited for real-time activity recognition on wearable devices. I have rigorously evaluated the framework on multiple benchmark datasets of multivariate time series and systematically compared its performance against state-of-the-art methods, where it has demonstrated both competitive accuracy and improved interpretability.

Is there an aspect of your research that has been particularly interesting?

Yes, what excites me most is discovering how different approaches can make human activity recognition both smarter and more practical. For instance, integrating fuzzy logic has been fascinating, because it allows the model to capture the natural uncertainty and variability of human movement. Instead of forcing rigid classifications, the system can reason in terms of degrees of confidence, making it more interpretable and closer to how humans actually think.

I also find the hierarchical design of my model particularly interesting. Recognizing simple activities first, and then building toward more complex behaviors, mirrors the way humans often understand actions in layers. This structure not only makes the model efficient but also provides insights into how different activities relate to one another.

Beyond methodology, what motivates me is the real-world potential. The fact that these models can run efficiently on wearables means they could eventually support personalized healthcare, elderly care, and long term activity monitoring in people’s everyday lives. And since the techniques I’m developing apply broadly to time series data, their impact could extend well beyond HAR, into areas like medical diagnostics, IoT monitoring, or even audio recognition. That sense of both depth and versatility is what makes the research especially rewarding for me.

What are your plans for building on your research so far during the PhD – what aspects will you be investigating next?

Moving forward, I plan to further enhance the scalability and adaptability of my framework so that it can effectively handle large scale datasets and support real-time applications. A major focus will be on improving both the computational efficiency and interpretability of the model, ensuring it is not only powerful but also practical for deployment in real-world scenarios.

While my current research has focused on human activity recognition, I am excited to broaden the scope to the wider domain of time series classification. I see great potential in applying my framework to areas such as sound classification, physiological signal analysis, and other time-dependent domains. This will allow me to demonstrate the generalizability and robustness of my approach across diverse applications where time-based data is critical.

In the longer term, my goal is to develop a unified, scalable model for time series analysis that balances adaptability, interpretability, and efficiency. I hope such a framework can serve as a foundation for advancing not only HAR but also a broad range of healthcare, environmental, and AI-driven applications that require real-time, data-driven decision-making.

What made you want to study AI, and in particular the area of wearables?

My interest in wearables began during my time in Paris, where I was first introduced to the potential of sensor-based monitoring in healthcare. I was immediately drawn to how discreet and non-invasive wearables are compared to video-based methods, especially for applications like elderly care and patient monitoring.

More broadly, I have always been fascinated by AI’s ability to interpret complex data and uncover meaningful patterns that can enhance human well-being. Wearables offered the perfect intersection of my interests, combining cutting-edge AI techniques with practical, real-world impact, which naturally led me to focus my research on this area.

What advice would you give to someone thinking of doing a PhD in the field?

A PhD in AI demands both technical expertise and resilience. My advice would be:

  • Stay curious and adaptable, because research directions evolve quickly, and the ability to pivot or explore new ideas is invaluable.
  • Investigate combining disciplines. AI benefits greatly from insights in fields like psychology, healthcare, and human-computer interaction.
  • Most importantly, choose a problem you are truly passionate about. That passion will sustain you through the inevitable challenges and setbacks of the PhD journey.

Approaching your research with curiosity, openness, and genuine interest can make the PhD not just a challenge, but a deeply rewarding experience.

Could you tell us an interesting (non-AI related) fact about you?

Outside of research, I’m passionate about leadership and community building. As president of the Purdue Tango Club, I grew the group from just 2 students to over 40 active members, organized weekly classes, and hosted large events with internationally recognized instructors. More importantly, I focused on creating a welcoming community where students feel connected and supported. For me, tango is more than dance, it’s a way to bring people together, bridge cultures, and balance the intensity of research with creativity and joy.

I also apply these skills in academic leadership. For example, I serve as Local Organizer and Safety Chair for the AAMAS 2025 and 2026 conferences, which has given me hands-on experience managing events, coordinating teams, and creating inclusive spaces for researchers worldwide.

About Zahra

Zahra Ghorrati is a PhD candidate and teaching assistant at Purdue University, specializing in artificial intelligence and time series classification with applications in human activity recognition. She earned her undergraduate degree in Computer Software Engineering and her master’s degree in Artificial Intelligence. Her research focuses on developing scalable and interpretable fuzzy deep learning models for wearable sensor data. She has presented her work at leading international conferences and journals, including AAMAS, PAAMS, FUZZ-IEEE, IEEE Access, System and Applied Soft Computing. She has served as a reviewer for CoDIT, CTDIAC, and IRC, and has been invited to review for AAAI 2026. Zahra also contributes to community building as Local Organizer and Safety Chair for AAMAS 2025 and 2026.
]]> RoboCup Logistics League: an interview with Alexander Ferrein, Till Hofmann and Wataru Uemura https://robohub.org/robocup-logistics-league-an-interview-with-alexander-ferrein-till-hofmann-and-wataru-uemura/ Thu, 25 Sep 2025 11:34:59 +0000 https://aihub.org/?p=18370 The Industrial League arena at RoboCup2025.

RoboCup is an international scientific initiative with the goal of advancing the state of the art of intelligent robots, AI and automation. The annual RoboCup event took place from 15-21 July in Salvador, Brazil. The Logistics League forms part of the Industrial League and is an application-driven league inspired by the industrial scenario of a smart factory. Ahead of the Brazil meeting, we spoke with three key members of the league to find out more. Alexander Ferrein is a RoboCup Trustee overseeing the Industrial League, and Till Hofmann and Wataru Uemura are Logistics League Executive Committee members.

Could you start by giving us an introduction to the Logistics League?

Alexander Ferrein: The idea of the Logistics League is to have robots helping in intra-production logistics. The playing field is set up with different machines and the robots need to bring raw materials and products to the machines and pick products up from them. There are orders coming in for different products of different complexities. The idea is that the robots deliver these products after they have been machined, at a certain handover point, and then the team will be awarded points. The setup is that we have six machines per team, and three robots operating in the smart factory.

There are two teams competing at the same time on different sides of the field. Most of the machines are on the home side of the field, but some machines are also on the opponent’s side of the field. The teams need to show basic robotics skills like navigation skills and collision avoidance. For the production of products we have little discs that have different colours and can be stacked on top of each other, and they have different caps. We have around 550 different items that can be produced. We don’t focus so much on the handling of the parts, therefore the manipulating mechanisms are quite simple and are usually custom built by the teams. They just need to grab these discs and drive them around and put them on conveyor belts on the machines.

The main focus is on the production logistics and the planning phase. The complex products need to be produced or machined by a team of robots – without this planning and team work it wouldn’t be possible to deliver the products within the allotted time in the competition.

Until now, we had been supported by Festo didactics, who supplied the machines. However, they pulled out in February and told us that they won’t support us in bringing the machines to Brazil. Our team in Aachen has a complete field set up, so we are in the (not so easy) process of packing the machines up in pallets and shipping them to Brazil.

Till Hofmann: One important detail is that all the products that need to be manufactured, the orders for those come online, and the number of possible products is very high. Therefore, you can’t do any planning in advance – you can’t just create a big database that contains one sequence that you execute for every possible product, that doesn’t really work. So the robots need to do online planning. Due to the fact that we have multiple robots in the team and then also the opponents teams’ robots on the field, many things go differently than planned, so a very big aspect of the competition is execution monitoring and online replanning. Basically you create an initial plan, but you need to constantly adapt that plan to what actually happens during execution. I just want to stress that in contrast to other RoboCup competitions, it’s really a long-horizon planning task in the sense that we usually need to do actions on a time horizon of five to ten minutes to actually get to an intermediate goal of producing one of those products.

Wataru Uemura: The focus of our league is on how to handle the production line. The three mobile robots are a very important part. At first these were autonomous guided robots, but now they are completely autonomous mobile robots. The robots need to decide on their path to make the product.

Could you talk about some of the challenges the teams face in the league, and is there a challenge or aspect of the competition that the teams have found particularly difficult?

Alexander: First of all, one needs to say that it is a really tough problem that we are facing here. So the teams that are starting new, they have to deal with all of the robotic aspects, so mobile robots, autonomous intelligence systems, they need to drive around, map, interact with the machines. As these items that we are pushing around are non-standard things, they also need to build their own manipulating devices. And then there is this massive planning aspect of the league, which is also not so easy. When we started this in 2011, 2012, we were thinking that this must be a solved problem, that one could use scheduling systems, that production is digital, and that everything would be easy. However, we found out it’s not actually easy, and there are no off-the-shelf solutions for a fleet of robots doing planning and production.

Looking at our team (which has become particularly successful over the years) and from observing the other teams as a trustee, I think that the integration aspect of all the different tasks is really a hard thing. Having a software system that is capable of so many things, communicating with the centralized referee box, and making it run within the time limit of a match, is the major challenge. Today, navigation of a robot is not the big issue, basically, but getting it integrated into your software system and building all the rest around the planning components and so on, this is, from my point of view, the major challenge.

Till: As I mentioned, I think the combination of long-horizon planning and execution monitoring is particularly difficult. We do have a lot of failures during one production run, because of hardware limitations and problems with the robots. Sometimes the machines themselves fail and they need to deal with this, without being able to solve the problem itself, because it’s not in their control. So they need to do a lot of reasoning that considers all the different cases that may happen. For example, suddenly you have a product appearing in a machine and you no longer know the configuration because this information was lost on the way. How do you deal with this? Another example is that the robot drops a piece while it is driving around and then tries to feed it into the machine. Then the machine reports a failure and the team needs to keep its world model up-to-date to know that this product is no longer where they thought it was. How do we deal with this?


Will there be any new challenges introduced for RoboCup 2025?

Alexander: Because the problem we are trying to solve is so hard, we don’t have many teams in the league. We had a new team at RoboCup2023, in Bordeaux, and they are now consistently coming to the RoboCups, which is nice. Apart from this, we have a core of three to five teams that participate in the league. As the challenge itself is hard to get into, the skills only develop slowly, so new challenges are not really introduced. There are slight changes here and there. One of the major changes in recent years was during Coronavirus times where we had to abandon the whole match aspect because we couldn’t play with two teams at the same time. We introduced aspects of the game as challenges. Now we have a challenge track as well, where teams can just focus on certain aspects of the league and do not need to play the full game.

The skills of the teams are not developing in such a way that we really need to add new challenges because it is still unsolved and hard for all the teams to get the robots running, because we have so many different aspects.

I understand that you are thinking about some changes to the league. Could you say more about this?

Alexander: Yes, we have some ideas for a new Industrial League. As Festo are pulling out we don’t see the need to stick with this particular machine type. We had a workshop earlier this year with all the teams and organising committees and we’re proposing a future challenge, or league. We will discuss this at RoboCup2025.

Till: We actually already had discussions with @work, and agreed that the long-term goal is to merge the two competitions into one big Industrial League. Next year, we will start converging by doing some kind of collaboration challenge or crossover challenge where teams from @work will collaborate with what’s now the Logistics League, but will be the Smart Manufacturing League by that time. Then hopefully, in two or three years, this will be one big league rather than separate industrial competitions. And the idea of the league that we’re currently planning to migrate to is really a broader smart manufacturing scenario where we have different aspects of smart manufacturing. So currently it’s really only the production logistics part. But in the future, we also want to include the assembly itself as part of the competition and also extend this to humanoid robots and also focus on human-robot collaboration in this manufacturing setting.

So this will be very different to the Logistics League as it is right now. How we will do the migration from what we have to that new league without losing all the teams is something that’s still in the making.

Alexander: I also don’t think we shouldn’t restrict ourselves to just one type of robot. As we see, there is something going on with respect to humanoid robots, and the Rescue League is proposing a quadruped robot. In the @Home League there are ideas to introduce the staircases where you need more agility in these settings. So for me, I wouldn’t restrict the type of robots that we’re using. We are just thinking about proposing challenges that are looking towards the future. So far we have been doing things that we thought might be relevant to industry, but industry is not very interested in what we are doing here. At least, they are not knocking at our doors and asking what our solutions are. That’s also another aspect that we will possibly discuss at Robocup 2025 – how we could increase our impact as a league for the outside world.

So that’s one of the aims, I guess, to evolve in a way such that industry will be more interested?

Alexander: Yes, be relevant, right? I mean, you see so much going on, in particular in China with the robot manufacturing OEMs [original equipment manufacturers] that build these robots in a few years that have quite impressive capabilities. And well, we stand aside and just watch. So maybe we should use those robots, and integrate them into our process. It’s very important that we are opening our minds to envision a future that’s different from today.

About the interviewees

Alexander Ferrein received his MSc in Computer Science (Dipl.-Inform.) and his PhD (Dr. rer. nat) from Aachen University in 2001 and 2007, respectively. Between 2009-2011 he joined the Robotics and Agents Research Lab, University of Cape Town, as a postdoctoral research fellow with Feodor-Lynen scholarship granted by the Alexander-von-Humboldt Foundation. He then re-joined the Knowledge-Based Systems Group at Aachen University before he became a professor for Robotics and Computer Science at FH Aachen University of Applied Sciences. He is a heading the Mobile Autonomous Systems & Cognitive Robotics Institute at Aachen Applied Science University. His research focusses on the field of Artificial Intelligence and Cognitive Robotics. Since 2015 he is member of the Advisory Committee of the African-German Network of Excellence in Science whose Vice-president he was between 2019-2023. His research concentrates on the field of cognitive robotics. In particular, he is interested in high-level control and decision making of robots and agents acting under real-time constraints.

Till Hofmann is a Postdoc at RWTH Aachen University. His research focuses on planning, plan execution, generalized planning as well as reactive synthesis, with a particular focus on planning for robotics. He was a participant in the RoboCup Logistics League from 2016 until 2019 and member of the technical committee from 2017 until 2020. Since 2024, he is on the executive committee of the RCLL.

Wataru Uemura was born in 1977, and received B.E, M.E. and D.E. degrees from Osaka City University, in 2000, 2002, and 2005. He is an associate professor in Electronics, Information and Communication Engineering Course, Faculty of Advanced Science and Technology, Ryukoku University in Shiga, Japan. He is a member of IEEE, RoboCup and others. He is a chairperson of the RoboCup Japanese National Committee. He is an executive committee member of RoboCup Logistics League. He was a member of the Industrial Robotics Competition Committee, the World Robot Summit. He was TPC Vice Chairs of GCCE 2012, Conference Chair of GCCE 2016, and Publication Chairs of GCCE (Global Conference on Consumer Electronics). He is a member of the World Skills in Japan organizing committee of Autonomous Mobile Robots.
]]> Self-supervised learning for soccer ball detection and beyond: interview with winners of the RoboCup 2025 best paper award https://robohub.org/self-supervised-learning-for-soccer-ball-detection-and-beyond-interview-with-winners-of-the-robocup-2025-best-paper-award/ Fri, 19 Sep 2025 08:56:35 +0000 https://aihub.org/?p=18466 Presentation of the best paper award at the RoboCup 2025 symposium.

An important aspect of autonomous soccer-playing robots concerns accurate detection of the ball. This is the focus of work by Can Lin, Daniele Affinita, Marco Zimmatore, Daniele Nardi, Domenico Bloisi, and Vincenzo Suriani, which won the best paper award at the recent RoboCup symposium. The symposium takes place alongside the annual RoboCup competition, which this year was held in Salvador, Brazil. We caught up with some of the authors to find out more about the work, how their method can be transferred to applications beyond RoboCup, and their future plans for the competition.

Could you start by giving us a brief description of the problem that you were trying to solve in your paper “Self-supervised Feature Extraction for Enhanced Ball Detection on Soccer Robots”?

Daniele Affinita: The main challenge we faced was that deep learning generally requires a large amount of labeled data. This is not a major problem for common tasks that have already been studied, because you can usually find labeled datasets online. But when the task is highly specific, like in RoboCup, you need to collect and label the data yourself. That means gathering the data and manually annotating it before you can even start applying deep learning. This process is not scalable and demands a significant human effort.

The idea behind our paper was to reduce this human effort. We approached the problem through self-supervised learning, which aims to learn useful representations of the data. After all, deep learning is essentially about learning latent representations from the available data.

Could you tell us a bit more about your self-supervised learning framework and how you went about developing it?

Daniele: First of all, let me introduce what self-supervised learning is. It is a way of learning the structure of the data without having access to labels. This is usually done through what we call pretext tasks. These are tasks that don’t require explicit labels, but instead exploit the structure of the data. For example, in our case we worked with images. You can randomly mask some patches and train the model to predict the missing parts. By doing so, the model is forced to learn meaningful features from the data.

In our paper, we enriched the data by using not only raw images but also external guidance. This came from a larger model which we refer to as the teacher. This model was trained on a different task which is more general than the target task we aimed for. This way the larger model can provide guidance (an external signal) that helps the self-supervision to focus more on the specific task we care about.

In our case, we wanted to predict a tight circle around the ball. To guide this, we used an external pretrained model (YOLO) for object detection, which instead predicts a loose bounding box around the ball. We can arguably say that the bounding box, a rectangle, is more general than a circle. So in this sense, we were trying to use external guidance that doesn’t solve exactly the underlying task.

Overview of the data preparation pipeline.

Were you able to test this model out at RoboCup 2025?

Daniele: Yes, we deployed it at RoboCup 2025 and showed great improvements over our previous benchmark, which was the model we used in 2024. In particular, we noticed that the final training requires much less data. The model was also more robust under different lighting conditions. The issue we had with previous models was that they were tailored for specific situations. But of course, all the venues are different, the lighting and the brightness are different, there might be shadows on the field. So it’s really important to have a reliable model and we really noticed a great improvement this year.

What’s your team name, and could you talk a bit about the competition and how it went?

Daniele: So our team is SPQR. We are from Rome, and we have been competing in RoboCup for a long time.

Domenico Blois: We started in 1998, so we are one of the oldest teams in RoboCup.

Daniele: Yeah, I wasn’t even born then! Our team started with the four-legged robots. And then the league shifted more towards biped robots because they are more challenging, they require balance and, overall it’s harder to walk on just two legs.

Our team has grown a lot during recent years. We have been following a very positive trend, going from 9th place in 2019 to third place at the German Open in 2025, and we got 4th place at RoboCup 2025. Our recent success has attracted more students to the team. So it’s kind of a loop – you win more, you attract more students, and you can work more on the challenges proposed by RoboCup.

SPQR team.

Domenico: I want to add that also, from a research point of view, we have won three best paper awards in the last five years, and we have been proposing some new trends towards, for example, the use of LLMs for coding (as a robot’s behaviour generator under the supervision of a human coach). So we are trying to keep the open research field active in our team. We want to win the matches but we also want to solve the research problems that are bound together with the competition.

One of the important contributions of our paper is towards the use of our algorithms outside RoboCup. For example, we are trying to apply the ball detector in precision farming. We want to use the same approach to detect rounded fruits. This is something that is really important for us; to exit the context of Robocup and to use Robocup tools for new approaches in other fields. So if we lose a match, it’s not a big deal for us. We want our students, our team members, to be open minded towards the use of RoboCup as a starting point for understanding teamwork and for understanding how to deal with strict deadlines. This is something that RoboCup can give us. We try to have a team that is ready for every type of challenge, not only within RoboCup, but also other types of AI applications. Winning is not everything for us. We’d prefer to use our own code and not win, than win using code developed by others. This is not optimal for achieving first place, but we want to teach our students to be prepared for the research that is outside of RoboCup.

You said that you’ve previously won two other best paper awards. What did those papers cover?

Domenico: So the last two best papers were kind of visionary papers. In one paper, we wanted to give an insight in how to use the spectators to help the robots score. For example, if you cheer louder, the robots tend to kick the ball. So this is something that is not actually used in the competition now, but is something more towards the 2050 challenge. So we want to imagine how it will be 10 years from now.

The other paper was called “play everywhere”, so you can, for example, play with different types of ball, you can play outside, you can even play without a specific goal, you can play using Coca-Cola cans as goalposts. So the robot has to have a general approach that is not related to the specific field used in RoboCup. This is in contrast to other teams that are very specific. We have a different approach and this is something that makes it harder for us to win the competition. However, we don’t want to win the competition, we want to achieve this goal of having, in 2050, this match between the RoboCup winners and the FIFA World Cup winners.

I’m interested in what you said about transferring the method for ball detection to farming and other applications. Could you say more about that research?

Vincenzo Suriani: Our lab has been involved in some different projects relating to farming applications. The Flourish project ran from 2015 – 2018. More recently, the CANOPIES project has focussed on precision agriculture for permanent crops where farmworkers can efficiently work together with teams of robots to perform agronomic interventions, like harvesting or pruning.

We have another project that is about detecting and harvesting grapes. There is a huge effort in bringing knowledge back from RoboCup to other projects, and vice versa.

Domenico: Our vision now is to focus on the new generation of humanoid robots. We participated in a new event, the World Humanoid Robot Games, held in Beijing in August 2025, because we want to use the platform of RoboCup for other kinds of applications. The idea is to have a single platform with software that is derived from RoboCup code that can be used for other applications. If you have a humanoid robot that needs to move, you can reuse the same code from RoboCup because you can use the same stabilization, the same vision core, the same framework (more or less), and you can just change some modules and you can have a completely different type of application with the same robot with more or less the same code. We want to go towards this idea of reusing code and having RoboCup as a test bed. It is a very tough test bed, but you can use the results in other fields and in other applications.

Looking specifically at RoboCup, what are your future plans for the team? There are some big changes planned for the RoboCup Leagues, so could you also say how this might affect your plans?

Domenico: We have a very strong team and some of the team members will do a PhD in the coming years. One of our targets was to keep the students inside the university and the research ward, and we were successful in this, because now they are very passionate about the RoboCup competition and about AI in general.

In terms of the changes, there will be a new league within RoboCup that is a merger of the standard platform league (SPL) and the humanoid kid-size league. The humanoid adult-size league will remain, so we need to decide whether to join the new merged league, or move to adult-sized robots. At the moment we don’t have too many details, but what we know is that we will go towards a new era of robots. We acquired robots from Booster and we are now acquiring another G1 robot from Unitree. So we are trying to have a complete family of new robots. And then I think we will go towards the league that is chosen by the other teams in the SPL league. But for now we are trying to organize an event in October in Rome with two other teams to exchange ideas and to understand where we want to go. There will also be a workshop to discuss the research side.

Vincenzo: We are also in discussion about the best size of robot for the competition. We are going to have two different positions, because robots are becoming cheaper and there are teams that are pushing to move more quickly to a bigger platform. On the other hand, there are teams that want to stick with a smaller platform in order to do research on multi agents. We have seen a lot of applications for a single robot but not many applications with a set of robots that are cooperating. And this has been historically one of the core parts of research we did in RoboCup, and also outside of RoboCup.

There are plenty of points of view on which robot size to use, because there are several factors, and we don’t know how fast the world will change in two or three years. We are trying to shape the rules and the conditions to play for next year, but, because of how quickly things are changing, we don’t know what the best decision will be. And also the research we are going to do will be affected by the decision we make on this.

There will be some changes to other leagues in the near future too; the small and middle sizes will close in two years probably, and the simulation league also. A lot will happen in the next five years, probably more than during the last 10-15 years. This is a critical year because the decisions are based on what we can see, what we can spot in the future, but we don’t have all the information we need, so it will be challenging.

For example, the SPL has a big, probably the biggest, community among the RoboCup leagues. We have a lot of teams that are grouping by interest and so there are teams that are sticking to working on this specific problem with a specific platform and teams that are trying to move to another platform and another problem. So even inside the same community we are going to have more than one point of view and hopes for the future. At a certain point we will try to figure out what is the best for all of them.

Daniele: I just want to add that in order to achieve the 2050 challenge, in my opinion, it is necessary to have just one league encompassing everything. So up to this point, different leagues have been focusing on different research problems. There were leagues focusing only on strategy, others focusing only on the hardware, our league focusing mainly on the coordination and dynamic handling of the gameplay. But at the end of the day, in order to compete with humans, there must be only one league bringing all these single aspects together. From my point of view, it totally makes sense to keep merging leagues together.

About the authors

Daniele Affinita is a PhD student in Machine Learning at EPFL, specializing in the intersection of Machine Learning and Robotics. He has over four years of experience competing in RoboCup with the SPQR team. In 2024, he worked at Sony on domain adaptation techniques. He holds a Bachelor’s degree in Computer Engineering and a Master’s degree in Artificial Intelligence and Robotics from Sapienza University of Rome.

Vincenzo Suriani earned his Ph.D. in Computer Engineering in 2024 from Sapienza University of Rome, with a specialization in artificial intelligence, robotic vision, and multi-agent coordination. Since 2016, he has served as Software Development Leader of the Sapienza Soccer Robot Team, contributing to major robotic competitions and international initiatives such as EUROBENCH, SciRoc, and Tech4YOU. He is currently a Research Fellow at the University of Basilicata, where he focuses on developing intelligent environments for software testing automation. His research, recognized with award-winning papers at the RoboCup International Symposium (2021, 2023, 2025), centers on robotic semantic mapping, object recognition, and human–robot interaction.

Domenico Daniele Bloisi is an associate professor of Artificial Intelligence at the International University of Rome UNINT. Previously, he was associate professor at the University of Basilicata, assistant professor at the University of Verona, and assistant professor at Sapienza University of Rome. He received his PhD, master’s and bachelor’s degrees in Computer Engineering from Sapienza University of Rome in 2010, 2006 and 2004, respectively. He is the author of more than 80 peer-reviewed papers published in international journals and conferences in the field of artificial intelligence and robotics, with a focus on image analysis, multi-robot coordination, visual perception and information fusion. Dr. Bloisi conducts research in the field of melanoma and oral carcinoma prevention through automatic medical image analysis in collaboration with specialized medical teams in Italy. In addition, Dr. Bloisi is WP3 leader of the EU H2020 SOLARIS project, unit leader for the PRIN PNRR RETINA project, unit leader for the PRIN 2022 AIDA project. Since 2015, he is the team manager of the SPQR robot soccer team participating in the RoboCup world competitions

Can Lin is a master student in Data Science at Sapienza university of Rome. He holds a bachelor degree in Computer science and Artificial intelligence from the same university. He joined the SPQR team in September of 2024, focusing on tasks related to computer vision.

]]> RoboCup@Work League: Interview with Christoph Steup https://robohub.org/robocupwork-league-interview-with-christoph-steup/ Fri, 22 Aug 2025 12:15:27 +0000 https://aihub.org/?p=18234 RoboCup@Work League teams at the event in Brazil.

RoboCup is an international scientific initiative with the goal of advancing the state of the art of intelligent robots, AI and automation. The annual RoboCup event, where teams gather from across the globe to take part in competitions across a number of leagues, this year took place in Salvador, Brazil from 15-21 July. In a series of interviews, we’ve been meeting some of the RoboCup trustees, committee members, and participants, to find out more about their respective leagues. Christoph Steup is an Executive Committee member and oversees the @Work League. Ahead of the event in Brazil, we spoke to Christoph to find out more about the @Work League, the tasks that teams need to complete, and future plans for the League.

Could you start by giving us an introduction to the @Work league?

The @Work League, along with the Logistics League, forms the Industrial League. Our goal is to mimic some of the aspects of industrial production systems. An important aspect of this is factory automization and trying to mimic the factory of the future, where you have autonomous robots building products according to customer design. In these factories of the future, a single piece would be produced individually for each customer. Factories nowadays have big conveyor belts and a lot of automization, with the tasks mostly done in the same way, and you can only build stuff efficiently if you build millions of items. We are working on building individual pieces, where automization is still possible, and even a single piece can be built effectively. But obviously, in our RoboCup competitions, we are not interested in building on a factory scale – we are doing it on a very small scale. That means our robots are typically 80 centimeters long, the largest are around 70 centimeters wide, and some of them are also 80 centimeters high. So let’s say they fit in a one metre cubed box. Also, all our operations are done on the ground. This is just for simplification because building big tables to make it more realistic would also increase the cost for RoboCup and wouldn’t give much additional value.

What our robots need to do is transport objects from different workstations. So we have a default configuration where the arena starts, and there are workstations with objects lying on them, and some of these objects need to be transported to other workstations. The robot needs to do that completely autonomously. So this is one of the special things about the @Work League, that it’s completely autonomous and there is only a single restart allowed per team. So that means the robot really needs to be reliable. One of the big differences between medium teams and very good teams is that the very good teams perform well all the time whereas the medium teams have some good runs and some bad runs.

As well as the object transportation that you mentioned, and there other tasks that the teams need to carry out?

There are some special tasks in our league, like the precision placement task where the robot needs to fit an object into a cavity that is essentially the same shape and size as the object. It’s a little bit like the game that babies do to train their dexterity.

We also have a task that is inspired by a conveyor belt, but we are using a table that is constantly turning. The robots need to grasp stuff while the table is turning. This looks a little bit silly because no one would actually put a rotating table in a factory, however this is our way of actually mimicking a conveyor belt. The conveyor belt itself would be really, really difficult to integrate into the competition, so we just abstracted that and use this rotating table to actually have the same challenge but in a more manageable way. And it’s still a very, very hard challenge.

Then there are some special challenges that we integrate. For example, that robots need to report their state back so that we can observe what the robot is doing. We also have a challenge where humans are in the loop. For example, the robot brings pieces to a certain workstation where a human is present, the human assembles the pieces, and then the human needs to give a sign to the robot, and then the robot will take the piece away and put it somewhere else. This is designed to really mimic the automated factory flow.

In the past we also had a challenge where the robots needed to open a drawer, take something out and then close the drawer again. We’ve also had tasks where the robot has to handle fragile objects, like sweets, where the robot really needed to be careful in manipulating them. So in general, what differentiates us most from the Logistics League is that we are focusing a lot on manipulation and all the difficulties that come with manipulation and unknown objects, whereas Logistics is more tailored towards large-scale logistics processes with all their optimization and planning.

I was lucky enough to attend RoboCup last year in Eindhoven and what the teams were doing was really impressive. It was also interesting to see how varied the robots were, and how teams were approaching the tasks in distinct ways, with different grabbers and so on.

Yes, this difference in approaches is related to the history of our League, which is a little bit similar to the Logistics League. The Logistics League was originally a sponsored demonstration by Festo, which is a large company from Germany that creates tools, but they also have a didactics area where they provide tools to help people understand factory optimization. The @Work League was sponsored by Kuka, the robotics company, and, in the beginning, they required every team to compete with the Kuka youBot. So this was pretty much the default platform for our league, but at some point Kuka dropped from a sponsor to just an advisor to the league, and nowadays they are not part of the league at all. So when the Kuka youBot was going out of commission, the teams searched for alternatives and now we are presented with a wide variety of robots that are competing in the league, which I personally find really cool. Now we have all these different robots, all these different approaches, and some work better in some scenarios and worse in others. So we really have a scientific approach to the problem and we are really getting some insights into how you can tackle this problem on multiple levels.

Have you noticed that some of the challenges specifically are more difficult in general for all the teams?

In 2018 we introduced a challenge of so-called arbitrary surfaces which are surfaces unknown to the teams that are put on top of the workstations. The teams need to be able to deal with these surfaces. There are two surfaces that are really, really awful for the teams – one is grass that we pretty much stole from the soccer competitions! We just thought it would be funny to try it, and it was a really interesting problem, especially for some of the grippers of the different teams. For example, the current world champion, they have a rigid gripper so they can have force feedback when they grasp. However, the grass is really difficult for them – because of their rigidity, they always grasp the grass itself and then they pull up the grass with the object. And this led to some interesting problems, like they are transporting the actual surface around and not only the object. This isn’t a problem for teams using a flexible gripper. However, on the other hand, the flexible gripper makes it really difficult to assess if you have grasped the object because you have very bad force feedback. So there are two different approaches that have their pros and cons in different scenarios.

Three robots from the @Work competition in Brazil.

Are you introducing any new tasks for this year?

Yes and no. So actually we are introducing a completely new challenge which is different from what we’ve done before. The new challenge is the so-called smart farming challenge, which is opening our league to a whole new field of applications, because we are now looking at agriculture. We are working on this with Studica, a robotics company from Canada, whose hardware we are using. We’ve already given it a try at the German Open. This challenge comes with some new specifics and one these is that the teams only get the robot shortly before the competition. So they don’t really know the robot largely in advance, and they need to assemble, program and design the robot in a very short time. To compensate for this, we reduce the amount of optimization and robustness that is necessary. Because it’s an agriculture setting, we have different objects, like fruits, that the teams need to handle. This makes it a little bit more complicated because fruits have a more arbitrary shape and different levels of ripeness that need to be detected. We also have some grapes that are hanging on a wall, which is a completely different kind of manipulation task than before, because before the teams just needed to grasp things from surfaces, but now they really need to pluck stuff from a wall in a reliable way. So this is the new challenge. It also comes with a lot of software challenges because the computational power of this robot is very limited because it only has a Raspberry Pi. For example, there is not a lot of image processing possible, especially compared to the current robots, some of which even have GPUs embedded.

Is there a particular part of the hardware or software that you’ve seen some of the biggest developments in over the last year or so?

Yeah, I think one big change I observed over the years was a switch from custom neural networks for object detection to off-the-shelf components. So pretty much all teams nowadays use YOLO networks, which you can get pre-trained and they just deploy them on GPUs that they embedded into their robots. This is also one of the reasons why the robots really grew in size over the last few years because they needed space for the larger computational power. This actually made it possible for a lot of teams to reliably detect the objects. Object detection was a big problem in the beginning of the league and nowadays it’s not really a big issue – most teams are really good at that. Sometimes they are a little bit startled with decoy objects – these are objects in the arena that are not really part of the task, and they are unknown to the teams beforehand. Sometimes they are, let’s say, evil decoys that look like an object and there’s some mismatches that the teams do, but this is becoming very rare.

I think the second big change is a switch to larger manipulators with more degrees of freedom. So in the beginning, everyone had a very small manipulator with only five degrees of freedom, which limited the operating range, and nowadays pretty much all teams have a six degree of freedom manipulator with a large range. This means that they don’t need to move their robot when they are in front of a workstation, which makes them much faster and also much more precise.

Could you talk about the future plans for the League?

There are a few things we are thinking about.

With regards to the competition itself, we had a discussion with the teams about what they are interested in doing in the future. Two things came up that they really want to have. One is mobile obstacles, so they want other objects to move autonomously through the arena. We’re in the process of creating that, in cooperation with EduArt, which is a company from Germany that also provides small educational robots. And the second thing we want to introduce is a kind of humanoid robot that the teams can use to actually handle special manipulation tasks that cannot be done simply with a robot manipulator.

In terms of creating an entry-level League, we have been working on this and one potential idea is to use the smart-farming challenge as the entry point. Through the collaboration with Studica, we can provide teams with the robot and they get to keep it after the competition. At the German Open I spoke to the Rapidly-Manufactured Rescue League about this crossover between the Rescue and the Junior Leagues, and they are very keen to collaborate.

We’re also talking with the Logistics League. Their sponsor, Festo, has dropped out of their league and now they need to reorganize. We are wondering if it would be worthwhile to bring our leagues closer together, or even fuse them together to a single RoboCup Industrial league. The Logistics League wants to do more manipulation, and the @Work League wants to do more planning, so we are closing the gap naturally between the two. However, this is just a thought at the moment – we need to see how the teams react to that.

About Christoph

Christoph Steup is an active researcher specializing in various fields of robotics, including swarm robotics, precision farming, and weather-resilient autonomous driving. He currently works at the Fraunhofer Institute for Transportation and Infrastructure Systems (IVI), where he leads the Swarm Technology Group. Prior to this role, he headed the Computational Intelligence in Robotics group at the Otto von Guericke University Magdeburg. Christoph’s involvement with RoboCup began in 2015 when he joined the robOTTO team of Otto von Guericke University as team leader. His contributions to the RoboCup community expanded as he became a member of the Technical Committee for the @Work League in 2017. In 2019, he further advanced his participation by joining the Executive Committee of the league.
]]> Interview with Haimin Hu: Game-theoretic integration of safety, interaction and learning for human-centered autonomy https://robohub.org/interview-with-haimin-hu-game-theoretic-integration-of-safety-interaction-and-learning-for-human-centered-autonomy/ Thu, 21 Aug 2025 07:54:56 +0000 https://aihub.org/?p=18110

In this interview series, we’re meeting some of the AAAI/SIGAI Doctoral Consortium participants to find out more about their research. In this latest interview, Haimin Hu tells us about his research on the algorithmic foundations of human-centered autonomy and his plans for future projects, and gives some advice for PhD students looking to take the next step in their career.

Could you give us an overview of the research you carried out during your PhD?

My PhD research, conducted under the supervision of Professor Jaime Fernández Fisac in the Princeton Safe Robotics Lab, focuses on the algorithmic foundations of human-centered autonomy. By integrating dynamic game theory with machine learning and safety-critical control, my work aims to ensure autonomous systems, from self-driving vehicles to drones and quadrupedal robots, are performant, verifiable, and trustworthy when deployed in human-populated space. The core principle of my PhD research is to plan robots’ motion in the joint space of both physical and information states, actively ensuring safety as they navigate uncertain, changing environments and interact with humans. Its key contribution is a unified algorithmic framework—backed by game theory—that allows robots to safely interact with their human peers, adapt to human preferences and goals, and even help humans refine their skills. Specifically, my PhD work contributes to the following areas in human-centered autonomy and multi-agent systems:

  • Trustworthy human–robot interaction: Planning safe and efficient robot trajectories by closing the computation loop between physical human-robot interaction and runtime learning that reduces the robot’s uncertainty about the human.
  • Verifiable neural safety analysis for complex robotic systems: Learning robust neural controllers for robots with high-dimensional dynamics; guaranteeing their training-time convergence and deployment-time safety.
  • Scalable interactive planning under uncertainty: Synthesizing game-theoretic control policies for complex and uncertain human–robot systems at scale.

Was there a project (or aspect of your research) that was particularly interesting?

Safety in human-robot interaction is especially difficult to define, because it hinges on an, I’d say, almost unanswerable question: How safe is safe enough when humans might behave in arbitrary ways? To give a concrete example: Is it sufficient if an autonomous vehicle can avoid hitting a fallen cyclist 99.9% of the time? What if this rate can only be achieved by the vehicle always stopping and waiting for the human to move out of the way?

I would argue that, for trustworthy deployment of robots in human-populated space, we need to complement standard statistical methods with clear-cut robust safety assurances under a vetted set of operation conditions as well established as those of bridges, power plants, and elevators. We need runtime learning to minimize the robot’s performance loss caused by safety-enforcing maneuvers; this calls for algorithms that can reduce the robot’s inherent uncertainty induced by its human peers, for example, their intent (does a human driver want to merge, cut behind, or stay in the lane?) or response (if the robot comes closer, how will the human react?). We need to close the loop between the robot’s learning and decision-making so that it can optimize efficiency by anticipating how its ongoing interaction with the human may affect the evolving uncertainty, and ultimately, its long-term performance.

What made you want to study AI, and the area of human-centered robotic systems in particular?

I’ve been fascinated by robotics and intelligent systems since childhood, when I’d spend entire days watching sci-fi anime like Mobile Suit Gundam, Neon Genesis Evangelion, or Future GPX Cyber Formula. What captivated me wasn’t just the futuristic technology, but the vision of AI as a true partner—augmenting human abilities rather than replacing them. Cyber Formula in particular planted the idea of human-AI co-evolution in my mind: an AI co-pilot that not only helps a human driver navigate high-speed, high-stakes environments, but also adapts to the driver’s style over time, ultimately making the human a better racer and deepening mutual trust along the way. Today, during my collaboration with Toyota Research Institute (TRI), I work on human-centered robotics systems that embody this principle: designing AI systems that collaborate with people in dynamic, safety-critical settings by rapidly aligning with human intent through multimodal inputs, from physical assistance to visual cues and language feedback, bringing to life the very ideas that once lived in my childhood imagination.

You’ve landed a faculty position at Johns Hopkins University (JHU) – congratulations! Could you talk a bit about the process of job searching, and perhaps share some advice and insights for PhD students who may be at a similar stage in their career?

The job search was definitely intense but also deeply rewarding. My advice to PhD students: start thinking early about the kind of long-term impact you want to make, and act early on your application package and job talk. Also, make sure you talk to people, especially your senior colleagues and peers on the job market. I personally benefited a lot from the following resources:

Do you have an idea of the research projects you’ll be working on at JHU?

I wish to help create a future where humans can unquestionably embrace the presence of robots around them. Towards this vision, my lab at JHU will investigate the following topics:

  • Uncertainty-aware interactive motion planning: How can robots plan safe and efficient motion by accounting for their evolving uncertainty, as well as their ability to reduce it through future interaction, sensing, communication, and learning?
  • Human–AI co-evolution and co-adaptation: How can embodied AI systems learn from human teammates while helping them refine existing skills and acquire new ones in a safe, personalized manner?
  • Safe human-compatible autonomy: How can autonomous systems ensure prescribed safety while remaining aligned with human values and attuned to human cognitive limitations?
  • Scalable and generalizable strategic decision-making: How can multi-robot systems make safe, coordinated decisions in dynamic, human-populated environments?

How was the experience attending the AAAI Doctoral Consortium?

I had the privilege of attending the 2025 AAAI Doctoral Consortium, and it was an incredibly valuable experience. I’m especially grateful to the organizers for curating such a thoughtful and supportive environment for early-career researchers. The highlight for me was the mentoring session with Dr Ming Yin (postdoc at Princeton, now faculty at Georgia Tech CSE), whose insights on navigating the uncertain and competitive job market were both encouraging and eye-opening.

Could you tell us an interesting (non-AI related) fact about you?

I am passionate about skiing. I learned to ski primarily by vision-based imitation learning from a chairlift, though I’m definitely paying the price now for poor generalization! One day, I hope to build an exoskeleton that teaches me to ski better while keeping me safe on the double black diamonds.

About Haimin

Haimin Hu is an incoming Assistant Professor of Computer Science at Johns Hopkins University, where he is also a member of the Data Science and AI Institute, the Institute for Assured Autonomy, and the Laboratory for Computational Sensing and Robotics. His research focuses on the algorithmic foundations of human-centered autonomy. He has received several awards and recognitions, including a 2025 Robotics: Science and Systems Pioneer, a 2025 Cyber-Physical Systems Rising Star, and a 2024 Human-Robot Interaction Pioneer. Additionally, he has served as an Associate Editor for IEEE Robotics and Automation Letters since his fourth year as a PhD student. He obtained a PhD in Electrical and Computer Engineering from Princeton University in 2025, an MSE in Electrical Engineering from the University of Pennsylvania in 2020, and a BE in Electronic and Information Engineering from ShanghaiTech University in 2018.
]]> Interview with Kate Candon: Leveraging explicit and implicit feedback in human-robot interactions https://robohub.org/interview-with-kate-candon-leveraging-explicit-and-implicit-feedback-in-human-robot-interactions/ Fri, 25 Jul 2025 07:36:28 +0000 https://aihub.org/?p=17886

In this interview series, we’re meeting some of the AAAI/SIGAI Doctoral Consortium participants to find out more about their research. Kate Candon is a PhD student at Yale University interested in understanding how we can create interactive agents that are more effectively able to help people. We spoke to Kate to find out more about how she is leveraging explicit and implicit feedback in human-robot interactions.

Could you start by giving us a quick introduction to the topic of your research?

I study human-robot interaction. Specifically I’m interested in how we can get robots to better learn from humans in the way that they naturally teach. Typically, a lot of work in robot learning is with a human teacher who is only tasked with giving explicit feedback to the robot, but they’re not necessarily engaged in the task. So, for example, you might have a button for “good job” and “bad job”. But we know that humans give a lot of other signals, things like facial expressions and reactions to what the robot’s doing, maybe gestures like scratching their head. It could even be something like moving an object to the side that a robot hands them – that’s implicitly saying that that was the wrong thing to hand them at that time, because they’re not using it right now. Those implicit cues are trickier, they need interpretation. However, they are a way to get additional information without adding any burden to the human user. In the past, I’ve looked at these two streams (implicit and explicit feedback) separately, but my current and future research is about combining them together. Right now, we have a framework, which we are working on improving, where we can combine the implicit and explicit feedback.

In terms of picking up on the implicit feedback, how are you doing that, what’s the mechanism? Because it sounds incredibly difficult.

It can be really hard to interpret implicit cues. People will respond differently, from person to person, culture to culture, etc. And so it’s hard to know exactly which facial reaction means good versus which facial reaction means bad.

So right now, the first version of our framework is just using human actions. Seeing what the human is doing in the task can give clues about what the robot should do. They have different action spaces, but we can find an abstraction so that we can know that if a human does an action, what the similar actions would be that the robot can do. That’s the implicit feedback right now. And then, this summer, we want to extend that to using visual cues and looking at facial reactions and gestures.

So what kind of scenarios have you been kind of testing it on?

For our current project, we use a pizza making setup. Personally I really like cooking as an example because it’s a setting where it’s easy to imagine why these things would matter. I also like that cooking has this element of recipes and there is a formula, but there’s also room for personal preferences. For example, somebody likes to put their cheese on top of the pizza, so it gets really crispy, whereas other people like to put it under the meat and veggies, so that maybe it is more melty instead of crispy. Or even, some people clean up as they go versus others who wait until the end to deal with all the dishes. Another thing that I’m really excited about is that cooking can be social. Right now, we’re just working in dyadic human-robot interactions where it’s one person and one robot, but another extension that we want to work on in the coming year is extending this to group interactions. So if we have multiple people, maybe the robot can learn not only from the person reacting to the robot, but also learn from a person reacting to another person and extrapolating what that might mean for them in the collaboration.

Could you say a bit about how the work that you did earlier in your PhD has led you to this point?

When I first started my PhD, I was really interested in implicit feedback. And I thought that I wanted to focus on learning only from implicit feedback. One of my current lab mates was focused on the EMPATHIC framework, and was looking into learning from implicit human feedback, and I really liked that work and thought it was the direction that I wanted to go into.

However, that first summer of my PhD it was during COVID and so we couldn’t really have people come into the lab to interact with robots. And so instead I did an online study where I had people play a game with a robot. We recorded their face while they were playing the game, and then we tried to see if we could predict based on just facial reactions, gaze, and head orientation if we could predict what behaviors they preferred for the agent that they were playing with in the game. We actually found that we could decently well predict which of the behaviors they preferred.

The thing that was really cool was we found how much context matters. And I think this is something that is really important for going from just a solely teacher-learner paradigm to a collaboration – context really matters. What we found is that sometimes people would have really big reactions but it wasn’t necessarily to what the agent was doing, it was to something that they had done in the game. For example, there’s this clip that I always use in talks about this. This person’s playing and she has this really noticeably confused, upset look. And so at first you might think that’s negative feedback, whatever the robot did, the robot shouldn’t have done that. But if you actually look at the context, we see that it was the first time that she lost a life in this game. For the game we made a multiplayer version of Space Invaders, and she got hit by one of the aliens and her spaceship disappeared. And so based on the context, when a human looks at that, we actually say she was just confused about what happened to her. We want to filter that out and not actually consider that when reasoning about the human’s behavior. I think that was really exciting. After that, we realized that using implicit feedback only was just so hard. That’s why I’ve taken this pivot, and now I’m more interested in combining the implicit and explicit feedback together.

You mentioned the explicit element would be more binary, like good feedback, bad feedback. Would the person-in-the-loop press a button or would the feedback be given through speech?

Right now we just have a button for good job, bad job. In an HRI paper we looked at explicit feedback only. We had the same space invaders game, but we had people come into the lab and we had a little Nao robot, a little humanoid robot, sitting on the table next to them playing the game. We made it so that the person could give positive or negative feedback during the game to the robot so that it would hopefully learn better helping behavior in the collaboration. But we found that people wouldn’t actually give that much feedback because they were focused on just trying to play the game.

And so in this work we looked at whether there are different ways we can remind the person to give feedback. You don’t want to be doing it all the time because it’ll annoy the person and maybe make them worse at the game if you’re distracting them. And also you don’t necessarily always want feedback, you just want it at useful points. The two conditions we looked at were: 1) should the robot remind someone to give feedback before or after they try a new behavior? 2) should they use an “I” versus “we” framing? For example, “remember to give feedback so I can be a better teammate” versus “remember to give feedback so we can be a better team”, things like that. And we found that the “we” framing didn’t actually make people give more feedback, but it made them feel better about the feedback they gave. They felt like it was more helpful, kind of a camaraderie building. And that was only explicit feedback, but we want to see now if we combine that with a reaction from someone, maybe that point would be a good time to ask for that explicit feedback.

You’ve already touched on this but could you tell us about the future steps you have planned for the project?

The big thing motivating a lot of my work is that I want to make it easier for robots to adapt to humans with these subjective preferences. I think in terms of objective things, like being able to pick something up and move it from here to here, we’ll get to a point where robots are pretty good. But it’s these subjective preferences that are exciting. For example, I love to cook, and so I want the robot to not do too much, just to maybe do my dishes whilst I’m cooking. But someone who hates to cook might want the robot to do all of the cooking. Those are things that, even if you have the perfect robot, it can’t necessarily know those things. And so it has to be able to adapt. And a lot of the current preference learning work is so data hungry that you have to interact with it tons and tons of times for it to be able to learn. And I just don’t think that that’s realistic for people to actually have a robot in the home. If after three days you’re still telling it “no, when you help me clean up the living room, the blankets go on the couch not the chair” or something, you’re going to stop using the robot. I’m hoping that this combination of explicit and implicit feedback will help it be more naturalistic. You don’t have to necessarily know exactly the right way to give explicit feedback to get the robot to do what you want it to do. Hopefully through all of these different signals, the robot will be able to hone in a little bit faster.

I think a big future step (that is not necessarily in the near future) is incorporating language. It’s very exciting with how large language models have gotten so much better, but also there’s a lot of interesting questions. Up until now, I haven’t really included natural language. Part of it is because I’m not fully sure where it fits in the implicit versus explicit delineation. On the one hand, you can say “good job robot”, but the way you say it can mean different things – the tone is very important. For example, if you say it with a sarcastic tone, it doesn’t necessarily mean that the robot actually did a good job. So, language doesn’t fit neatly into one of the buckets, and I’m interested in future work to think more about that. I think it’s a super rich space, and it’s a way for humans to be much more granular and specific in their feedback in a natural way.

What was it that inspired you to go into this area then?

Honestly, it was a little accidental. I studied math and computer science in undergrad. After that, I worked in consulting for a couple of years and then in the public healthcare sector, for the Massachusetts Medicaid office. I decided I wanted to go back to academia and to get into AI. At the time, I wanted to combine AI with healthcare, so I was initially thinking about clinical machine learning. I’m at Yale, and there was only one person at the time doing that, so I was looking at the rest of the department and then I found Scaz (Brian Scassellati) who does a lot of work with robots for people with autism and is now moving more into robots for people with behavioral health challenges, things like dementia or anxiety. I thought his work was super interesting. I didn’t even realize that that kind of work was an option. He was working with Marynel Vázquez, a professor at Yale who was also doing human-robot interaction. She didn’t have any healthcare projects, but I interviewed with her and the questions that she was thinking about were exactly what I wanted to work on. I also really wanted to work with her. So, I accidentally stumbled into it, but I feel very grateful because I think it’s a way better fit for me than the clinical machine learning would have necessarily been. It combines a lot of what I’m interested in, and I also feel it allows me to flex back and forth between the mathy, more technical work, but then there’s also the human element, which is also super interesting and exciting to me.

Have you got any advice you’d give to someone thinking of doing a PhD in the field? Your perspective will be particularly interesting because you’ve worked outside of academia and then come back to start your PhD.

One thing is that, I mean it’s kind of cliche, but it’s not too late to start. I was hesitant because I’d been out of the field for a while, but I think if you can find the right mentor, it can be a really good experience. I think the biggest thing is finding a good advisor who you think is working on interesting questions, but also someone that you want to learn from. I feel very lucky with Marynel, she’s been a fabulous advisor. I’ve worked pretty closely with Scaz as well and they both foster this excitement about the work, but also care about me as a person. I’m not just a cog in the research machine.

The other thing I’d say is to find a lab where you have flexibility if your interests change, because it is a long time to be working on a set of projects.

For our final question, have you got an interesting non-AI related fact about you?

My main summertime hobby is playing golf. My whole family is into it – for my grandma’s 100th birthday party we had a family golf outing where we had about 40 of us golfing. And actually, that summer, when my grandma was 99, she had a par on one of the par threes – she’s my golfing role model!

About Kate

Kate Candon is a PhD candidate at Yale University in the Computer Science Department, advised by Professor Marynel Vázquez. She studies human-robot interaction, and is particularly interested in enabling robots to better learn from natural human feedback so that they can become better collaborators. She was selected for the AAMAS Doctoral Consortium in 2023 and HRI Pioneers in 2024. Before starting in human-robot interaction, she received her B.S. in Mathematics with Computer Science from MIT and then worked in consulting and in government healthcare.
]]> Tackling the 3D Simulation League: an interview with Klaus Dorer and Stefan Glaser https://robohub.org/tackling-the-3d-simulation-league-an-interview-with-klaus-dorer-and-stefan-glaser/ Tue, 15 Jul 2025 14:26:05 +0000 https://aihub.org/?p=17971 A screenshot from the new simulator that will be trialled for a special challenge at RoboCup2025.

The annual RoboCup event, where teams gather from across the globe to take part in competitions across a number of leagues, will this year take place in Brazil, from 15-21 July. In advance of kick-off, we spoke to two members of the RoboCup Soccer 3D Simulation League: Executive Committee Member Klaus Dorer, and Stefan Glaser, who is on the Maintenance Committee and who has been recently developing a new simulator for the League.

Could start by just giving us a quick introduction to the Simulation League?
Klaus Dorer: There are two Simulation Leagues in Soccer: the 2D Simulation League and the 3D Simulation League. The 2D Simulation League, as the name suggests, is a flat league where the players and ball are simulated with simplified physics and the main focus is on team strategy. The 3D Simulation League is much closer to real robots; it simulates 11 versus 11 Nao robots. The level of control is like with real robots, where you move each motor of the legs and the arms and so on to achieve movement.

I understand that you have been working on a new simulator for the 3D League. What was the idea behind this new simulator?
Klaus: The aim is to bring us closer to the hardware leagues so that the simulator can be more useful. The current simulator that we use in the 3D Simulation League is called SimSpark. It was created in the early 2000s with the aim of making it possible to play 11 vs 11 players. With the hardware constraints of that time, there had to be some compromises on the physics to be able to simulate 22 players at the same time. So the simulation is physically somewhat realistic, but not in the sense that it’s easy to transpose it to a real Nao robot.

Stefan Glaser: The idea for developing a new simulator has been around for a few years. SimSpark is a very powerful simulation framework. The base framework is domain independent (not soccer specific) and specific simulations are realized via plugins. It supports multiple physics engines in the backend and provides a flexible scripting interface for configuration and adaptations of the simulation. However, all this flexibility comes with the price of complexity. In addition to that, SimSpark uses custom robot model specifications and communication protocols, limiting the amount of available robot models and requiring teams to develop custom communication layers only for communicating with SimSpark. As a result of this, SimSpark has not been widely adopted in the RoboCup community.

With the new simulator, I would like to address these two major issues: complexity and standardization. In the ML community, the MuJoCo physics engine has become a very popular choice for learning environments after Google DeepMind acquired it and released it open source. Its standards for world and robot model specifications are widely adopted in the community and there exist a lot of ready-to-use robot model specifications for a wide variety of virtual as well as real-world robots. In the middle of last year, they (MuJoCo) added a feature which allows you to manipulate the world representation during simulation (adding and removing objects to / from the simulation while preserving the simulation state). This is one essential requirement we have in the simulation league, where we start with an empty field and then the agents connect on demand and form the teams. When this feature has been added, I decided to make a step forward and try to implement a new simulator for the 3D Simulation League based on MuJoCo. Initially, I wanted to start development in C/C++ to achieve maximum performance, but then decided to start in Python to reduce complexity and make it more accessible for other developers. I started development on Easter Monday so it’s not even three months old!

I think it might be useful to explain a little bit more about the setup of our league and the requirements of the simulator. If we take the FIFA game (on your favorite gaming device) as an example, there is one simulation happening which simulates 22 players and the decision making is part of the simulation having full access to the state of the world. In the 3D Simulation League we have two teams with 11 robots on the field, but we also have 22 individual agent softwares which are connected to the simulation server, each controlling one single robot. Each connected agent only receives sensor information related to their robot in the simulation. They are also only allowed to communicate via the server – there is no direct communication between the agents allowed in Simulation League. So we have a general setup where the simulation server has to be able to accept up to 22 connections and manage the situation there. This functionality has been the major focus for me for the last couple of months and this part is already working well. Teams can connect their agents, which will receive sensor information and can actuate joints of the robot in the simulation and so on. They are also able to select different robot models if they like.

An illustration of the simulator set-up.

Presumably the new simulator has a better representation of the physics of a real robot.
Klaus: Exactly. For example, how the motors are controlled is now a bit different and much closer to real robots. So when I did my first experiments, I saw the robot collapse and I thought it was exactly how a real robot would collapse! In SimSpark we also had falling robots but the motor control in the new simulator is different. Now you can control the motors by speed, by force, by position, which is much more flexible – it’s closer to what we know from real robots.

I think that, at least initially, it will be more difficult for the Simulation League teams to get the robots to do what they want them to do, because it’s more realistic. For example, in SimSpark the ground contact was much more forgiving. So if you step hard on the ground, you don’t fall immediately with a SimSpark robot but with a MuJoCo robot this will be much more realistic. Indeed, in real robots ground contact is somewhat less forgiving.

I had a question about the vision aspect – how do the individual agents “see” the position of the other agents on the field?
Stefan: We simulate a virtual vision pipeline on the server side. You have a restricted field of view of ±60° horizontally and vertically. Within that field of view you will detect the head, the arms, the feet of other players, or the ball, for example, or different features of the field. Similar to common real-world vision pipelines, each detection consists of a label, a direction vector and the distance information. The information has some noise on it like real robots have, too, but teams don’t need to process camera images. They get the detections directly from the simulation server.

We’ve previously had a discussion about moving towards getting camera images of the simulation to integrate into the vision pipeline on the agent side. This was never really realistic in SimSpark with the implementation we had there. However, it should be possible with MuJoCo. However, for the first version, I used the same way the traditional simulator treated the vision. This means that teams don’t need to train a vision model, and don’t need to handle camera images to get started. This reduces the load significantly and also shifts the focus of the problem towards motion and decision making.

Will the simulator be used at RoboCup 2025?
Stefan: We plan to have a challenge with a new simulator and I will try to provide some demo games. At the moment it’s not really in a state where you can play a whole competition.

Klaus: That’s usually how we proceed with new simulators. We would not move from one to the other without any intermediate step. We will have a challenge this year at RoboCup 2025 with the new MuJoCo simulator where each participating team will try to teach the robot to kick as far as possible. So, we will not be playing a whole game, we won’t have multiple robots, just a single robot stepping in front of the ball and kicking the ball. That’s the technical challenge for this year. Teams will get an idea of how the simulator works, and we’ll get an idea of what has to be changed in the simulator to proceed.

This new challenge will be voluntary, so we are not sure how many teams will participate. Our team (MagmaOffenburg) will certainly take part. It will be interesting to see how well the teams perform because no one knows how far a good kick is in this simulator. It’s a bit like in Formula One when the rules change and no one knows which team will be the leading team.

Do you have an idea of how much adaptation teams will have to make if and when you move to the new simulator for the full matches?
Stefan: As a long-term member of 3D Simulation League, I know the old simulator SimSpark pretty well, and know the protocols involved and how the processes work. So the first version of the new simulator is designed to use the same basic protocol, the same sensor information, and so on. The idea is that the teams can use the new simulator with minimal effort in adapting their current agent software. So they should be able to get started pretty fast.

Although, when designing a new platform, I would like to take the opportunity to make a step forward in terms of protocols, because I also want to integrate other Leagues in the long-term. They usually have other control mechanisms, and they don’t use the same protocol that is prominent in 3D Simulation. Therefore there has to be some flexibility in the future. But for the first version, the idea was to get the Simulation League ready with minimal effort.

Klaus: The big idea is that this is not just used in the 3D Simulation league, but also as a useful simulator for the Humanoid League and also for the Standard Platform League (SPL). So if that turns out to be true, then it will be completely successful. For the Kick Challenge this year, for example, we use a T1 robot that is a Humanoid League robot.

Could you say something about this simulation to real world (Sim2Real) aspect?
Stefan: We’d like it to be possible for the motions and behaviors in the simulator to be ported to real robots. From my point of view, it would be useful the other way round too.

We, as a Simulation League, usually develop for the Simulation League and therefore would like to get the behaviors running on a real robot. But the hardware teams usually have a similar issue when they want to test high-level decision making. They might have two to five robots on the field, and if they want to play a high-level decision-making match and train in that regard, they always have to deploy a lot of robots. If they also want to have an opponent, they have to double the amount of robots in order to play a game to see how the strategy would turn out. The Sim2Real aspect is also interesting for these teams, because they should be able to take what they deployed on the real robot and it should also work in the simulation. They can then use the simulation to train high-level skills like team play, player positioning and so on, which is a challenging aspect for the real robot leagues like SPL or the Humanoid Leagues.

Klaus: And the reason we know this is because we have a team in the Simulation League and we have a team in the Humanoid League. So that’s another reason why we are keen to bring these things closer together.

How does the refereeing work in the Simulation League?
Klaus: A nice thing about Simulation Leagues is that there is a program which knows the real state of the world so we can build in the referee inside the simulator and it will not fail. For things like offside, whether the ball passed the goal line, that’s fail safe. All the referee decisions are taken by the system itself. We have a human referee but they never need to intervene. However, there are situations where we would like artificial intelligence to play a role. This is not currently the case in SimSpark because the rules are all hard coded. We have a lot of fouls that are debatable. For example, there are many fouls that teams agree should not have been a foul, and other fouls that are not called that should have been. It would be a nice AI learning task to get some situations judged by human referees and then train an AI model to better determine the rules for what is a foul and what isn’t a foul. But this is currently not the case.

Stefan: On the new simulator I am not that far into the development that I have implemented the automatic referee yet. I have some basic set of rules which progress the game as such, but judging fouls and deciding on special situations is not yet implemented in the new simulator.

What are the next steps for developing the simulator?
Stefan: One of the next major steps will be to refine the physics simulation. For instance, even though there exists a ball in the simulation, it is not yet really well refined. There are a lot of physics parameters which we have to decide on to reflect the real world as good as possible. This will likely require a series of experiments in order to get to the correct values for various aspects. In this aspect I’m hoping for some engagement of the community, as it is a great research opportunity and I personally would prefer the community to decide on a commonly accepted parameter set based on a level of evidence that I can’t easily provide all by myself. So in case someone is interested in refining the physics of the simulation such that it best reflects the real world, you are welcome to join!

Another major next step will be the development of the automatic referee of the soccer simulation, deciding on fouls, handling misbehaving agents and so on. In the first version, foul conditions will likely be judged by an expert system specifically designed for this purpose. The simulation league has developed a set of foul condition specifications which I plan to adapt. In a second step, I would like to integrate and support the development of AI based foul detection models. But yeah, one step after the other.

What are you particularly looking forward to at RoboCup2025?
Klaus: Well, with our team we have been vice world champion seven times in a row. This year we are really hoping to make it to world champion. We are very experienced in getting losses in finals and this year we are looking forward to changing that, from a team perspective.

Stefan: I’m going to Brazil in order to promote the simulator, not just for the Simulation League, but also across the boundaries for the Humanoid Leagues and the SPL Leagues. I think that this simulator is a great chance to bring people from all the leagues together. I’m particularly interested in the specific requirements of all the teams of the different leagues. This understanding will help me tailor the new simulator towards their needs. This is one of my major highlights for this year, I would say.

You can find out more about the new simulator at the project webpage, and from the documentation.

Klaus Dorer is professor for artificial intelligence, autonomous systems and software engineering at Offenburg University, Germany. He is also a member of the Institute for Machine Learning and Analytics IMLA. He has been team leader of the RoboCup simulation league teams magmaFreiburg (since 1999), living systems, magmaFurtwangen and is now team leader of magmaOffenburg since 2009. Since 2014, he has also been part of the humanoid adult size league team Sweaty.

Stefan Glaser is teaching assistant for artificial intelligence and intelligent autonomous systems at the Offenburg University, Germany. He has been part of the RoboCup simulation league team magmaOffenburg since 2009 and the RoboCup humanoid adult size league team Sweaty since 2014.
]]> An interview with Nicolai Ommer: the RoboCup Soccer Small Size League https://robohub.org/an-interview-with-nicolai-ommer-the-robocupsoccer-small-size-league/ Tue, 01 Jul 2025 10:26:04 +0000 https://aihub.org/?p=17947 Kick-off in a Small Size League match. Image credit: Nicolai Ommer.

RoboCup is an international scientific initiative with the goal of advancing the state of the art of intelligent robots, AI and automation. The annual RoboCup event is due to take place from 15-21 July in Salvador, Brazil. The Soccer component of RoboCup comprises a number of Leagues, with one of these being the Small Size League (SSL). We caught up with Executive Committee member Nicolai Ommer to find out more about the SSL, how the auto referees work, and how teams use AI.

Could start by giving us a quick introduction to the Small Size League?

In the Small Size League (SSL) we have 11 robots per team – the only physical RoboCup soccer league to have the full number of players. The robots are small, cylindrical robots on wheels and they can move in any direction. They are self-built by the teams, so teams have to do both the hardware and the programming, and a lot of things have to work together to make a team work. The AI is central. We don’t have agents, so teams have a central computer at the field where they can do all the computation and then they send the commands to the robots in different abstractions. Some teams will just send velocity commands, other teams send a target.

We have a central vision system – this is maintained by the League, and has been since 2010. There are cameras above the field to track all the robots and the ball, so everyone knows where the robots are.

The robots can move up to 4 meters per second (m/s), after this point it gets quite unstable for the robots. They can change direction very quickly, and the ball can be kicked at 6.5 m/s. It’s quite fast and we’ve already had to limit the kick speed. Previously we had a limit of 8 m/s and before that 10m/s. However, no robot can catch a ball with this speed, so we decided to reduce it and put more focus on passing. This gives the keeper and the defenders a chance to actually intercept a kick.

It’s so fast that for humans it’s quite difficult to understand all the things that are going on. And that’s why, some years ago, we introduced auto refs, which help a lot in tracking, especially things like collisions and so on, where the human referee can’t watch everything at the same time.

How do the auto refs work then, and is there more than one operating at the same time?

When we developed the current system, to keep things fair, we decided to have multiple implementations of an auto ref system. These independent systems implement the same rules and then we do a majority vote on the decisions.

To do this we needed a middle component, so some years ago I started this project to have a new game controller. This is the user interface (UI) for the human referee who sits at a computer. In the UI you see the current game state, you can manipulate the game state, and this component coordinates the auto refs. The auto refs can connect and report fouls. If only one auto ref detects the foul, it won’t count it. But, if both auto refs report the foul within the time window, then it is counted. Part of the challenge was to make this all visual for the operator to understand. The human referee has the last word and makes the final decision.

We managed to establish two implementations. The aim was to have three implementations, which makes it easier to form a majority. However, it still works with just two implementations and we’ve had this for multiple years now. The implementations are from two different teams who are still active.

How do the auto refs deal with collisions?

We can detect collisions from the data. However, even for human referees it’s quite hard to determine who was at fault when two robots collide. So we had to just define a rule, and all the implementations of the auto ref implement the same rule. We wrote in the rulebook really specifically how you calculate if a collision happened and who was at fault. The first consideration is based on the velocity – below 1.5m/s it’s not a collision, above 1.5m/s it is. There is also another factor, relating to the angle calculation, that we also take into account to determine which robot was at fault.

What else do the auto refs detect?

Other fouls include the kick speed, and then there’s fouls relating to the adherence to normal game procedure. For example, when the other team has a free kick, then the opposing robots should maintain a certain distance from the ball.

The auto refs also observe non-fouls, in other words game events. For example, when the ball leaves the field. That’s the most common event. This one is actually not so easy to detect, particularly if there is a chip kick (where the ball leaves the playing surface). With the camera lens, the parabola of the ball can make it look like it’s outside the field of play when it isn’t. You need a robust filter to deal with this.

Also, when the auto refs detect a goal, we don’t trust them completely. When a goal is detected, we call it a “possible goal”. The match is halted immediately, all the robots stop, and the human referee can check all the available data before awarding the goal.

You’ve been involved in the League for a number of years. How has the League and the performance of the robots evolved over that time?

My first RoboCup was in 2012. The introduction of the auto refs has made the play a lot more fluent. Before this, we also introduced the concept of ball placement, so the robots would place the ball themselves for a free kick, or kick off, for example.

From the hardware side, the main improvement in recent years has been dribbling the ball in one-on-one situations. There has also been an improvement in the specialized skills performed by robots with a ball. For example, some years ago, one team (ZJUNlict) developed robots that could pull the ball backwards with them, move around defenders and then shoot at the goal. This was an unexpected movement, which we hadn’t seen before. Before this you had to do a pass to trick the defenders. Our team, TIGERs Mannheim, has also improved in this area now. But it’s really difficult to do this and requires a lot of tuning. It really depends on the field, the carpet, which is not standardized. So there’s a little bit of luck that your specifically built hardware is actually performing well on the competition carpet.

The Small Size League Grand Final at RoboCup 2024 in Eindhoven, Netherlands. TIGERs Mannheim vs. ZJUNlict. Video credit: TIGERs Mannheim. You can find the TIGERs’ YouTube channel here.

What are some of the challenges in the League?

One big challenge, and also maybe it’s a good thing for the League, is that we have a lot of undergraduate students in the teams. These students tend to leave the teams after their Bachelor’s or Master’s degree, the team members all change quite regularly, and that means that it’s difficult to retain knowledge in the teams. It’s a challenge to keep the performance of the team; it’s even hard to reproduce what previous members achieved. That’s why we don’t have large steps forward, because teams have to repeat the same things when new members join. However, it’s good for the students because they really learn a lot from the experience.

We are continuously working on identifying things which we can make available for everyone. In 2010 the vision system was established. It was a huge factor, meaning that teams didn’t have to do computer vision. And we are currently looking at establishing standards for wireless communication – this is currently done by everyone on their own. We want to advance the League, but at the same time, we also want to have this nature of being able to learn, being able to do all the things themselves if they want to.

You really need to have a team of people from different areas – mechanical engineering, electronics, project management. You also have to get sponsors, and you have to promote your project, get interested students in your team.

Could you talk about some of the AI elements to the League?

Most of our software is script-based, but we apply machine learning for small, subtle problems.

In my team, for example, we do model calibration with quite simple algorithms. We have a specific model for the chip kick, and another for the robot. The wheel friction is quite complicated, so we come up with a model and then we collect the data and use machine learning to detect the parameters.

For the actual match strategy, one nice example is from the team CMDragons. One year you could really observe that they had trained their model so that, once they scored goal, they upvoted the strategy that they applied before that. You could really see that the opponent reacted the same way all the time. They were able to score multiple goals, using the same strategy again and again, because they learned that if one strategy worked, they could use it again.

For our team, the TIGERs, our software is very much based on calculating scores for how good a pass is, how well can a pass be intercepted, and how we can improve the situation with a particular pass. This is hard-coded sometimes, with some geometry-based calculations, but there is also some fine-tuning. If we score a goal then we track back and see where the pass came from and we give bonuses on some of the score calculations. It’s more complicated than this, of course, but in general it’s what we try to do by learning during the game.

People often ask why we don’t do more with AI, and I think the main challenge is that, compared to other use cases, we don’t have that much data. It’s hard to get the data. In our case we have real hardware and we cannot just do matches all day long for days on end – the robots would break, and they need to be supervised. During a competition, we only have about five to seven matches in total. In 2016, we started to record all the games with a machine-readable format. All the positions are encoded, along with the referee decisions, and everything is in a log file which we publish centrally. I hope that with this growing amount of data we can actually apply some machine learning algorithms to see what previous matches and previous strategies did, and maybe get some insights.

What plans do you have for your team, the TIGERs?

We have actually won the competition for the last four years. We hope that there will be some other teams who can challenge us. Our defence has not really been challenged so we have a hard time finding weaknesses. We actually play against ourselves in simulation.

One thing that we want to improve on is precision because there is still some manual work to get everything calibrated and working as precisely as we want it. If some small detail is not working, for example the dribbling, then it risks the whole tournament. So we are working on making all these calibration processes easier, and to do more automatic data processing to determine the best parameters. In recent years we’ve worked a lot on dribbling in the 1 vs 1 situations. This has been a really big improvement for us and we are still working on that.

About Nicolai

Nicolai Ommer is a Software Engineer and Architect at QAware in Munich, specializing in designing and building robust software systems. He holds a B.Sc. in Applied Computer Science and an M.Sc. in Autonomous Systems. Nicolai began his journey in robotics with Team TIGERs Mannheim, participating in his first RoboCup in 2012. His dedication led him to join the RoboCup Small Size League Technical Committee and, in 2023, the Executive Committee. Passionate about innovation and collaboration, Nicolai combines academic insight with practical experience to push the boundaries of intelligent systems and contribute to the global robotics and software engineering communities.
]]> RoboCupRescue: an interview with Adam Jacoff https://robohub.org/robocuprescue-an-interview-with-adam-jacoff/ Wed, 25 Jun 2025 13:28:20 +0000 https://aihub.org/?p=17908 The RoboCupRescue arena at RoboCup2024, Eindhoven.

RoboCup is an international scientific initiative with the goal of advancing the state of the science of intelligent robots, AI and automation. The annual RoboCup event will take place from 15-21 July in Salvador, Brazil. The RoboCupRescue League is an important element of the competition and focuses on the challenges involved in search and rescue applications. We caught up with Adam Jacoff, co-founder of the RoboCupRescue league, former RoboCup Trustee, and chair of the organising committee, to find out more.

Could you start by giving us an overview of the Rescue League?

The RoboCupRescue League is now in its 25th year hosting competitions and workshops all around the world. We’re focused on developing autonomous robots that can enable emergency responders to perform extremely hazardous tasks from safer stand-off distances. That includes search and rescue scenarios in compromised or collapsed structures, for example, but there are many tasks that firefighters and other public safety organizations do every day that would be safer with robots doing the dangerous part. We are really the only League at RoboCup that’s focused on emergency responders, so our arenas appear a little more chaotic than the other leagues, but in a controlled sort of way. We have twenty standard test methods that we use as challenge tasks that emergency responders and our league of teams around the world have helped develop and validate over many years. These competitions do three things at once.

First, they guide the research with tangible terrains, obstacles, and tasks that are representative of emergency response operations. The researchers likely do not know any emergency responders, but we do, and we have distilled their requirements into twenty or so standard test methods. So the teams know that if they can solve the challenges presented to them in the test lanes, maybe not all twenty, but ten of them, or five of them, that combination of capabilities are applicable to some public safety mission. And the more autonomy the researchers can implement into their robots, the more likely the robots will be effective and easy to use in the field. The main objective of our league is to quicken the development of effective and reliable autonomous behaviors for these complex environments.

RoboCup is an excellent incubator for developing and evaluating cutting edge research and at the same time for developing, validating, and disseminating new standard test methods. Our standard test methods are reproducible and approach the reality of very complex and hazardous environments in an incremental way. We have increasingly difficult settings for each test. So we start easy in the Preliminaries with all the terrain lanes flat so teams can optimize their approaches for each particular challenge. By Semi-Finals the terrain lanes are inclined into crossover 15° slopes with a complex terrain obstacle in the middle. The teams start performing sequences of multiple lanes, so they need to intelligently switch between different behaviors to succeed. Then for the Finals we inject more complexity with slippery features for the robots to lose traction, pinch points to force more steering, and higher step-over obstacles to make the sequences even harder. This is how we guide and challenge both autonomous and remotely operated robots beyond their comfort zone.

Second, the intensity of the experience and education for engineers and computer scientists leads to recruiting opportunities and careers. Everybody understands that teams who do well at RoboCupRescue have done something special. They can go on to more intense research or can help implement that technology in the commercial market. Our League has inspired and validated new robotic approaches to very complex environments and helped make the jump from research to commercial implementation.

The third piece of the puzzle is our connection to various communities of emergency responders. Our sequence of test lanes is exactly what emergency responders use to support their procurement and training. They use trial data captured with the robot manufacturers to help decide what robot to buy. They also use the test lanes to train remote operators and measure their proficiency. So RoboCupRescue helps emergency responders understand what the next wave of emerging technologies can do. For example, the nuclear waste cleanup industry wants to credential their remote robot operators to ensure robots don’t get stranded within extremely harsh radiation-filled environments. The robots need to keep working for 1,000 hours or more to be cost effective. They also want to ensure remote operators are proficient enough to safely perform the intended tasks without crashing the robot on the stairs and blocking the way for all other robots. Or worse yet, force humans in protective suits to go in and clear the way. Different organizations come to RoboCupRescue competitions and see various aspects of the league in action that are useful to them. In this case, as an obstacle course to get a robot driver’s license.

At Eindhoven last year (for RoboCup2024) we had the local bomb squads provide what we call a “reverse demonstration.” All these researchers give demonstrations of their robots at their schools to sponsors, family, friends, and maybe even to emergency responders. But they never get to see what the emergency responders do for a living – how dangerous their jobs are. So the RoboCupRescue league typically pauses for a while during the competition and comes together to watch a bomb technician get into their padded suit and go through the sequence of test lanes doing all the tasks that we’re asking the robots to perform. The researchers see how much of a burden it is just to wear the padded suits, which are very heavy and require forced air into the helmet. It’s not only physically difficult, they’re walking to and interacting with known explosives to render them safe – which is unconscionable given the capabilities of robots these days. They need to be working remotely from a safe stand-off distance. These “reverse demonstrations” are likely the first time the researchers ever get to meet people in bomb suits, and understand that it is these emergency responders we are working to help. Whether it’s firefighters or police, a natural or manmade disaster, we’re trying to help emergency responders stay out of harm’s way. They need capable, reliable, and easy to use robots to deal with extreme hazards more safely and effectively.

The RoboCupRescue arena at ICRA2025 (one of the qualifying events for RoboCup2025). Credit: Adam Jacoff.

So you mentioned trying to promote autonomy over teleoperation. How do you go about trying to encourage people to use more autonomy in the league?

Competitions need to have simple and clear rules. They need to challenge robots, lead them toward practical solutions, and highlight teams that are successful in a variety of challenges. So RoboCupRescue has developed scoring tasks for mobility, dexterity, and mapping within complex terrains. To score mobility points, for example, the robot needs to drive from one end of a terrain lane to the other end, or ascend and descend obstacles like stairs. They earn 1 point for each successful traverse up to 10 total repetitions to add some statistical significance. The operator is always out of sight of the robot, so working only through their interface as if the robot is inside a building. The terrain lanes and obstacles start easy and get harder, but all teams are being evaluated in the same difficulty setting as the competition progresses, so the points stay the same.

If the robot can successfully traverse the lane or perform the dexterity tasks without the operator touching the interface, we call that autonomy. At the beginning of each traverse the operator can manually set a goal point at the other end of the lane. Then they hit “go” and we all watch and learn together. If the robot gets to the other end without the operator touching the controls, the team gets 4 points instead of 1 point for that autonomous traverse. That 4:1 one ratio appears to be enough incentive to make sure teams develop autonomous behaviors and try them at each start point. However, they can always repair back to remote operation anytime the robot stalls to score at the lesser rate, so the rules encourage trying autonomy first. Also, we give Best-In-Class Autonomy awards to recognize teams that earned the most points autonomously throughout the competition.

Why do we need autonomy in emergency response operations? First, because the onboard autonomy typically makes the robot easier to use for a remote operator, which can also mean it is more likely to succeed down range or maybe be more reliable. Also, inside buildings there are often radio communications drop-out zones. Think about your cell phone reception inside concrete buildings or in basements under buildings, these places still need to be searched even when radio communication from a remote stand-off location outside the structure doesn’t support real-time video or control. So each RoboCupRescue lane, and the entire Labyrinth or Maze used for mapping, is considered a “radio drop-out zone” with tasks that score 4 to 1 if you can perform them autonomously. That applies to mobility, dexterity, and mapping tasks.

An example of one of the lanes (challenges) that teams tackle during competition. Credit: Adam Jacoff.

Will there be any new innovations to look out for this year?

What you’re going to see new this year are four-legged robots, called quadrupeds, that now have wheels as feet. There was a commercial robot that came to RoboCup2024 in Eindhoven. I invited them to demonstrate in the test lanes while the teams were at lunch, and this new robot design literally dominated our Semi-Finals sequence. Now that was a demonstration, conducted with the operator in direct line-of-sight of the robot, so much easier than remotely controlling the robot in the same terrains. But it didn’t have a remote interface. That’s exactly what our league of teams do best by adding interfaces, sensors, mapping, manipulators, and the autonomy to make it all effective and easier to use remotely.

These legged robots with wheels are a literal step function improvement in mobility, the likes of which I’ve only seen twice in thirty plus years. They have a level of mobility relative to their size that is astonishing. So we’re turning that new robot into our league’s first “standard platform.”

One of the four-legged robots with wheels as feet. Image from video taken at ICRA 2025, of the robot tackling one of the test lanes. Credit: Adam Jacoff.

Yes, I wanted to ask about the League’s plans for introducing a standard platform. Could you talk more about that?

This robot will provide a common development platform across teams with inherently good mobility within our terrains and obstacles. The league is negotiating with the manufacturer so that teams can purchase it at a relatively low price, which will bring down the cost of entry for new teams into the league. That level of mobility within our complex terrains and obstacles makes the autonomy a lot easier to develop – teams can focus on higher-level planning because the lower-level autonomous gaits and behaviors are already relatively effective. Similarly equipped teams can collaborate more closely, compare performance more directly, and share software solutions.

As a league, we can also start introducing teaming tasks for multiple robots. What kind of teaming might we explore? Maybe a certain object of interest is too heavy to drag for one robot, but could three of them work together to get it done like a team of dogs pulling a sled? Or maybe to carry payloads and supplies down-range, or even a non-ambulatory victim in a so-called litter back up-range to the base of operations. There will be lots of possibilities.

This new class of robots will operate in all the same test lanes along with all the other wheeled, tracked, and other robots developed by teams. We score every robot similarly of course, but then can separately compare the quantitative trial results for different classes of robots. It might turn out that it’s unfair to compare a new legged robot with wheels with a slower but stronger tracked robot that’s been winning RoboCupRescue for years. It’s not like tracked robots don’t have their use cases in emergency response operations. As soon as you get to tasks that involve exerting forces on the environment, like turning valves, the robots benefit from being heavier and stronger, which typically means tracks with independent front and rear flippers to negotiate the complex terrains and climb stairs. Everyone doesn’t need to adopt the new robot to compete, teams should work on implementations that push the state of the science in whatever direction they can. Then our Best-In-Class Awards can recognize the most effective robots in a variety of like-kind implementations, wheeled, tracked, legged, and maybe now legged with wheels.

When humanoids come into the league, and they will at some point, they’ll be welcomed and will be compared directly against other humanoids at least initially. We can also aggregate performance across a class of robots to identify advantages and apparent weaknesses. Emergency responders can then look at all the trial results to see which systems might work in their environments.

Do you know how many teams will be using this new robot?

I think we have almost twenty teams involved this year, but I’m not sure how many will be using the new legged robot with wheels. I expect maybe three or four initially, which would make a reasonable bracket of teams that can collaborate and compare more closely than others. There is a team from Austria that was in the German Open that is planning to come to Brazil. I’m sure when more people see the robot in action they’ll want one. Especially any new team coming into the league starting from scratch. Meanwhile, the RoboCupRescue league is poised to evaluate that robot design fully, add essential operational capabilities, and push the state of the science a bit further.

An example of the legged, wheeled robot in action. Credit: Adam Jacoff.

Will there be any different challenges this year?

Yes, this year we’re adding a new version of our original Stepfield terrain. They were our first rather complex terrain that inspired drastic changes in robot designs twenty years ago. The Stepfield terrain ensures the robots are always in contact with multiple different terrain elevations, and there is no respite throughout, even for legged robots with wheels. For the past couple of years, we got away from the Stepfield terrain because it was the hardest terrain to replicate at a temporary venue. But the year off for COVID was a growth time for us in terms of making the test methods as inexpensive and stowable as possible, so that small research organizations can replicate the tests to practice, refine approaches, and evaluate themselves during their development process. We first found a way to fabricate new Stepfields terrains from purchased, stackable crates. Now we have a fabricated version that accomplishes the same objectives less expensively. So RoboCupRescue teams will validate that the test apparatus as fabricated can stand up to the constant rigor of a very wide variety of robots across hundreds of trials in less than a week. That’s why RoboCupRescue is an excellent incubator for standard test methods too.

Is there anything else you’d like to highlight about the League?

One of the key features of RoboCupRescue is that the test lanes fabricated for the competition typically become a robot test facility for a local organization, like a nearby emergency responder facility or university. It can be set up perpetually, or it can move periodically to support regional competitions within the country. We usually try to connect the local organizers with their nearest emergency responders who already have robots or are seeking to purchase robots. The Olympic Village is the best analogy because it stays behind after the competition is over to be useful to the local region. So as we host our 25th year of RoboCupRescue in Brazil, we add to our world map of standard test facilities and new friends that we can continue to collaborate with professionally, and hopefully help their regional emergency responders use robots to stay out of harm’s way.

About Adam

Adam Jacoff is a robotics research engineer at the National Institute of Standards and Technology (NIST) which is part of the U.S. Department of Commerce. Over the past thirty years he has developed a variety of innovative robots and directed evaluations of more than a hundred others in a range of sizes, including the first technology readiness level assessment of autonomous mobility for the U.S. Army’s Experimental Unmanned Vehicle (XUV) (2002-2003).
 
His current efforts are focused toward developing 50 standard test methods to objectively evaluate ground robots, aerial drones, and aquatic vehicles for public safety operations. He has conducted 50 international robot competitions using the test methods as challenge tasks to guide innovation and measure progress (2000-present), and more than a hundred robot exercises to refine and validate the test methods with emergency responders and robot manufacturers (2005-present). These include dozens of comprehensive robot evaluations using the test methods to quantify key capabilities guiding more than $200M of purchasing decisions for civilian and military organizations (2010-present). He is now validating use of the test methods as repeatable practice tasks to focus remote operator/pilot training and measure proficiency for credentialing (2015-present).  He received a B.S. degree in Mechanical Engineering from the University of Maryland and a M.S. degree in Computer Science from Johns Hopkins University.
]]> Gearing up for RoboCupJunior: Interview with Ana Patrícia Magalhães https://robohub.org/gearing-up-for-robocupjunior-interview-with-ana-patricia-magalhaes/ Wed, 18 Jun 2025 09:20:05 +0000 https://aihub.org/?p=17845 Action from RoboCupJunior Rescue at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

The annual RoboCup event, where teams gather from across the globe to take part in competitions across a number of leagues, will this year take place in Brazil, from 15-21 July. An important part of the week is RoboCupJunior, which is designed to introduce RoboCup to school children, and sees hundreds of kids taking part in a variety of challenges across different leagues. This year, the lead organizer for RoboCupJunior is Ana Patrícia Magalhães. We caught up with her to find out how the preparations are going, what to expect at this year’s competition, and how RoboCup inspires communities.

Could you tell us about RoboCupJunior and the plans you have for the competition this year?

RoboCup will take place from 15-21 July, in Salvador, Brazil. We expect to receive people from more than 40 countries, across the Junior and Major Leagues. We are preparing everything to accommodate all the students taking part in RoboCupJunior, who will participate in the Junior Leagues of Soccer, Rescue and OnStage. They are children and teenagers, so we have organized shuttles to take them from the hotels to the convention center. We’ve also prepared a handbook with recommendations about security, places they can visit, places to eat. The idea is to provide all the necessary support for them, because they are so young. We’re also organizing a welcome party for the Juniors so that they can experience a little bit of our culture. It will hopefully be a good experience for them.

The Juniors will be located on the first level of the mezzanine at the convention center. They will be separate from the Major Leagues, who will be on the ground floor. Of course, they’ll be able to visit the Major Leagues, and talk to the students and other competitors there, but it will be nice for them to have their own space. There will also be some parents and teachers with them, so we decided to use this special, dedicated space.

RoboCupJunior On Stage at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

Do you have any idea of roughly how many teams will be taking part?

Yes, so we’ll have about 48 teams in the Soccer Leagues, 86 teams in the Rescue Leagues, and 27 in OnStage. That’s a lot of teams. Each team has about three or four students, and many of the parents, teachers and professors travel with them too. In total, we expect about 600 people to be associated with RoboCupJunior.

RoboCupJunior Soccer at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

Have you got more RoboCupJunior participants from Brazil this year due to the location?

Yes, we have many teams from Brazil competing. I don’t know the exact number, but there are definitely more Brazilian teams this year, because it’s a lot cheaper and easier for them to travel here. When we have competitions in other countries, it’s expensive for them. For example, I have a team here in Salvador that qualified for the super regional event in the US and our team couldn’t go. They had qualified, but they couldn’t go because they didn’t have money to pay for the ticket. Now, it will be possible for all the Brazilian teams qualified to participate because it’s cheaper for them to come here. So it’s a big opportunity for development and to live the RoboCup experience. It’s very important for children and teenagers to share their research, meet people from other countries, and see what they are doing, and what research path they are following. They are very grateful for the opportunity to have their work tested against others. In a competition, it is possible to compare your research with others. So it’s different from conferences where you present a paper and show your work, but it’s not possible to compare and evaluate the results with other similar work. In a competition you have this opportunity. It’s a good way to get insights and improve your research.

RoboCupJunior Rescue at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

Your role at this RoboCup will be organizing RoboCupJunior. Are you also involved in the Major Leagues?

Yes, so my main role is organizing RoboCupJunior and I am also one of the chairs of the RoboCup Symposium. Besides, some teams from my laboratory are competing in the Major leagues. My team participates in the @Home league, but I haven’t had much time to help them recently, with all the preparations for RoboCup2025. Our laboratory also has teams from the 3d Simulation Soccer League, and the Flying Robots Demo. This will be the first time we’ll see a flying robot demo league at a RoboCup.

We’ll also have two junior teams from the Rescue Simulation League. They are very excited about taking part.

RoboCupJunior Rescue at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

RoboCup was last held in Brazil in 2014, and I understand that there were quite a lot of new people that were inspired to join a team after that. Do you think the 2025 RoboCup will have the same effect and will inspire more people in Brazil to take part?

Yes, I hope so. The last one inspired many, many students. We could perceive the difference before and after RoboCup at that time, related to projects in schools. In 2014, RoboCup was held in João Pessoa, a city in the north east that is not as developed or populated as many other states in Brazil. It really improved the research in that place and the interest in robotics especially. After the 2014 RoboCup, we’ve had many projects submitted to the Brazilian RoboCup competition from that state every year. We believe that it was because of RoboCup being held there.

We hope that RoboCup2025 next month will have the same effect. We think it might have an even bigger impact, because there is more social media now and the news can spread a lot further. We are expecting many visitors. We will have a form where schools that want to visit can enroll on a guided visit of RoboCup. This will go live on the website next week, but we are already receiving many messages from schools asking how they can participate with their group. They are interested in the events, so we have high expectations.

We have been working on organizing RoboCup2025 for over a year, and there is still much to do. We are excited to receive everybody here, both for the competition and to see the city. We have a beautiful city on the coast, and some beautiful places to visit, so I recommend that people come and stay for some days after the competition to get to know our city.

About Ana Patrícia

Ana Patrícia F. Magalhães Mascarenhas received her PhD in Computer Science from the Federal University of Bahia (2016) and Master in Mechatronics from the Federal University of Bahia (2007). She is currently an adjunct professor at the State University of Bahia (UNEB) at the Information Systems course. She is a researcher and vice coordinator of the Center for Research in Computer Architecture, Intelligent Systems and Robotics (ACSO). Her current research focuses on service robotics and software engineering, especially related to the use of Artificial Intelligence (AI) in the software development process and in Model-Driven Development (DDM).
]]> Preparing for kick-off at RoboCup2025: an interview with General Chair Marco Simões https://robohub.org/preparing-for-kick-off-at-robocup2025-an-interview-with-general-chair-marco-simoes/ Thu, 12 Jun 2025 09:30:56 +0000 https://aihub.org/?p=17798 The Salvador Convention Center, where RoboCup 2025 will take place.

RoboCup is an international scientific initiative with the goal of advancing the state of the art of intelligent robots, AI and automation. The annual RoboCup event, where teams gather from across the globe to take part in competitions across a number of leagues, will this year take place in Brazil, from 15-21 July. We spoke to Marco Simões, one of the General Chairs of RoboCup 2025 and President of RoboCup Brazil, to find out what plans they have for the event, some new initiatives, and how RoboCup has grown in Brazil over the past ten years.

Marco Simões

Could you give us a quick introduction to RoboCup 2025?

RoboCup will be held in Salvador, Brazil. When RoboCup was held in Brazil 11 years ago, in 2014, we had a total of 100,000 visitors, so that was a great success. This year, we expect even more, around 150,000, during all the events. Nowadays, AI and robotics are attracting more attention. We are also in a town, Salvador, with a bigger population than the previous location (João Pessoa). For these reasons, we estimate the attendance to be about 150,000 people.

Regarding the number of teams, registration has not closed yet, so we’re unsure about the final numbers. However, we expect to have about 300-400 teams and around 3000 competitors. We have been helping with visas, so we hope to see higher participation from teams who couldn’t attend in the previous two years due to visa issues. We are doing our best to ensure people can come and have fun at RoboCup!

This is also a great year for the RoboCup community: We have just agreed on new global league partners, including the Chinese companies Unitree, Fourier, and Booster Robotics. They will bring their humanoids and four-legged robots to RoboCup. These will not only be exhibited to the public but also used by some teams. They are amazing robots with very good skills. So, I think this will be an amazing edition of RoboCup this year.

Did the 2014 event in Brazil inspire more teams to participate in RoboCup?

Yes, we have seen a significant increase in our RoboCup community. In the last two years, Brazil has had the fourth-largest number of teams and participants at RoboCup in Bordeaux (2023) and Eindhoven (2024). This was a very big increase because ten years ago, we were not even in the top eight or nine.

We’ve made a significant effort with RoboCupJunior in the last ten years. Most people who’ve taken part in RoboCupJunior have carried on and joined the RoboCup Major League. So, the number of teams in Brazil has been increasing year by year over the last ten years. This year, we have a great number of participants because of the lower travel costs. We are expecting to be in the top three this year in terms of the highest number of participants.

Photo of participants at RoboCup 2024, which took place in Eindhoven. Photo credit: RoboCup/Bart van Overbeeke

It’s impressive that so many RoboCupJunior participants go on to join a Major League team.

Yes, we have an initiative here in Brazil called the Brazilian Robotics Olympiad. In this event, we chose two Junior Leagues – OnStage and the Rescue Line League – and we organized a competition based on these two Leagues. We run it in regional competitions all over Brazil – so 27 states. We organize at least one competition in each state during the year, and the best teams from each state come to the national competition together with the Major Leagues. We organize the Brazilian Olympiad to get RoboCupJunior to more students. This is how we’ve managed to increase participation in RoboCupJunior. Then, when students go to university, many of them continue to participate, but in the Major Leagues. So that’s a very successful strategy we’ve used in Brazil in the last 10 years.

Could you tell us about some more of the highlights from the Brazilian RoboCup community in recent years?

Two or three years ago, one of the Brazilian teams was the champion of RoboCup @Home. We have seen a big increase in the number of teams in the @Home League. In the national competition in Brazil, we have more than 12 teams participating. Back in 2014, we only had one team participating. So we’ve had a great increase—this League is one of the highlights in Brazil.

More teams are also participating in the Small Size League (part of the soccer League). Two years ago, one of the Brazilian teams was the champion of the division B of the Small Size League. So, over the last five years, we’ve seen some Brazilian teams in the top three positions in Major Leagues in the RoboCup world competition. This is a result of the increase in the number of teams and the quality of what the teams are developing. So at this time, we have an increased number of publications and teams participating in the competition with good results, so that’s very important.

Another excellent contribution for this year is a league we created five years ago – a flying robot league, where autonomous drones perform some missions and tasks. We’ve proposed this League as a demo for RoboCup2025, and we will have a Flying Robot Demo at the competition this year. This will be the first time we’ll have autonomous drones at the RoboCup competition, and the Brazilian community proposed it.

RoboCup @Home with Toyota HSR robots in the Domestic Standard Platform League, RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

Will you be taking part in the competition this year, or will you be focusing entirely on your role as General Chair?

This year, my laboratory (ACSO/Uneb) has qualified for the 3d Simulation League (soccer), the Flying Robot Demo, and RoboCup @Home, so we are participating in three Leagues. We also supervise RoboCupJunior Teams in the Rescue Simulation League. This year, my students have had only a little supervision from me because I’ve been very engaged with the organization.

In our 3D simulation team, we have lots of developments with deep reinforcement learning and some new novel strategies that allow our teams to gain new skills, and we are combining the new skills with our former multi-agent coordination strategy. For this reason, I think we will have a robust team in the competition because we are not only working on skills, we are also working on multi-agency strategies. When both aspects are joined, you can have a really good soccer team that plays very well. We have a good team and expect to achieve a greater position this year. In the latter years, we were in the top four or five, but we hope to get into the top three this year.

In 3D, you not only work on multi-agent aspects but also need to work on skills such as walking, kicking, and running. Teams are now trying to develop new skills. For example, in recent years, our team has developed the sprint running movement, which was a result of deep reinforcement learning. It is not a natural running motion but a running movement that works according to the League’s rules. It makes the robots go very fast from one point to another, making the team very competitive.

Most teams are learning skills but don’t know how to exploit them strategically in the game. Our focus is not only on creating new skills but also on using them strategically. We are currently working on a very innovative approach.

This year, the simulation league will run a challenge using a new simulator based on MuJoCo. If the challenge goes well, we may move to this new simulator in the following years, which can more realistically simulate real humanoid robots.

Action from the semi-finals of RoboCup Soccer Humanoid League at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

Finally, is there anything else you’d like to highlight about RoboCup2025?

We are working on partnerships with local companies. For example, we have sponsorship from Petrobras, one of the biggest oil companies in the world. They will discuss how they are using robotics and AI in their industry. They were also one of the first sponsors of the Flying Robots League. It’s important to have these links between industry and the RoboCup community.

We also have excellent support from local companies and the government. They will be showing the community their latest developments. In the Rescue League, for example, we’ll have a demonstration from the local force showing what they do to support people in disaster situations.

This event is also an excellent opportunity for RoboCuppers, especially those who have never been to Brazil, to spend some days after the event in Salvador, visiting some tourist spots. Salvador was the first Brazilian capital, so we have a rich history. There are a lot of historical sites to see and some great entertainment options, such as beaches or parties. People can have fun and enjoy the country!

About Marco

Marco Simões is an Associate Professor at Bahia State University, Salvador, Brazil. He is the General Chair of RoboCup2025, and President of RoboCup Brazil.
]]> #RoboCup2024 – daily digest: 21 July https://robohub.org/robocup2024-daily-digest-21-july/ Sun, 21 Jul 2024 19:48:45 +0000 https://aihub.org/?p=15404 A break in play during a Small Size League match.

Today, 21 July, saw the competitions draw to a close in a thrilling finale. In the third and final of our round-up articles, we provide a flavour of the action from this last day. If you missed them, you can find our first two digests here: 19 July | 20 July.

My first port of call this morning was the Standard Platform League, where Dr Timothy Wiley and Tom Ellis from Team RedbackBots, RMIT University, Melbourne, Australia, demonstrated an exciting advancement that is unique to their team. They have developed an augmented reality (AR) system with the aim of enhancing the understanding and explainability of the on-field action.

The RedbackBots travelling team for 2024 (L-to-R: Murray Owens, Sam Griffiths, Tom Ellis, Dr Timothy Wiley, Mark Field, Jasper Avice Demay). Photo credit: Dr Timothy Wiley.

Timothy, the academic leader of the team explained: “What our students proposed at the end of last year’s competition, to make a contribution to the league, was to develop an augmented reality (AR) visualization of what the league calls the team communication monitor. This is a piece of software that gets displayed on the TV screens to the audience and the referee, and it shows you where the robots think they are, information about the game, and where the ball is. We set out to make an AR system of this because we think it’s so much better to view it overlaid on the field. What the AR lets us do is project all of this information live on the field as the robots are moving.”

The team has been demonstrating the system to the league at the event, with very positive feedback. In fact, one of the teams found an error in their software during a game whilst trying out the AR system. Tom said that they’ve received a lot of ideas and suggestions from the other teams for further developments. This is one of the first (if not, the first) AR system to be trialled across the competition, and first time it has been used in the Standard Platform League. I was lucky enough to get a demo from Tom and it definitely added a new level to the viewing experience. It will be very interesting to see how the system evolves.

Mark Field setting up the MetaQuest3 to use the augmented reality system. Photo credit: Dr Timothy Wiley.

From the main soccer area I headed to the RoboCupJunior zone, where Rui Baptista, an Executive Committee member, gave me a tour of the arenas and introduced me to some of the teams that have been using machine learning models to assist their robots. RoboCupJunior is a competition for school children, and is split into three leagues: Soccer, Rescue and OnStage.

I first caught up with four teams from the Rescue league. Robots identify “victims” within re-created disaster scenarios, varying in complexity from line-following on a flat surface to negotiating paths through obstacles on uneven terrain. There are three different strands to the league: 1) Rescue Line, where robots follow a black line which leads them to a victim, 2) Rescue Maze, where robots need to investigate a maze and identify victims, 3) Rescue Simulation, which is a simulated version of the maze competition.

Team Skollska Knijgia, taking part in the Rescue Line, used a YOLO v8 neural network to detect victims in the evacuation zone. They trained the network themselves with about 5000 images. Also competing in the Rescue Line event were Team Overengeniering2. They also used YOLO v8 neural networks, in this case for two elements of their system. They used the first model to detect victims in the evacuation zone and to detect the walls. Their second model is utilized during line following, and allows the robot to detect when the black line (used for the majority of the task) changes to a silver line, which indicates the entrance of the evacuation zone.

Left: Team Skollska Knijgia. Right: Team Overengeniering2.

Team Tanorobo! were taking part in the maze competition. They also used a machine learning model for victim detection, training on 3000 photos for each type of victim (these are denoted by different letters in the maze). They also took photos of walls and obstacles, to avoid mis-classification. Team New Aje were taking part in the simulation contest. They used a graphical user interface to train their machine learning model, and to debug their navigation algorithms. They have three different algorithms for navigation, with varying computational cost, which they can switch between depending on the place (and complexity) in the maze in which they are located.

Left: Team Tanorobo! Right: Team New Aje.

I met two of the teams who had recently presented in the OnStage event. Team Medic’s performance was based on a medical scenario, with the team including two machine learning elements. The first being voice recognition, for communication with the “patient” robots, and the second being image recognition to classify x-rays. Team Jam Session’s robot reads in American sign language symbols and uses them to play a piano. They used the MediaPipe detection algorithm to find different points on the hand, and random forest classifiers to determine which symbol was being displayed.

Left: Team Medic Bot Right: Team Jam Session.

Next stop was the humanoid league where the final match was in progress. The arena was packed to the rafters with crowds eager to see the action.
Standing room only to see the Adult Size Humanoids.

The finals continued with the Middle Size League, with the home team Tech United Eindhoven beating BigHeroX by a convincing 6-1 scoreline. You can watch the livestream of the final day’s action here.

The grand finale featured the winners of the Middle Size League (Tech United Eindhoven) against five RoboCup trustees. The humans ran out 5-2 winners, their superior passing and movement too much for Tech United.

]]> #RoboCup2024 – daily digest: 20 July https://robohub.org/robocup2024-daily-digest-20-july/ Sat, 20 Jul 2024 18:34:07 +0000 https://aihub.org/?p=15402 The Standard Platform Soccer League in action.

This is the second of our daily digests from RoboCup2024 in Eindhoven, The Netherlands. If you missed the first digest, which gives some background to RoboCup, you can find it here.

Competitions continued across all the leagues today, with participants vying for a place in Sunday’s finals.

The RoboCup@Work league focusses on robots in work-related scenarios, utilizing ideas and concepts from other RoboCup competitions to tackle open research challenges in industrial and service robotics.

I arrived at the arena in time to catch the advanced navigation test. Robots have to autonomously navigate, picking up and placing objects at different work stations. In this advanced test, caution tape is added to the arena floor, which the robots should avoid travelling over. There is also a complex placing element where teams have to put an object that they’ve collected into a slot – get the orientation or placement of the object slightly wrong and the it won’t fall into the slot.

The RoboCup@Work arena just before competition start.

Eight teams are taking part in the league this year. Executive Committee member Asad Norouzi said that there are plans to introduce a sub-league which would provide an entry point for new teams or juniors to get into the league proper.

I caught up with Harrison Burns, Mitchell Torok and Jasper Arnold from Team MiRobot. They are based at the University of New South Wales and are attending RoboCup for the first time.

Team MiRobot from UNSW.

The team actually only started six months ago, so final preparations have been a bit stressful. However, the experience has been great fun, and the competition has gone well so far. Like most teams, they’ve had to make many refinements as the competition has progressed, leading to some late nights.

One notable feature of the team’s robot is the bespoke, in-house-designed grasping mechanism on the end of the arm. The team note that “it has good flexible jaws, so when it grabs round objects it actually pulls the object directly into it. Because it uses a linear motion, compared to a lot of other rotating jaws, it has a lot better reliability for picking up objects”.

Here is some footage from the task, featuring Team bi-t-bots and Team Singapore.

In the Middle Size Soccer league (MSL), teams of five fully autonomous robots play with a regular size FIFA ball. Teams are free to design their own hardware but all sensors have to be on-board and there is a maximum size and weight limit of 40kg for the robots. The research focus is on mechatronics design, control and multi-agent cooperation at plan and perception levels. Nine teams are competing this year.

I spoke to António Ribeiro, who is a member of the technical committee and part of Team LAR@MSL from the University of Minho, Portugal. The team started in 1998, but António and most of his colleagues on the current team have only been involved in the MSL since September 2022. The robots have evolved as the competition has progressed, and further improvements are in progress. Refinements so far have included communication, the detection system, and the control system. They are pleased with the improvements from the previous RoboCup. “Last year we had a lot of hardware issues, but this year the hardware seems pretty stable. We also changed our coding architecture and it is now much easier and faster for us to develop code because we can all work on the code at the same time on different modules”.

António cited versatility and cost-effective solutions as strengths of the team. “Our robot is actually very cheap compared to other teams. We use a lot of old chassis, and our solutions always go to the lowest cost possible. Some teams have multiple thousand dollar robots, but, for example, our vision system is around $70-80. It works pretty well – we need to improve the way we handle it, but it seems stable”.

Team LAR@MSL

The RoboCup@Home league aims to develop service and assistive robot technology with high relevance for future personal domestic applications. A set of benchmark tests is used to evaluate the robots’ abilities and performance in a realistic non-standardized home environment setting. These tests include helping to prepare breakfast, clearing the table, and storing groceries.

I arrived in time to watch the “stickler for the rules” challenge, where robots have to navigate different rooms and make sure that the people inside (“guests” at a party) are sticking to four rules: 1) there is one forbidden room – if a guest is in there the robot must alert them and ask them to follow it into another room), 2) everyone must have a drink in their hand – if not, the robot directs them to a shelf with drinks, 3) no shoes to be worn in the house, 4) there should be no rubbish left on the floor.

After watching an attempt from the LAR@Home robot, Tiago from the team told me a bit about the robot. “The goal is to develop a robot capable of multi general-purpose tasks in home and healthcare environments.” With the exception of the robotic arm, all of the hardware was built by the team. The robot has two RGBD cameras, two LIDARs, a tray (where the robot can store items that it needs to carry), and two emergency stop buttons that deactivate all moving parts. Four omnidirectional wheels allow the robot to move in any direction at any time. The wheels have independent suspension systems which guarantees that they can all be on the ground at all times, even if there are bumps and cables on the venue floor. There is a tablet that acts as a visual interface, and a microphone and speakers to enable communication between humans and the robot, which is all done via speaking and listening.

Tiago told me that the team have talked to a lot healthcare practitioners to find out the main problems faced by elderly people, and this inspired one of their robot features. “They said that the two main injury sources are from when people are trying to sit down or stand up, and when they are trying to pick something up from the floor. We developed a torso that can pick objects from the floor one metre away from the robot”.

The LAR@Home team.

You can keep up with the latest news direct from RoboCup here.

Click here to see all of our content pertaining to RoboCup.

]]> #RoboCup2024 – daily digest: 19 July https://robohub.org/robocup2024-daily-digest-19-july/ Fri, 19 Jul 2024 16:10:00 +0000 https://aihub.org/?p=15398 The main soccer arena.

RoboCup is an international scientific initiative with the goal to advance the state of the art of intelligent robots. As part of this initiative, a series of competitions and events are held throughout the year. The main showcase event is an international affair with teams travelling from far and wide to put their machines through their paces.

This year, RoboCup is being held in three arenas in the Genneper Parken, Eindhoven, The Netherlands. The organisers are expecting over 2,000 participants, from 45 different countries, with around 300 teams signed up to take part in the various competitions.

Although RoboCup started out as a football (or soccer) playing competition, other leagues have since been introduced, focussing on robots in industrial, rescue, and home settings. There is even a dedicated league for young roboticists – RoboCupJunior – where participants can take part in either football, rescue, or artistic events.

I am lucky enough to be able to attend this year, and, for the next three days, I’ll be bringing you a daily digest of some of the exciting happenings from Eindhoven.

Today, 19 July, sees the competition in full swing. The main soccer arena, boasting multiple pitches, hosts a number of the different leagues which form RoboCupSoccer.

Some of the pitches in the main soccer arena.

My first port of call was the Standard Platform League, where the round 5 champions cup match between SPQR Team vs rUNSWift was taking place. SPQR ran out winners and advance to round 6. In this league, all teams compete with identical robots (currently the humanoid NAO by Aldebaran). The robots operate fully autonomously, meaning that there is no external control from neither humans nor computers.

Standard platform league. Round 5 champions cup match between SPQR Team vs rUNSWift.

The Humanoid AdultSize league is arguably the most challenging of the leagues, with many constraints placed on the robots to make them as human-like as possible. For example, they must have roughly human-like body proportions, they need to walk on two legs, and they are only allowed to use human-like sensors (up to two cameras to sense the environment). In this AdultSize competition, two robots from each team compete, and the team members walk behind the robots to catch them in case of a fall. Such a mishap could prove costly in terms of potential hardware damage.

Action from the Humanoid AdultSize League.

The RoboCup Rescue Robot League sees teams developing robotic systems with the goal of enabling emergency responders to perform extremely hazardous tasks from safer stand-off distances. During the competition, teams compete in a round-robin, putting their robots through their paces on a number of different challenges. The leading teams following this initial phase progress to the finals on Sunday. The tasks include navigating in complex environments, opening doors, and sensing. Teams may run the machines completely autonomously, or with some assistive control. More points are awarded for completely autonomous operation.

RoboCup Rescue arena from above.

You can keep up with more RoboCup2024 news here.

]]> AI holidays 2023 https://robohub.org/ai-holidays-2023/ Sun, 31 Dec 2023 08:43:52 +0000 https://aihub.org/?p=13873

Thanks to those that sent and suggested AI and robotics-themed holiday videos, images, and stories. Here’s a sample to get you into the spirit this season.

Josh Miller used ChatGPT to create the 12 days of Christmas. Prompt: “I want to make 12 images in an early 1980s era to depict the twelve days of Christmas”.

See the pictures in higher definition here.

“Santa’s Secret: Robots Saved Christmas! | CPS XMAS video 2023” from Chair of Cyber-Physical Systems.

“AI rewriting classic Christmas stories” from GingerPale.

Using PaLM2 to generate haikus.

Prompt: “Write me a haiku about an alternative xmas”.
No gifts, no grand feast,
Just laughter and joy, warmth spread,
Kindness fills the air.

Prompt: “Write a haiku about the winter holiday season”.
Snow blankets the land,
Hearts aglow with love’s warmth,
Joyous cheer surrounds.

“Put a Bow on It | Happy Holidays” from Boston Dynamics.

“Christmas Magic: Animated Backgrounds for an Enchanting Atmosphere” from AI-ArtificialIntelligence.

“XMAS – Episode 3: AI – Christmas music for the HAKOM Christmas party” from HAKOM Time Series.

“Christmas tree in python notebook with Turtle” from Learn Upwards.

“FZI Living Lab Christmas Robotics 2023 – Robotic Reindeer Rescue” from FZI channel.

A letter to Santa generated by PaLM2. Prompt: “Write me a letter to Santa Claus. I don’t want any presents, just inspiration in my machine learning research for 2024.”

Dear Santa Claus,

As the year comes to a close, I find myself reflecting on the remarkable advancements in machine learning that have transformed our world. From self-driving cars to personalized medicine, these innovations have opened up new possibilities and enriched our lives in ways we could have only imagined a few decades ago.

I am writing to you today not to ask for material gifts, but rather for something more precious: inspiration to continue exploring the frontiers of machine learning in the year to come. As a researcher in this field, I am eager to contribute to the development of algorithms and techniques that can further enhance our understanding of the world around us and create solutions to pressing global challenges.

I am particularly fascinated by the potential of machine learning to address the climate crisis. By harnessing the power of data and computation, we can develop more efficient renewable energy sources, optimize transportation systems, and mitigate the impact of extreme weather events. Additionally, I believe that machine learning can play a crucial role in improving education, healthcare, and environmental sustainability.

I know that you have a special ability to see the world in a unique and insightful way. I hope that you can share some of your wisdom with me and guide me towards promising research directions. I am eager to embark on new challenges and make a positive impact on the world through machine learning.

Thank you for considering my request, Santa Claus. I look forward to your guidance and inspiration as I pursue my research endeavors in the year ahead.

With warmest regards,

[Your Name]

]]> AAAI Fall Symposium: Patrícia Alves-Oliveira on human-robot interaction design https://robohub.org/aaai-fall-symposium-patricia-alves-oliveira-on-human-robot-interaction-design/ Tue, 31 Oct 2023 06:25:33 +0000 https://robohub.org/?p=208623 An illustration containing electronical devices that are connected by arm-like structuresAnton Grabolle / Better Images of AI / Human-AI collaboration / Licenced by CC-BY 4.0

The AAAI Fall Symposium Series took place in Arlington, USA, and comprised seven different symposia. One of these, the tenth Artificial Intelligence for Human-Robot Interaction (AI-HRI) symposium was run as a hybrid in-person/online event, and we tuned in to the opening keynote, which was given by Patrícia Alves-Oliveira.

As a psychology student, Patrícia’s dream was to become a therapist. However, an internship, where she encountered a robot for the first time, inspired her to change her plans, and she decided to go into the field of human-robot interaction. Following a PhD in the field, she worked as a postdoc, before heading to industry as a designer in the Amazon Astro robot team.

Patrícia has worked on a number of interesting projects during her time in academia and in industry. Thinking about how to design robots for specific user needs, and keeping the user at the forefront during the design process, has been core to her work. She began by summarising three very different academic projects.

Creativity and robotics

The objective of this project was to design, fabricate, and evaluate robots as creativity-provoking tools for kids. Patrícia created a social robot named YOLO (or Your Own Living Object) that she designed to be child-proof (in other words, it could withstand being dropped and knocked over), with the aim of trying to help children explore their creativity during play. A machine learning algorithm learns the pattern of play that the child has and adapts the robot behaviour accordingly. You can see the robot in action in the demo below:

FLEXI robot

As a postdoc project, Patrícia worked on building FLEXI, a social robot embodiment kit. This kit consists of a robot (with a face, and a torso with a screen on the front), which can be customised, and an open-source end-user programming interface designed to be user-friendly. The customisation element means that it can be used for many applications. The team has deployed FLEXI across three application scenarios: community-support, mental health, and education, with the aim of assessing the flexibility of the system. You can see the robot in action, in different scenarios, here.

Social dining

This project centred on a robotic arm for people with impaired mobility. Such systems already exist for assisting people with tasks such as eating. However, in a social context they can often form a barrier between the user and the rest of the group. The idea behind this project was to consider how such a robot could be adapted to work well in a social context, for example, during a meal with family or friends. The team interviewed people with impaired mobility to assess their needs, and came up with a set of design principles for creating robot-assisted feeding systems and an implementation guide for future research in this area. You can read the research paper on this project here.

You can find out more about these three projects, and the other projects that Patrícia has been involved in, here.

Astro robot

Patrícia has long been interested in robots for the real world, and how this real-world experience is aligned with the study of robots in academia and industry. She decided to leave academia and join the Astro robot programme, which she felt was a great opportunity to work on a large-scale real-world robot project.

The Astro robot is a home robot designed to assist with tasks such as monitoring your house, delivering small objects within the home, recognising your pet, telling a story, or playing games.

Patrícia took us through a typical day in the life of a designer where she always has in mind the bigger picture of what the team is aiming for, in other words, what the ideal robot, and its interactions with humans, would look like. Coupled to that, the process is governed by core design tenets, such as the customer needs, and non-negotiable core elements that the robot should include. When considering a particular element of the robot design, for example, the delivery of an item in the robot tray, Patrícia uses storyboards to map out details of potential human-robot interactions. An important aspect of design concerns edge cases, which occur regularly in the real world. Good design will consider potential edge cases and incorporate ways to deal with them.

Patrícia closed by emphasising the importance of teamwork in the design process, in particular, the need for interdisciplinary teams; by considering design from many different points of view, the chance of innovation is higher.

You can find out more about the Artificial Intelligence for Human-Robot Interaction (AI-HRI) symposium here.

]]> Code to Joy: Why Everyone Should Learn a Little Programming – Interview with Michael Littman https://robohub.org/code-to-joy-why-everyone-should-learn-a-little-programming-interview-with-michael-littman/ Thu, 26 Oct 2023 05:26:55 +0000 https://robohub.org/?p=208586

Code to Joy: Why Everyone Should Learn a Little Programming is a new book from Michael Littman, Professor of Computer Science at Brown University and a founding trustee of AIhub. We spoke to Michael about what the book covers, what inspired it, and how we are all familiar with many programming concepts in our daily lives, whether we realize it or not.

Could you start by telling us a bit about the book, and who the intended audience is?

The intended audience is not computer scientists, although I have been getting a very warm reception from computer scientists, which I appreciate. The idea behind the book is to try to help people understand that telling machines what to do (which is how I view much of computer science and AI) is something that is really accessible to everyone. It builds on skills and practices that people already have. I think it can be very intimidating for a lot of people, but I don’t think it needs to be. I think that the foundation is there for everybody and it’s just a matter of tapping into that and building on top of it. What I’m hoping, and what I’m seeing happening, is that machine learning and AI is helping to meet people part way. The machines are getting better at listening as we try to get better at telling them what to do.

What made you decide to write the book, what was the inspiration behind it?

I’ve taught large introductory computer science classes and I feel like there’s an important message in there about how a deeper knowledge of computing can be very empowering, and I wanted to bring that to a larger audience.

Could you talk a bit about the structure of the book?

The meat of the book talks about the fundamental components that make up programs, or, in other words, that make up the way that we tell computers what to do. Each chapter covers a different one of those topics – loops, variables, conditionals, for example. Within each chapter I talk about the ways in which this concept is already familiar to people, the ways that it shows up in regular life. I point to existing pieces of software or websites where you can make use of that one particular concept to tell computers what to do. Each chapter ends with an introduction to some concepts from machine learning that can help create that particular programming construct. For example, in the chapter on conditionals, I talk about the ways that we use the word “if” in regular life all the time. Weddings, for example, are very conditionally structured, with statements like “if anyone has anything to say, speak now or forever hold your peace”. That’s kind of an “if-then” statement. In terms of tools to play with, I talk about interactive fiction. Partway between video games and novels is this notion that you can make a story that adapts itself while it’s being read. What makes that interesting is this notion of conditionals – the reader can make a choice and that will cause a branch. There are really wonderful tools for being able to play with this idea online, so you don’t have to be a full-fledged programmer to make use of conditionals. The machine learning concept introduced there is decision trees, which is an older form of machine learning where you give a system a bunch of examples and then it outputs a little flowchart for decision making.

Do you touch on generative AI in the book?

The book was already in production by the time ChatGPT came out, but I was ahead of the curve, and I did have a section specifically about GPT-3 (pre-ChatGPT) which talks about what it is, how machine learning creates it, and how it itself can be helpful in making programs. So, you see it from both directions. You get the notion that this tool actually helps people tell machines what to do, and also the way that humanity created this tool in the first place using machine learning.

Did you learn anything while you were writing the book that was particularly interesting or surprising?

Researching the examples for each chapter caused me to dig into a whole bunch of topics. This notion of interactive fiction, and that there’s tools for creating interactive fiction, I found pretty interesting. When researching another chapter, I found an example from a Jewish prayer book that was just so shocking to me. So, Jewish prayer books (and I don’t know if this is true in other belief systems as well, but I’m mostly familiar with Judaism), contain things you’re supposed to read, but they have little conditional markings on them sometimes. For example, one might say “don’t read this if it’s a Saturday”, or “don’t read this if it’s a full moon”, or “don’t read if it’s a full moon on a Saturday”. I found one passage that actually had 14 different conditions that you had to check to decide whether or not it was appropriate to read this particular passage. That was surprising to me – I had no idea that people were expected to do so much complex computation during a worship activity.

Why is it important that everybody learns a little programming?

It’s really important to keep in mind the idea that at the end of the day what AI is doing is making it easier for us to tell machines what to do, and we should share that increased capability with a broad population. It shouldn’t just be the machine learning engineers who get to tell computers what to do more easily. We should find ways of making this easier for everybody.

Because computers are here to help, but it’s a two-way street. We need to be willing to learn to express what we want in a way that can be carried out accurately and automatically. If we don’t make that effort, then other parties, companies often, will step in and do it for us. At that point, the machines are working to serve some else’s interest instead of our own. I think it’s become absolutely essential that we restore a healthy relationship with these machines before we lose any more of our autonomy.

Any final thoughts or takeaways that we should bear in mind?

I think there’s a message here for computer science researchers, as well. When we tell other people what to do, we tend to combine a description or a rule, something that’s sort of program-like, with examples, something that’s more data-like. We just intermingle them when we talk to each other. At one point when I was writing the book, I had a dishwasher that was acting up and I wanted to understand why. I read through its manual, and I was struck by how often it was the case that in telling people what to do with the dishwasher, the authors would consistently mix together a high-level description of what they are telling you to do with some particular, vivid examples: a rule for what to load into the top rack, and a list of items that fit that rule. That seems to be the way that people want to both convey and receive information. What’s crazy to me is that we don’t program computers that way. We either use something that’s strictly programming, all rules, no examples, or we use machine learning, where it’s all examples, no rules. I think the reason that people communicate this way with each other is because those two different mechanisms have complementary strengths and weaknesses and when you combine the two together, you maximize the chance of being accurately understood. And that’s the goal when we’re telling machines what to do. I want the AI community to be thinking about how we can combine what we’ve learned about machine learning with something more programming-like to make a much more powerful way of telling machines what to do. I don’t think this is a solved problem yet, and that’s something that I really hope that people in the community think about.

Code to Joy: Why Everyone Should Learn a Little Programming is available to buy now.

michael littman

Michael L. Littman is a University Professor of Computer Science at Brown University, studying machine learning and decision making under uncertainty. He has earned multiple university-level awards for teaching and his research on reinforcement learning, probabilistic planning, and automated crossword-puzzle solving has been recognized with three best-paper awards and three influential paper awards. Littman is co-director of Brown’s Humanity Centered Robotics Initiative and a Fellow of the Association for the Advancement of Artificial Intelligence and the Association for Computing Machinery. He is also a Fellow of the American Association for the Advancement of Science Leshner Leadership Institute for Public Engagement with Science, focusing on Artificial Intelligence. He is currently serving as Division Director for Information and Intelligent Systems at the National Science Foundation.
]]> Interview with Marek Šuppa: insights into RoboCupJunior https://robohub.org/interview-with-marek-suppa-insights-into-robocupjunior/ Mon, 16 Oct 2023 09:30:22 +0000 https://robohub.org/?p=208492 robocupjunior soccer match in actionA RoboCupJunior soccer match in action.

In July this year, 2500 participants congregated in Bordeaux for RoboCup2023. The competition comprises a number of leagues, and among them is RoboCupJunior, which is designed to introduce RoboCup to school children, with the focus being on education. There are three sub-leagues: Soccer, Rescue and OnStage.

Marek Šuppa serves on the Executive Committee for RoboCupJunior, and he told us about the competition this year and the latest developments in the Soccer league.

What is your role in RoboCupJunior and how long have you been involved with this league?

I started with RoboCupJunior quite a while ago: my first international competition was in 2009 in Graz, where I was lucky enough to compete in Soccer for the first time. Our team didn’t do all that well in that event but RoboCup made a deep impression and so I stayed around: first as a competitor and later to help organise the RoboCupJunior Soccer league. Right now I am serving as part of the RoboCupJunior Execs who are responsible for the organisation of RoboCupJunior as a whole.

How was the event this year? What were some of the highlights?

I guess this year’s theme or slogan, if we were to give it one, would be “back to normal”, or something like that. Although RoboCup 2022 already took place in-person in Thailand last year after two years of a pandemic pause, it was in a rather limited capacity, as COVID-19 still affected quite a few regions. It was great to see that the RoboCup community was able to persevere and even thrive throughout the pandemic, and that RoboCup 2023 was once again an event where thousands of robots and roboticists meet.

It would also be difficult to do this question justice without thanking the local French organisers. They were actually ready to organise the event in 2020 but it got cancelled due to COVID-19. But they did not give up on the idea and managed to put together an awesome event this year, for which we are very thankful.

robots used for robocupjuniorExamples of the robots used by the RoboCupJunior Soccer teams.

Turning to RoboCupJunior Soccer specifically, could you talk about the mission of the league and how you, as organisers, go about realising that mission?

The mission of RoboCupJunior consists of two competing objectives: on one hand, it needs to be a challenge that’s approachable, interesting and relevant for (mostly) high school students and at the same time it needs to be closely related to the RoboCup “Major” challenges, which are tackled by university students and their mentors. We are hence continuously trying to both make it more compelling and captivating for the students and at the same time ensure it is technical enough to help them grow towards the RoboCup “Major” challenges.

One of the ways we do that is by introducing what we call “SuperTeam” challenges, in which teams from respective countries form a so-called “SuperTeam” and compete against another “SuperTeam” as if these were distinct teams. In RoboCupJunior Soccer the “SuperTeams” are composed of four to five teams and they compete on a field that is six times larger than the “standard” fields that are used for the individual games. While in the individual matches each team can play with two robots at most (resulting in a 2v2 game) in a SuperTeam match each SuperTeam fields five robots, meaning there are 10 robots that play on the SuperTeam field during a SuperTeam match. The setup is very similar to the Division B of the Small Size League of RoboCup “Major”.

The SuperTeam games have existed in RoboCupJunior Soccer since 2013, so for quite a while, and the feedback we received on them was overwhelmingly positive: it was a lot of fun for both the participants as well as the spectators. But compared to the Small Size League games there were still two noticeable differences: the robots did not have a way of communicating with one another and additionally, the referees did not have a way of communicating with the robots. The result was that not only was there little coordination among robots of the same SuperTeam, whenever the game needed to be stopped, the referees had to physically run after the robots on the field to catch them and do a kickoff after a goal was scored. Although hilarious, it’s far from how we would imagine the SuperTeam games to look.

The RoboCupJunior Soccer Standard Communication Modules aim to do both. The module itself is a small device that is attached to each robot on the SuperTeam field. These devices are all connected via Bluetooth to a single smartphone, through which the referee can send commands to all robots on the field. The devices themselves also support direct message exchange between robots on a single SuperTeam, meaning the teams do not have to invest into figuring out how to communicate with the other robots but can make use of a common platform. The devices, as well as their firmware, are open source, meaning not only that everyone can build their own Standard Communication Module if they’d like but also that the community can participate in its development, which makes it an interesting addition to RoboCupJunior Soccer.

two teams setting up their robotsRoboCupJunior Soccer teams getting ready for the competition.

How did this new module work out in the competition? Did you see an improvement in experience for the teams and organisers?

In this first big public test we focused on exploring how (and whether) these modules can improve the gameplay – especially the “chasing robots at kickoff”. Although we’ve done “lab experiments” in the past and had some empirical evidence that it should work rather well, this was the first time we tried it in a real competition.

All in all, I would say that it was a very positive experiment. The modules themselves did work quite well and for some of us, who happened to have experience with “robot chasing” mentioned above, it was sort of a magical feeling to see the robots stop right on the main referee’s whistle.

We also found out potential areas for improvement in the future. The modules themselves do not have a power source of their own and were powered by the robots themselves. We didn’t think this would be a problem but in the “real world” test it transpired that the voltage levels the robots are capable of providing fluctuates significantly – for instance when the robot decides to aggressively accelerate – which in turn means some of the modules disconnect when the voltage is lowered significantly. However, it ended up being a nice lesson for everyone involved, one that we can certainly learn from when we design the next iterations.


The livestream from Day 4 of RoboCupJunior Soccer 2023. This stream includes the SuperTeam finals and the technical challenges. You can also view the livestream of the semifinals and finals from day three here.

Could you tell us about the emergence of deep-learning models in the RoboCupJunior leagues?

This is something we started to observe in recent years which surprised us organisers, to some extent. In our day-to-day jobs (that is, when we are not organising RoboCup), many of us, the organisers, work in areas related to robotics, computer science, and engineering in general – with some of us also doing research in artificial intelligence and machine learning. And while we always thought that it would be great to see more of the cutting-edge research being applied at RoboCupJunior, we always dismissed it as something too advanced and/or difficult to set up for the high school students that comprise the majority of RoboCupJunior students.

Well, to our great surprise, some of the more advanced teams have started to utilise methods and technologies that are very close to the current state-of-the-art in various areas, particularly computer vision and deep learning. A good example would be object detectors (usually based on the YOLO architecture), which are now used across all three Junior leagues: in OnStage to detect various props, robots and humans who perform on the stage together, in Rescue to detect the victims the robots are rescuing and in Soccer to detect the ball, the goals, and the opponents. And while the participants generally used an off-the-shelf implementations, they still needed to do all the steps necessary for a successful deployment of this technology: gather a dataset, finetune the deep-learning model and deploy it on their robots – all of which is far from trivial and is very close to how these technologies get used in both research and industry.

Although we have seen only the more advanced teams use deep-learning models at RoboCupJunior, we expect that in the future we will see it become much more prevalent, especially as the technology and the tooling around it becomes more mature and robust. It does show, however, that despite their age, the RoboCupJunior students are very close to cutting-edge research and state-of-the-art technologies.

Two teams ready to start - robots on the fieldAction from RoboCupJunior Soccer 2023.

How can people get involved in RCJ (either as a participant or an organiser?)

A very good question!

The best place to start would be the RoboCupJunior website where one can find many interesting details about RoboCupJunior, the respective leagues (such as Soccer, Rescue and OnStage), and the relevant regional representatives who organise regional events. Getting in touch with a regional representative is by far the easiest way of getting started with RoboCup Junior.

Additionally, I can certainly recommend the RoboCupJunior forum, where many RoboCupJunior participants, past and present, as well as the organisers, discuss many related topics in the open. The community is very beginner friendly, so if RoboCupJunior sounds interesting, do not hesitate to stop by and say hi!

About Marek Šuppa

Marek Suppa

Marek stumbled upon AI as a teenager when building soccer-playing robots and quickly realised he is not smart enough to do all the programming by himself. Since then, he’s been figuring out ways to make machines learn by themselves, particularly from text and images. He currently serves as the Principal Data Scientist at Slido (part of Cisco), improving the way meetings are run around the world. Staying true to his roots, he tries to provide others with a chance to have a similar experience by organising the RoboCupJunior competition as part of the Executive Committee.

]]> Interview with Roberto Figueiredo: the RoboCup experience https://robohub.org/interview-with-roberto-figueiredo-the-robocup-experience/ Fri, 04 Aug 2023 10:54:26 +0000 https://robohub.org/?p=207916 Five people holding kid-sized robots
Roberto Figueiredo is a master’s student at the University of Aveiro. He is a member of the Bold Hearts RoboCup team which competes in the Humanoid KidSize soccer league. He is currently the local representative for the Junior Rescue Simulation. We spoke to Roberto about his RoboCup journey, from the junior to the major leagues, and his experience of RoboCup events.

When was your first RoboCup event and which competition did you take part in?

I started in 2016 in the Junior leagues with my high school and I took part in the rescue simulation competition (although I originally joined the on-stage competition). This first event actually happened in Portugal, and it was similar to a workshop. We qualified to go to the world cup in rescue simulation, in Leipzig, Germany, and we ended up in second place. That was really nice, and it was my first contact with RoboCup, and with robotics generally. I’d been working with electronics in the past, but simulation gave me a bit of an introduction to the more theoretical aspects of robotics, and to AI in general. Rescue simulation makes you think of ways to make the robots independent and not manually controlled by humans.

RoboCup team in front of the rescue setupRoberto’s first RoboCup in 2016, Leipzig, pictured with the Singapore team celebrating after the finals.

Could you tell us about the subsequent RoboCup events that you took part in?

In 2017 we qualified to go to Nagoya, Japan, which was not just an amazing RoboCup, but an amazing trip. That’s another good thing about robotics, you get to meet a lot of new people in new countries. We did quite well in this competition as well, I think we reached 5th place.

After that we went to European RoboCup Junior in Italy. The following year was my last RoboCup as a junior, which was in Sydney. That was also an interesting event and I got to chat a bit more with the majors and understand how their teams worked. By this point, I had gained more experience, and I felt ready to get involved with a major league RoboCup team.

There is a big gap between the junior and major leagues. When I joined my team (the Bold Hearts), most of the team were PhDs and I was just a second year bachelor’s student so it was quite hard to pick up all the knowledge. However, if you are persistent enough and you are interested in, and passionate about, robotics you’ll get the hang of it and you’ll learn by trial and error.

Seven people standing and one kneelingEuroRoboCup 2022 in Portugal. Roberto (kneeling in photo) was part of the organising committee.

When was your first competition with the team in the major league?

My first competition was actually last year, in Thailand. We didn’t perform as we would like to, however, there is much more to RoboCup than just the competition – it is now more of a scientific and knowledge-sharing event, it’s unique. Just this year, in Bordeaux, we had a problem with our robots. Every time we disconnected the ethernet cable, the robot just stopped playing, and we couldn’t figure out what was happening. I asked another team that was using the same software – they had figured out the problem before and they told us how to solve it. I don’t think you’ll see that in other competitions. Every team has a joint objective which is making science progress, making friendships, and making other teams better by sharing their knowledge. That’s really unique.

How did you join the Bold Hearts team?

I decided to do my master’s in the UK (at the University of Hertfordshire), to experience a different country and a different style of education. When I joined, I knew there was a team so I was already looking forward to joining. After a couple of years of work, we finally got to go to a competition as a team. It’s been an amazing time and a huge learning experience.

What is your role on the team?

In our team, everyone does a bit of everything. We still have a lot of problems to solve – on both the hardware and software side. All of us currently are computer scientists so it’s a bit more of a struggle to work on the hardware side. So, I do a bit of everything, both AI and non-AI related problems. For example, I’ve done some 3d modelling for the robots, and I’m currently working on the balancing problem. We all work together on the problems which is amazing because you get to see a bit of everything and learn from everyone. Robotics is a very multidisciplinary field. You get to learn about all kinds of topics: mechanical engineering, electrical engineering, machine learning, coding in general.

The Bold Hearts’ qualification video for this year’s RoboCup competition

Could you tell us about this year’s competition (which took place in Bordeaux)?

This year we were a lot more prepared than last year, when we’d just come back from COVID, and all of our experienced members had recently left the team, due to finishing their PhDs and starting work. Creating a successful robot team is a huge integration problem. There are so many pieces that need to go together and work perfectly for the robots to function, and if one fails it looks like your system isn’t doing anything. We got walking working perfectly this year, we had vision working well too, and we had a stable decision tree, and we were able to listen to the controller (which is like a referee and passes on information about fouls, game start and stops etc.). However, we had some bugs in the decision tree that made everything fall apart and we spent a lot of time trying to debug it. This happens to a lot of teams. However, you can still appreciate the work and progress of what they have done.

Five people holding kid-sized robotsRoboCup 2023 in Bordeaux. Roberto (left) with Bold Hearts teammates.

What are the immediate plans for the team?

We are now thinking about joining the simulation competition, which is part of our league. It takes place in the winter season and we’re planning on joining to work on our software. The transition between simulation and hardware is quite hard. You need a good simulation base to be able to transfer directly the knowledge to the robot. We’re working on having a very good simulation so we can transfer, at least more easily, the knowledge learnt in simulation to the robots.

RoboCup is moving more towards AI and learning, which we can see in the 3d simulation. The robots learn a lot of the motion through reinforcement learning, for example. In the physical leagues it’s not as easy as we have to transfer that to the real world, where there is play in the joints, there’s backlash, there’s play in the 3d parts – there are a lot of variables that are not taken into account in simulations.

How has being part of RoboCup inspired your studies and research?

Every time I go to RoboCup I come out thinking about what I’m going to do next. I couldn’t be more inspired. It’s a really intense field but I love it. It makes you want to work really hard and it makes you passionate about science. I did my bachelor’s project related to RoboCup, I joined a master’s course on robotics, I keep asking my Professors if they want to start a team back in Portugal. I’m going to do my master’s thesis on robotics, on humanoids. I think humanoids are a very complex and interesting challenge. There is no one single solution.

About Roberto

Roberto Figueiredo

Roberto Figueiredo is a Portuguese, AI-focused computer scientist with a bachelor’s degree from the University of Hertfordshire. He currently pursuing a master’s in Robotics and Intelligent Systems from the University of Aveiro, and is passionate about advancing his expertise in robotics. He has long been very enthusiastic about robots and AI, being a participant in RoboCup since 2016 in the Rescue Simulation league. He has since become local representative for the Rescue League in Portugal and joined a Major team, Bold Hearts, in the Kid Size league, one of the most challenging in RoboCup Humanoid Soccer.
]]> #RoboCup2023 in tweets – part 2 https://robohub.org/robocup2023-in-tweets-part-2/ Sun, 16 Jul 2023 09:31:52 +0000 https://robohub.org/?p=207790

As this year’s RoboCup draws to a close, we take a look back at some of the highlights from the second half of the conference. Over the course of the weekend, the event focussed on the latter stages of the competitions, with the winners in all the different leagues decided. If you missed our round-up of the first half of RoboCup, you can catch up here.

]]> #RoboCup2023 in tweets – part 1 https://robohub.org/robocup2023-in-tweets-part-1/ Sat, 15 Jul 2023 09:31:45 +0000 https://robohub.org/?p=207787

This year’s RoboCup kicked off on 4 July and will run until 10 July. Taking place in Bordeaux, the event will see around 2500 participants, from 45 different countries take part in competitions, training sessions, and a symposium. Find out what attendees have been up to in preparation for, and in the first half of, the event.

]]> What’s coming up at #RoboCup2023? https://robohub.org/whats-coming-up-at-robocup2023/ Mon, 03 Jul 2023 11:47:30 +0000 https://robohub.org/?p=207677 robocup2023 logo
This year, RoboCup will be held in Bordeaux, from 4-10 July. The event will see around 2500 participants, from 45 different countries take part in competitions, training sessions, and a symposium. You can see the schedule for the week here.

The leagues and their competitions

The league competitions will take place on 6-9 July. You can find out more about the different leagues at these links:

Symposium

The RoboCup symposium will take place on 10 July. The programme can be found here.

There will be three keynote talks:

  • Cynthia Breazeal, Social Robots: Reflections and Predictions of Our Future Relationship with Personal Robots
  • Ben Moran and Guy Lever, Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
  • Laurence Devillers, Socio-affective robots: ethical issues

Find out more at the event website.

]]> RoboCup humanoid league: Interview with Jasper Güldenstein https://robohub.org/robocup-humanoid-league-interview-with-jasper-guldenstein/ Tue, 20 Sep 2022 07:22:53 +0000 https://robohub.org/?p=205558

RoboCup is an international scientific initiative with the goal of advancing the state of the art of intelligent robots, AI and automation. The annual RoboCup event returned to an in-person format for 2022, taking place from 13-17 July in Bangkok. RoboCup comprises a number of leagues, with perhaps the most well-known being the soccer leagues.

In the Humanoid League, autonomous robots with a human-inspired body plan and senses play soccer against each other. We spoke to Jasper Güldenstein, a member of the technical committee, about the competition at RoboCup 2022, and also about the Humanoid League Virtual Season. As a biweekly virtual competition taking place between the physical RoboCup events it provides teams with an opportunity to test their ideas and keep in touch throughout the year.

Could you give us an overview of the Humanoid League competition at RoboCup this year?

This year we had the first in-person event after a three year break. It was really good to have the majority of the teams coming back together and working on their robots again. Although a lot of teams came with fewer participants than they usually do, we still managed to have 12 teams in the kid size league and three teams in the adult size. Unfortunately, some teams could not participate due to travel restrictions, but we hope to see them back next year.

Humanoid teamsHumanoid league finalists with their robots

What was the format for the competition?

At the beginning we had a drop-in round, which is where we play games with one robot from each team participating in a joint team. So, we have eight robots playing from eight different teams. That is useful for finding out which teams are likely to perform the best in the competition. Next, the results from the drop-in games were used as seeding for the round robin phase of the regular competition, with the strongest teams separated into different groups. After the round robin, we have a knockout competition. The seeding method means that we can hopefully avoid the situation where very good teams get kicked out early. We saw that the most interesting games were towards the end of the competition when the teams performed really well.

Have you noticed improvements since the last physical competition?

I’d say definitely that one big thing that has improved for a lot of teams is the localisation. A lot of teams are more easily able to localise themselves on the field, and they don’t run off randomly. They are more certain that they are in the correct position.

Furthermore, I think the kicking has improved. The robots kick the ball much further than they used to. People have been tuning their kicking motions to increase the distance.

In terms of computer vision, this has definitely improved quite a bit. Something we did differently this time, which was inspired by what we did in the virtual season, is that we had a set of six different balls, all from previous FIFA competitions. For each game a ball was drawn randomly, so the teams couldn’t really prepare for all the balls. Although they were visually quite different, the teams didn’t really have any problems detecting the ball. We’ve seen, in general, that computer vision approaches have improved and these improvements have been transferred to the RoboCup competition. I think that almost all teams are using a neural network to detect the ball. This is a change from three, four, five years ago, where many teams used hand-tuned classical computer vision algorithms.

To talk a bit more about ball detection, it will be interesting to see what happens if we move to an environment with natural and/or varying light conditions. This year we were in a convention hall with uniform lighting. I believe next year, in Bordeaux, there is going to be some form of natural light coming in, and perhaps even fields that are outside. It’s still at the planning stage but we are looking forward to that. It will be a challenge and I strongly believe that the participants will find approaches to make their vision approach robust against these varying conditions.

Teams in actionThe setup and testing for the Humanoid League competition at RoboCup 2022, Bangkok.

Thinking about the transfer from the simulation to the real world, are there any specific elements that lend themselves well to being transferred?

In terms of computer vision, we had a bit of transfer. In the virtual season we concentrated a lot on changing the lighting conditions and having varied backgrounds, to be able to emulate the real world a bit better. I think a few teams used their vision approaches from the virtual season in the real world.

However, I think the strongest part is behaviour. Teams were able to test their strategies in the virtual competition and adapt every other week. For example, CIT Brains, which won the virtual season and the physical competition, made quite a few changes to their strategy and they had robust systems running. Their strategy worked really well, and in the final they managed to score several goals win against the previous world champions (Rhoban Football Club).

How did the competition go for your team (Hamburg Bit-Bots)?

We actually had quite a few hardware problems, especially on the mechanics side. The motors are wearing out and warped due to wear, and flexed more than we expected. This meant we had difficulties walking stably. And, if you can’t walk in a stable manner that defeats the purpose of everything else. It’s a really integrated system – if one component breaks, you are out of luck as you are very restricted in what you can change during the competition as you don’t have much spare equipment with you.

However, what was good for us was that we had a lot of software up and running, and a lot of it had been tested really well during the virtual league. We had to try and find a way round the walking problem algorithmically, to try to find walking parameters that were more stable. We also switched from [robot operating system] ROS 1 to ROS 2 which brought with it many challenges. We actually did a write up on the issues we faced, with some tricks and tips.

Will there be a new virtual league season this year?

Yes, we’ve discussed this in our technical committee and we plan on doing it again. The last event was successful and the teams enjoyed it. We plan on making some changes, such as logging the games to extract some interesting metrics and doing some analysis on those.

Another thing we want to do is domain randomisation, making the environment a bit more varied. This means that the approaches have to be more robust. The hope is that, when they are more robust, they can be transferred more easily to the real world. We were thinking about making the terrain slightly uneven. Another approach could be to modify the models of the robots such that the joints emulate a bit of wear, so they simulated actuators might be a bit weaker or stronger randomly, and teams have to find robust approaches to deal with that.

We won’t do everything at the beginning. We’ll move through the season and talk to the teams and form some organising groups to develop the simulation further, to run the games and to organise the competition itself. We are always happy to have input and we always talk to the teams to see what they think. It’s a competition but it’s something we build together.

Robots at the humanoid league BangkokHumanoid robots at RoboCup 2022, Bangkok.

Could you tell us about the satellite event that took place at RoboCup?

This was a discussion about how to get teams more interested in participating and how to bridge the junior leagues and the major leagues.

We know that some people who participated in RoboCup Junior specifically selected a University that has a RoboCup team so that they could join that team. It would be awesome for more people to do this, and for more juniors to know what the major league is about.

To bridge the gap between the junior and major leagues we don’t want to introduce another league, but we want some form of events where the two groups can meet, and where the juniors can show off how well they are doing. It would be good to have more interaction between the leagues, although we haven’t decided on the exact format yet.

About Jasper

Jasper

Jasper Güldenstein is a PhD student at University of Hamburg. His research focuses on humanoid navigation. He has been participating in RoboCup as a member of the Hamburg Bit-Bots since 2016 where his focus is developing not only the software of the team but also the electronics of the robot platform. In his recent master thesis he evaluated using reinforcement learning to perform path planning and execution for humanoid robots.

]]> #IJCAI invited talk: engineering social and collaborative agents with Ana Paiva https://robohub.org/ijcai-invited-talk-engineering-social-and-collaborative-agents-with-ana-paiva/ Sun, 04 Sep 2022 08:53:33 +0000 https://robohub.org/?p=205440 An illustration containing electronical devices that are connected by arm-like structuresAnton Grabolle / Better Images of AI / Human-AI collaboration / Licenced by CC-BY 4.0

The 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence (IJACI-ECAI 2022) took place from 23-29 July, in Vienna. In this post, we summarise the presentation by Ana Paiva, University of Lisbon and INESC-ID. The title of her talk was “Engineering sociality and collaboration in AI systems”.

Robots are widely used in industrial settings, but what happens when they enter our everyday world, and, specifically, social situations? Ana believes that social robots, chatbots and social agents have the potential to change the way we interact with technology. She envisages a hybrid society where humans and AI systems work in tandem. However, for this to be realised we need to carefully consider how such robots will interact with us socially and collaboratively. In essence, our world is social, so when machines enter they need to have some capabilities to interact with this social world.

Ana took us through the theory of what it means to the social. There are three aspects to this:

  1. Social understanding: the capacity to perceive others, exhibit theory of mind and respond appropriately.
  2. Intrapersonal competencies: the capability to communicate socially, establish relationships and adapt to others.
  3. Social responsibility: the capability to take actions towards the social environment, follow norms and adopt morally appropriate actions.

Ana talkingScreenshot from Ana’s talk.

Ana wants to go from this notion of social intelligence to what is called artificial social intelligence, which can be defined as: “the capability to perceive and understand social signals, manage and participate in social interactions, act appropriately in social settings, establish social relations, adapt to others, and exhibit social responsibility.”

As an engineer, she likes to build things, and, on seeing the definition above, wonders how she can pass from said definition to a model that will allow her to build social machines. This means looking at social perception, social modelling and decision making, and social acting. A lot of Ana’s work revolves around design, study and development for achieving this kind of architecture.

Ana gave us a flavour of some of the projects that she and her groups have carried out with regards to trying to engineer sociality and collaboration in robots and other agents.

One of these projects was called “Teach me how to write”, and it centres on using robots to improve the handwriting abilities of children. In this project the team wanted to create a robot that kids could teach to write. Through teaching the robot it was hypothesised that they would, in turn, improve their own skills.

The first step was to create and train a robot that could learn how to write. They used learning from demonstration to train a robotic arm to draw characters. The team realised that if they wanted to teach the kids to write, the robot had to learn and improve, and it had to make mistakes in order to be able to improve. They studied the taxonomy of handwriting mistakes that are made by children, so that they could put those mistakes into the system, and so that the robot could learn from the kids how to fix the mistakes.

You can see the system architecture in the figure below, and it includes the handwriting task element, and social behaviours. To add these social behaviours they used a toolkit developed in Ana’s lab, called FAtiMA. This toolkit can be integrated into a framework and is an affective agent architecture for creating autonomous characters that can evoke empathic responses.

system architectureScreenshot from Ana’s talk. System architecture.

In terms of actually using and evaluating the effectiveness of the robot, they couldn’t actually put the robot arm in the classroom as it was too big, unwieldy and dangerous. Therefore, they had to use a Nao robot, which moved its arms like it was writing, but it didn’t actually write.

Taking part in the study were 24 Portuguese-speaking children, and they participated in four sessions over the course of a few weeks. They assigned the robot two contrasting competencies: “learning” (where the robot improved over the course of the sessions) and “non-learning” (where the robot’s abilities remained constant). They measured the kids’ writing ability and improvement, and they used questionnaires to find out what the children thought about the friendliness of the robot, and their own teaching abilities.

They found that the children who worked with learning robot significantly improved their own abilities. They also found that the robot’s poor writing abilities did not affect the children’s fondness for it.

You can find out more about this project, and others, on Ana’s website.

]]> RoboCup2022 underway – where to find the livestream action https://robohub.org/robocup2022-underway-where-to-find-the-livestream-action/ Thu, 14 Jul 2022 13:59:14 +0000 https://robohub.org/?p=205042

RoboCup 2022 kicked off yesterday, and there have already been lots of competitions within the various leagues. Many of these are livestreamed to YouTube, and the recordings are available for anyone to watch.

Below are the links to the livestream (and recorded) channels for the leagues that have them.

In addition to these channels, there are also some stand-alone recordings.

Here are some highlights from the humanoid league drop-in tournament:

This video features a match between the HTWK-Robots and rUNSWift.

Find out more about RoboCup 2022 here.

]]> Radhika Nagpal at #NeurIPS2021: the collective intelligence of army ants https://robohub.org/radhika-nagpal-at-neurips2021-the-collective-intelligence-of-army-ants/ Tue, 01 Feb 2022 12:34:46 +0000 https://robohub.org/?p=203423

The 35th conference on Neural Information Processing Systems (NeurIPS2021) featured eight invited talks. In this post, we give a flavour of the final presentation.

The collective intelligence of army ants, and the robots they inspire

Radhika Nagpal

Radhika’s research focusses on collective intelligence, with the overarching goal being to understand how large groups of individuals, with local interaction rules, can cooperate to achieve globally complex behaviour. These are fascinating systems. Each individual is miniscule compared to the massive phenomena that they create, and, with a limited view of the actions of the rest of the swarm, they achieve striking coordination.

Looking at collective intelligence from an algorithmic point-of-view, the phenomenon emerges from many individuals interacting using simple rules. When run by these large, decentralised groups, these simple rules result in highly intelligent behaviour.

The subject of Radhika’s talk was army ants, a species which spectacularly demonstrate collective intelligence. Without any leader, millions of ants work together to self-assemble nests and build bridge structures using their own bodies.

One particular aspect of study concerned self-assembly of such bridges. Radhika’s research team, which comprised three roboticists and two biologists, found that the ants created bridges adapt to traffic flow and terrain. The ants also disassembled the bridge when the flow of ants had stopped and it wasn’t needed any more.

The team proposed the following simple hypothesis to explain this behaviour using local rules: if an ant is walking along, and experiences congestion (i.e. another ant steps on it), then it becomes stationary and turns into a bridge, allowing other ants to walk over it. Then, if no ants are walking on it any more, it can get up and leave.

These observations, and this hypothesis, led the team to consider two research questions:

  • Could they build a robot swarm with soft robots that can self-assemble amorphous structures, just like the ant bridges?
  • Could they formulate rules which allowed these robots to self-assemble temporary and adaptive bridge structures?

There were two motivations for these questions. Firstly, the goal of moving closer to realising robot swarms that can solve problems in a particular environment. Secondly, the use of a synthetic system to better understand the collective intelligence of army ants.

Screenshot from Radhika's talkScreenshot from Radhika’s talk

Radhika showed a demonstration of the soft robot designed by her group. It has two feet and a soft body, and moves by flipping – one foot remains attached, while the other detaches from the surface and flips to attach in a different place. This allows movement in any orientation. Upon detaching, a foot searches through space to find somewhere to attach. By using grippers on the feet that can hook onto textured surfaces, and having a stretchable Velcro skin, the robots can climb over each other, like the ants. The robot pulses, and uses a vibration sensor, to detect whether it is in contact with another robot. A video demonstration of two robots interacting showed that they have successfully created a system that can recreate the simple hypothesis outlined above.

In order to investigate the high-level properties of army ant bridges, which would require a vast number of robots, the team created a simulation. Modelling the ants to have the same characteristics as their physical robots, they were able to replicate the high level properties of army ant bridges with their hypothesized rules.

You can read the round-ups of the other NeurIPS invited talks at these links:
#NeurIPS2021 invited talks round-up: part one – Duolingo, the banality of scale and estimating the mean
#NeurIPS2021 invited talks round-up: part two – benign overfitting, optimal transport, and human and machine intelligence

]]> Maria Gini wins the 2022 ACM/SIGAI Autonomous Agents Research Award https://robohub.org/maria-gini-wins-the-2022-acm-sigai-autonomous-agents-research-award/ Tue, 18 Jan 2022 09:28:50 +0000 https://robohub.org/?p=202975

Congratulations to Professor Maria Gini on winning the ACM/SIGAI Autonomous Agents Research Award for 2022! This prestigious prize recognises years of research and leadership in the field of robotics and multi-agent systems.

Maria Gini is Professor of Computer Science and Engineering at the University of Minnesota, and has been at the forefront of the field of robotics and multi-agent systems for many years, consistently bringing AI into robotics.

Her work includes the development of:

  • novel algorithms to connect the logical and geometric aspects of robot motion and learning,
  • novel robot programming languages to bridge the gap between high-level programming languages and programming by guidance,
  • pioneering novel economic-based multi-agent task planning and execution algorithms.

Her work has spanned both the design of novel algorithms and practical applications. These applications have been utilized in settings as varied as warehouses and hospitals, with uses such as surveillance, exploration, and search and rescue.

Maria has been an active member and leader of the agents community since its inception. She has been a consistent mentor and role model, deeply committed to bringing diversity to the fields of AI, robotics, and computing. She is also the former President of International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

Maria will be giving an invited talk at AAMAS 2022. More details on this will be available soon on the conference website.

]]> Interview with Tao Chen, Jie Xu and Pulkit Agrawal: CoRL 2021 best paper award winners https://robohub.org/interview-with-tao-chen-jie-xu-and-pulkit-agrawal-corl-2021-best-paper-award-winners/ Wed, 24 Nov 2021 13:39:08 +0000 https://robohub.org/?p=202555

Congratulations to Tao Chen, Jie Xu and Pulkit Agrawal who have won the CoRL 2021 best paper award!

Their work, A system for general in-hand object re-orientation, was highly praised by the judging committee who commented that “the sheer scope and variation across objects tested with this method, and the range of different policy architectures and approaches tested makes this paper extremely thorough in its analysis of this reorientation task”.

Below, the authors tell us more about their work, the methodology, and what they are planning next.

What is the topic of the research in your paper?

We present a system for reorienting novel objects using an anthropomorphic robotic hand with any configuration, with the hand facing both upwards and downwards. We demonstrate the capability of reorienting over 2000 geometrically different objects in both cases. The learned controller can also reorient novel unseen objects.

Could you tell us about the implications of your research and why it is an interesting area for study?

Our learned skill (in-hand object reorientation) can enable fast pick-and-place of objects in desired orientations and locations. For example, in logistics and manufacturing, it is a common demand to pack objects into slots for kitting. Currently, this is usually achieved via a two-stage process involving re-grasping. Our system will be able to achieve it in one step, which can substantially improve the packing speed and boost the manufacturing efficiency.

Another application is enabling robots to operate a wider variety of tools. The most common end-effector in industrial robots is a parallel-jaw gripper, partially due to its simplicity in control. However, such an end-effector is physically unable to handle many tools we see in our daily life. For example, even using pliers is difficult for such a gripper as it cannot dexterously move one handle back and forth. Our system will allow a multi-fingered hand to dexterously manipulate such tools, which opens up a new area for robotics applications.

Could you explain your methodology?

We use a model-free reinforcement learning algorithm to train the controller for reorienting objects. In-hand object reorientation is a challenging contact-rich task. It requires a tremendous amount of training. To speed up the learning process, we first train the policy with privileged state information such as object velocities. Using the privileged state information drastically improves the learning speed. Other than this, we also found that providing a good initialization on the hand and object pose is critical for training the controller to reorient objects when the hand faces downward. In addition, we develop a technique to facilitate the training by building a curriculum on gravitational acceleration. We call this technique “gravity curriculum”.

With these techniques, we are able to train a controller that can reorient many objects even with a downward-facing hand. However, a practical concern of the learned controller is that it makes use of privileged state information, which can be nontrivial to get in the real world. For example, it is hard to measure the object’s velocity in the real world. To ensure that we can deploy a controller reliably in the real world, we use teacher-student training. We use the controller trained with the privileged state information as the teacher. Then we train a second controller (student) that does not rely on any privileged state information and hence has the potential to be deployed reliably in the real world. This student controller is trained to imitate the teacher controller using imitation learning. The training of the student controller becomes a supervised learning problem and is therefore sample-efficient. In the deployment time, we only need the student controller.

What were your main findings?

We developed a general system that can be used to train controllers that can reorient objects with either the robotic hand facing upward or downward. The same system can also be used to train controllers that use external support such as a supporting surface for object re-orientation. Such controllers learned in our system are robust and can also reorient unseen novel objects. We also identified several techniques that are important for training a controller to reorient objects with a downward-facing hand.

A priori one might believe that it is important for the robot to know about object shape in order to manipulate new shapes. Surprisingly, we find that the robot can manipulate new objects without knowing their shape. It suggests that robust control strategies mitigate the need for complex perceptual processing. In other words, we might need much simpler perceptual processing strategies than previously thought for complex manipulation tasks.

What further work are you planning in this area?

Our immediate next step is to achieve such manipulation skills on a real robotic hand. To achieve this, we will need to tackle many challenges. We will investigate overcoming the sim-to-real gap such that the simulation results can be transferred to the real world. We also plan to design new robotic hand hardware through collaboration such that the entire robotic system can be dexterous and low-cost.

About the authors

Tao ChenTao Chen is a Ph.D. student in the Improbable AI Lab at MIT CSAIL, advised by Professor Pulkit Agrawal. His research interests revolve around the intersection of robot learning, manipulation, locomotion, and navigation. More recently, he has been focusing on dexterous manipulation. His research papers have been published in top AI and robotics conferences. He received his master’s degree, advised by Professor Abhinav Gupta, from the Robotics Institute at CMU, and his bachelor’s degree from Shanghai Jiao Tong University.

Jie XuJie Xu is a Ph.D. student at MIT CSAIL, advised by Professor Wojciech Matusik in the Computational Design and Fabrication Group (CDFG). He obtained a bachelor’s degree from Department of Computer Science and Technology at Tsinghua University with honours in 2016. During his undergraduate period, he worked with Professor Shi-Min Hu in the Tsinghua Graphics & Geometric Computing Group. His research mainly focuses on the intersection of Robotics, Simulation, and Machine Learning. Specifically, he is interested in the following topics: robotics control, reinforcement learning, differentiable physics-based simulation, robotics control and design co-optimization, and sim-to-real.

Pulkit AgrawalDr Pulkit Agrawal is the Steven and Renee Finn Chair Professor in the Department of Electrical Engineering and Computer Science at MIT. He earned his Ph.D. from UC Berkeley and co-founded SafelyYou Inc. His research interests span robotics, deep learning, computer vision and reinforcement learning. Pulkit completed his bachelor’s at IIT Kanpur and was awarded the Director’s Gold Medal. He is a recipient of the Sony Faculty Research Award, Salesforce Research Award, Amazon Machine Learning Research Award, Signatures Fellow Award, Fulbright Science and Technology Award, Goldman Sachs Global Leadership Award, OPJEMS, and Sridhar Memorial Prize, among others.

Find out more

  • Read the paper on arXiv.
  • The videos of the learned policies are available here, as is a video of the authors’ presentation at CoRL.
  • Read more about the winning and shortlisted papers for the CoRL awards here.
]]> International conference on intelligent robots and systems (IROS) https://robohub.org/international-conference-on-intelligent-robots-and-systems-iros/ Fri, 23 Oct 2020 15:10:35 +0000 https://robohub.org/international-conference-on-intelligent-robots-and-systems-iros/

This Sunday sees the start of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). This year the event is online and free for anyone to attend. Content will be available from the platform on demand, with access available from 25 October to 25 November 2020.

IROS conferences have traditionally had a theme and this year is no different with the emphasis being on “consumer robotics and our future”. You can sign up here.

Plenaries

IROS will feature three plenary talks. The speakers and topics are as follows:

  • Danica Kragic Robotics and artificial intelligence impacts on the fashion industry
  • Cynthia Breazeal Living with social robots: from research to commercialization and back
  • Yukie Nagai Cognitive development in humans and robots: new insights into intelligence

Keynote speakers

There are also nine keynote talks covering a number of topics.

  • Frank Dellaert Air, sea, and space robots
  • Anya Petrovskaya Driverless vehicles and field robots
  • Ashish Deshpande Rehabilitation robotics
  • Jonathan Hurst Humanoids
  • I-Ming Chen Food handling robotics
  • Steve LaValle Perception, action and control
  • Nikolaus Correll Grasping, haptics and end-effectors
  • Andrea Thomaz Human-robot interaction
  • Sarah Bergbreiter Design, micro and bio-inspired robotics

Technical talks

The technical talks have been divided into 12 topic areas.

  • Air, sea, and space robots
  • Driverless vehicles and field robots
  • Medical, cellular, micro and nano robots
  • Humanoids, exoskeletons, and rehab robots
  • Localization, mapping and navigation
  • Dynamics, control and learning
  • Design, mechanisms, actuators, soft and bio-inspired robots
  • Perception, action, and cognition
  • Grasping, haptics and end-effectors
  • Human-robot interaction, teleoperation, and virtual reality
  • Swarms and multi-robots
  • Industry 4.0

Each talk will feature its digest slide, pre-recorded video presentation, and the paper’s PDF. These will be available from 25 October, so keep an eye on the website.

Workshops

There are a whopping 35 workshops to choose from. These have on-demand content and also live sessions (dates vary so visit the webpages below for more information about each specific workshop).

  1. 3rd workshop on proximity perception in robotics: towards multi-modal cognition, Stefan Escaida Navarro*, Stephan Mühlbacher-Karrer, Hubert Zangl, Keisuke Koyama, Björn Hein, Ulrike Thomas, Hosam Alagi, Yitao Ding, Christian Stetco
  2. Bringing geometric methods to robot learning, optimization and control, Noémie Jaquier*, Leonel Rozo, Søren Hauberg, Hans-Peter Schröcker, Suvrit Sra
  3. 12th IROS20 workshop on planning, perception and navigation for intelligent vehicles, Philippe Martinet*, Christian Laugier, Marcelo H Ang Jr, Denis Fernando Wolf
  4. Robot-assisted training for primary care: how can robots help train doctors in medical examinations?, Thrishantha Nanayakkara*, Florence Ching Ying Leong, Thilina Dulantha Lalitharatne, Liang He, Fumiya Iida, Luca Scimeca, Simon Hauser, Josie Hughes, Perla Maiolino
  5. Workshop on animal-robot interaction, Cesare Stefanini and Donato Romano*
  6. Ergonomic human-robot collaboration: opportunities and challenges, Wansoo Kim*, Luka Peternel, Arash Ajoudani, Eiichi Yoshida
  7. New advances in soft robots control, Concepción A. Monje*, Egidio Falotico, Santiago Martínez de la Casa
  8. Autonomous system in medicine: current challenges in design, modeling, perception, control and applications, Hang Su, Yue Chen*, Jing GUO, Angela Faragasso, Haoyong Yu, Elena De Momi
  9. MIT MiniCheetah workshop, Sangbae Kim*, Patrick M. Wensing, Inhyeok Kim
  10. Workshop on humanitarian robotics, Garrett Clayton*, Raj Madhavan, Lino Marques
  11. Robotics-inspired biology, Nick Gravish*, Kaushik Jayaram, Chen Li, Glenna Clifton, Floris van Breugel
  12. Robots building robots. Digital manufacturing and human-centered automation for building consumer robots, Paolo Dario*, George Q. Huang, Peter Luh, MengChu Zhou
  13. Cognitive robotic surgery, Michael C. Yip, Florian Richter*, Danail Stoyanov, Francisco Vasconcelos, Fanny Ficuciello, Emmanuel B Vander Poorten, Peter Kazanzides, Blake Hannaford, Gregory Scott Fischer
  14. Application-driven soft robotic systems: Translational challenges, Sara Adela Abad Guaman, Lukas Lindenroth, Perla Maiolino, Agostino Stilli*, Kaspar Althoefer, Hongbin Liu, Arianna Menciassi, Thrishantha Nanayakkara, Jamie Paik, Helge Arne Wurdemann
  15. Reliable deployment of machine learning for long-term autonomy, Feras Dayoub*, Tomáš Krajník, Niko Sünderhauf, Ayoung Kim
  16. Robotic in-situ servicing, assembly, and manufacturing, Craig Carignan*, Joshua Vander Hook, Chakravarthini Saaj, Renaud Detry, Giacomo Marani
  17. Benchmarking progress in autonomous driving, Liam Paull*, Andrea Censi, Jacopo Tani, Matthew Walter, Felipe Codevilla, Sahika Genc, Sunil Mallya, Bhairav Mehta
  18. ROMADO: RObotic MAnipulation of Deformable Objects, Miguel Aranda*, Juan Antonio Corrales Ramon, Pablo Gil, Gonzalo Lopez-Nicolas, Helder Araujo, Youcef Mezouar
  19. Perception, learning, and control for autonomous agile vehicles, Giuseppe Loianno*, Davide Scaramuzza, Sertac Karaman
  20. Planetary exploration robots: challenges and opportunities, Hendrik Kolvenbach*, William Reid, Kazuya Yoshida, Richard Volpe
  21. Application-oriented modelling and control of soft robots, Thomas George Thuruthel*, Cosimo Della Santina, Seyedmohammadhadi Sadati, Federico Renda, Cecilia Laschi
  22. State of the art in robotic leg prostheses: where we are and where we want to be, Tommaso Lenzi*, Robert D. Gregg, Elliott Rouse, Joost Geeroms
  23. Worskhop on perception, planning and mobility in forestry robotics (WPPMFR 2020), João Filipe Ferreira* and David Portugal
  24. Why robots fail to grasp? – failure ca(u)ses in robot grasping and manipulation, Joao Bimbo*, Dimitrios Kanoulas, Giulia Vezzani, Kensuke Harada
  25. Trends and advances in machine learning and automated reasoning for intelligent robots and systems, Abdelghani Chibani, Craig Schlenoff, Yacine Amirat*, Shiqi Zhang, Jong-Hwan Kim, Ferhat Attal
  26. Learning impedance modulation for physical interaction: insights from humans and advances in robotics, Giuseppe Averta*, Franco Angelini, Meghan Huber, Jongwoo Lee, Manolo Garabini
  27. New horizons of robot learning – from industrial challenges to future capabilities, Kim Daniel Listmann* and Elmar Rueckert
  28. Robots for health and elderly care (RoboHEC), Leon Bodenhagen*, Oskar Palinko, Francois Michaud, Adriana Tapus, Julie Robillard
  29. Wearable SuperLimbs: design, communication, and control, Harry Asada*
  30. Human Movement Understanding for Intelligent Robots and Systems, Emel Demircan*, Taizo Yoshikawa, Philippe Fraisse, Tadej Petric
  31. Construction and architecture robotics, Darwin Lau*, Yunhui Liu, Tobias Bruckmann, Thomas Bock, Stéphane Caro
  32. Mechanisms and design: from inception to realization, Hao Su*, Matei Ciocarlie, Kyu-Jin Cho, Darwin Lau, Claudio Semini, Damiano Zanotto
  33. Bringing constraint-based robot programming to real-world applications, Wilm Decré*, Herman Bruyninckx, Gianni Borghesan, Erwin Aertbelien, Lars Tingelstad, Darwin G. Caldwell, Enrico Mingo, Abderrahmane Kheddar, Pierre Gergondet
  34. Managing deformation: a step towards higher robot autonomy, Jihong Zhu*, Andrea Cherubini, Claire Dune, David Navarro-Alarcon
  35. Social AI for human-robot interaction of human-care service robots, Ho Seok Ahn*, Hyungpil Moon, Minsu Jang, Jongsuk Choi

Robot challenges

Another element of the conference that sounds interesting is the robot challenges. There are three of these and you should be able to watch the competitions in action next week.

  1. Open cloud robot table organization challenge (OCRTOC). This competition focusses on table organisation tasks. Participants will need to organize the objects in the scene according to a target configuration. This competition will be broadcast on 25-27 October.
  2. 8th F1Tenth autonomous Grand Prix @ IROS 2020. This competition will take the form of a virtual race with standardised vehicles and hardware. The qualifying phase is a timed trial. The Grand Prix phase pits virtual competitors against each other on the same track. The race will be broadcast on 27 October.
  3. Robotic grasping and manipulation competition. There are two sections to this competition. In the first the robot has to make five cups of iced Matcha green tea. The second involves disassembly and assembly using a NIST Task Board.
]]> #ICRA2020 workshops on robotics and learning https://robohub.org/icra2020-workshops-on-robotics-and-learning/ Wed, 15 Jul 2020 22:11:57 +0000 https://robohub.org/icra2020-workshops-on-robotics-and-learning/

This year the International Conference on Robotics and Automation (ICRA) is being run as a virtual event. One interesting feature of this conference is that it has been extended to run from 31 May to 31 August. A number of workshops were held on the opening day and here we focus on two of them: “Learning of manual skills in humans and robots” and “Emerging learning and algorithmic methods for data association in robotics”.

Learning of manual skills in humans and robots

This workshop was organised by Aude Billard, EPFL and Dagmar Sternad, Northeastern University. It brought together researchers from human motor control and from robotics to answer questions such as: How do humans achieve manual dexterity? What kind of practice schedules can shape these skills? Can some of these strategies be transferred to robots? To which extent is robot manual skill limited by the hardware, what can be learned and what cannot?

The third session of the workshop focussed on “Learning skills” and you can watch the two talks and the discussions below:

Jeannette Bohg – Learning to scaffold the development of robotic manipulation skills

Dagmar Sternad – Learning and control in skilled interactions with objects: A task-dynamic approach

Discussion with Jeannette Bohg and Dagmar Sternad

Emerging learning and algorithmic methods for data association in robotics

This workshop covered emerging algorithmic methods based on optimization and graph-theoretic techniques, learning and end-to-end solutions based on deep neural networks, and the relationships between these techniques.

You can watch the workshop in full here:

Below is the programme with the times indicating the position of that talk in the YouTube video:
11:00 Ayoung Kim – Learning motion and place descriptor from LiDARs for long-term navigation
34:11 Xiaowei Zhou – Learning correspondences for 3D reconstruction and pose estimation
51:30 Florian Bernard – Higher-order projected power iterations for scalable multi-matching
1:11:24 Cesar Cadena – High level understanding in the data association problem
1:34:55 Spotlight talk 1: Daniele Cattaneo – CMRNet++: map and camera agnostic monocular visual localization in LiDAR maps
1:50:45 Nicholas Roy – The role of semantics in perception
2:11:12 Kostas Daniilidis – Learning representations for matching
2:33:26 Jonathan How – Consistent multi-view data association
2:51:40 John Leonard – A research agenda for robust semantic SLAM
3:17:58 Luca Carlone – Towards certifiably robust spatial perception
3:39:36 Roberto Tron – Fast, consistent distributed matching for robotics applications
3:59:22 Randal Beard – Tracking moving objects from a moving camera in 3d environments
4:18:49 Nikolay Atanasov – A unifying view of geometry, semantics, and data association in SLAM
4:39:03 Spotlight talk 2: Nathaniel Glaser – Enhancing multi-robot perception via learned data association

]]> The Age of A.I. YouTube series https://robohub.org/the-age-of-a-i-youtube-series/ Mon, 06 Apr 2020 21:53:00 +0000 https://robohub.org/the-age-of-a-i-youtube-series/

The YouTube originals series “The Age of A.I.” was released in December 2019. If you haven’t already seen it now could be a good time to catch up – with much of the world in enforced or voluntary isolation many of us will be stuck at home with hours to fill. Sit back and marvel at the many incredible, and often heart-warming, applications of AI.

Episode 1 features researchers Mark Sagar, University of Auckland and Gil Weinberg, Georgia Tech. Mark works with will.i.am on a digital avatar and presents BabyX – a virtual animated baby that learns and reacts like a human baby. Gil makes music using AI and has teamed up other researchers at Georgia Tech to create smart prosthetics for amputees, combining ultrasound signals and machine learning.

Watch the other episodes:
Episode 2: Healed through A.I.

Episode 3: Using A.I. to build a better human

Episode 4: Love, art and stories: decoded

Episode 5: The ‘Space Architects’ of Mars

Episode 6: Will a robot take my job?

Episode 7: Saving the world one algorithm at a time

Episode 8: How A.I. is searching for Aliens

You can access the full series here.

]]>