Be it the news or the dwindling number of creatures hitting your windscreens, it will not have evaded you that the insect world in bad shape.

In the last three decades, the global biomass of flying insects has shrunk by 75%. Among the trend’s most notables victims is the world’s most important pollinator, the honeybee. In the United States, 48% of honeybee colonies died in 2023 alone, making it the second deadliest year on record. This significant loss is due in part to colony collapse disorder (CCD), the sudden disappearance of bees. In contrast, European countries report lower but still worrisome rates of colony losses, ranging from 6% to 32%.

This decline causes many of our essential food crops to be under-pollinated, a phenomenon that threatens our society’s food security.

Debunking the sci-fi myth of robotic bees

So, what can be done? Given pesticides’ role in the decline of bee colonies, commonly proposed solutions include a shift away from industrial farming and toward less pesticide-intensive, more sustainable forms of agriculture.

Others tend to look toward the sci-fi end of things, with some scientists imagining that we could eventually replace live honeybees with robotic ones. Such artificial bees could interact with flowers like natural insects, maintaining pollination levels despite the declining numbers of natural pollinators. The vision of artificial pollinators contributed to ingenious designs of insect-sized robots capable of flying.

In reality, such inventions are more effective at educating us over engineers’ fantasies than they are at reviving bee colonies, so slim are their prospects of materialising. First, these artificial pollinators would have to be equipped for much more more than just flying. Daily tasks carried out by the common bee include searching for plants, identifying flowers, unobtrusively interacting with them, locating energy sources, ducking potential predators, and dealing with adverse weather conditions. Robots would have to perform all of these in the wild with a very high degree of reliability since any broken-down or lost robot can cause damage and spread pollution. Second, it remains to be seen whether our technological knowledge would be even capable of manufacturing such inventions. This is without even mentioning the price tag of a swarm of robots capable of substituting pollination provided by a single honeybee colony.

Inside a smart hive

Rather than trying to replace honeybees with robots, our two latest projects funded by the European Union propose that the robots and honeybees actually team up. Were these to succeed, struggling honeybee colonies could be transformed into bio-hybrid entities consisting of biological and technological components with complementary skills. This would hopefully boost and secure the colonies’ population growth as more bees survive over harsh winters and yield more foragers to pollinate surrounding ecosystems.

The first of these projects, Hiveopolis, investigates how the complex decentralised decision-making mechanism in a honeybee colony can be nudged by digital technology. Begun in 2019 and set to end in March 2024, the experiment introduces technology into three observation hives each containing 4,000 bees, by contrast to 40,000 bees for a normal colony.

Within this honeybee smart home, combs have integrated temperature sensors and heating devices, allowing the bees to enjoy optimal conditions inside the colony. Since bees tend to snuggle up to warmer locations, the combs also enables us to direct them toward different areas of the hive. And as if that control weren’t enough, the hives are also equipped with a system of electronic gates that monitors the insects movements. Both technologies allow us to decide where the bees store honey and pollen, but also when they vacate the combs so as to enable us to harvest honey. Last but not least, the smart hive contains a robotic dancing bee that can direct foraging bees toward areas with plants to be pollinated.

Due to the experiment’s small scale, it is impossible to draw conclusions on the extent to which our technologies may have prevented bee losses. However, there is little doubt what we have seen thus far give reasons to be hopeful. We can confidently assert that our smart beehives allowed colonies to survive extreme cold during the winter in a way that wouldn’t otherwise be possible. To precisely assess how many bees these technologies have saved would require upscaling the experiment to hundreds of colonies.

Pampering the queen bee

Our second EU-funded project, RoboRoyale, focuses on the honeybee queen and her courtyard bees, with robots in this instance continuously monitoring and interacting with her Royal Highness.

Come 2024, we will equip each hive with a group of six bee-sized robots, which will groom and feed the honeybee queen to affect the number of eggs she lays. Some of these robots will be equipped with royal jelly micro-pumps to feed her, while others will feature compliant micro-actuators to groom her. These robots will then be connected to a larger robotic arm with infrared cameras, that will continuously monitor the queen and her vicinity.

As witnessed by the photo to the right and also below, we have already been able to successfully introduce the robotic arm within a living colony. There it continuously monitored the queen and determined her whereabouts through light stimuli.

Emulating the worker bees

In a second phase, it is hoped the bee-sized robots and robotic arm will be able to emulate the behaviour of the workers, the female bees lacking reproductive capacity who attend to the queen and feed her royal jelly. Rich in water, proteins, carbohydrates, lipids, vitamins and minerals, this nutritious substance secreted by the glands of the worker bees enables the queen to lay up to thousands of eggs a day.

Worker bees also engage in cleaning the queen, which involves licking her. During such interactions, they collect some of the queen’s pheromones and disperse them throughout the colony as they move across the hive. The presence of these pheromones controls many of the colony’s behaviours and notifies the colony of a queen’s presence. For example, in the event of the queen’s demise, a new queen must be quickly reared from an egg laid by the late queen, leaving only a narrow time window for the colony to react.

Finally, it is believed worker bees may also act as the queen’s guides, leading her to laying eggs in specific comb cells. The size of these cells can determine if the queen lays a diploid or haploid egg, resulting in the bee developing into either into drone (male) or worker (female) bee. Taking over these guiding duties could affect no less than the rate’s entire reproductive rate.

How robots can prevent bee cannibalism

This could have another virtuous effect: preventing cannibalism.

During tough times, such as long periods of rain, bees have to make do with little pollen intake. This forces them to feed young larvae to older ones so that at least the older larvae has a chance to survive. Through RoboRoyale, we will look not only to reduce chances of this behaviour occurring, but also quantify to what extent it occurs under normal conditions.

Ultimately, our robots will enable us to deepen our understanding of the very complex regulation processes inside honeybee colonies through novel experimental procedures. The insights gained from these new research tracks will be necessary to better protect these valuable social insects and ensure sufficient pollination in the future – a high stakes enterprise for food security.

This article is the result of The Conversation’s collaboration with Horizon, the EU research and innovation magazine.

Farshad Arvin is a member of the Department of Computer Science at Durham University in the UK. The research of Farshad Arvin is primarily funded by the EU H2020 and Horizon Europe programmes.

Martin Stefanec is a member of the Institute of Biology at the University of Graz. He has received funding from the EU programs H2020 and Horizon Europe.

Tomas Krajnik is member of the Institute of Electrical and Electronics Engineers (IEEE). The research of Tomas Krajnik is primarily funded by EU H2020 Horizon programme and Czech National Science Foundation.

Researchers have come up with an innovative approach to building deformable underwater robots using simple repeating substructures. The team has demonstrated the new system in two different example configurations, one like an eel, pictured here in the MIT tow tank. Credit: Courtesy of the researchers

By David L. Chandler | MIT News Office

Underwater structures that can change their shapes dynamically, the way fish do, push through water much more efficiently than conventional rigid hulls. But constructing deformable devices that can change the curve of their body shapes while maintaining a smooth profile is a long and difficult process. MIT’s RoboTuna, for example, was composed of about 3,000 different parts and took about two years to design and build.

Now, researchers at MIT and their colleagues — including one from the original RoboTuna team — have come up with an innovative approach to building deformable underwater robots, using simple repeating substructures instead of unique components. The team has demonstrated the new system in two different example configurations, one like an eel and the other a wing-like hydrofoil. The principle itself, however, allows for virtually unlimited variations in form and scale, the researchers say.

The work is being reported in the journal Soft Robotics, in a paper by MIT research assistant Alfonso Parra Rubio, professors Michael Triantafyllou and Neil Gershenfeld, and six others.

Existing approaches to soft robotics for marine applications are generally made on small scales, while many useful real-world applications require devices on scales of meters. The new modular system the researchers propose could easily be extended to such sizes and beyond, without requiring the kind of retooling and redesign that would be needed to scale up current systems.

The deformable robots are made with lattice-like pieces, called voxels, that are low density and have high stiffness. The deformable robots are made with lattice-like pieces, called voxels, that are low density and have high stiffness. Credit: Courtesy of the researchers

“Scalability is a strong point for us,” says Parra Rubio. Given the low density and high stiffness of the lattice-like pieces, called voxels, that make up their system, he says, “we have more room to keep scaling up,” whereas most currently used technologies “rely on high-density materials facing drastic problems” in moving to larger sizes.

The individual voxels in the team’s experimental, proof-of-concept devices are mostly hollow structures made up of cast plastic pieces with narrow struts in complex shapes. The box-like shapes are load-bearing in one direction but soft in others, an unusual combination achieved by blending stiff and flexible components in different proportions.

“Treating soft versus hard robotics is a false dichotomy,” Parra Rubio says. “This is something in between, a new way to construct things.” Gershenfeld, head of MIT’s Center for Bits and Atoms, adds that “this is a third way that marries the best elements of both.”

“Smooth flexibility of the body surface allows us to implement flow control that can reduce drag and improve propulsive efficiency, resulting in substantial fuel saving,” says Triantafyllou, who is the Henry L. and Grace Doherty Professor in Ocean Science and Engineering, and was part of the RoboTuna team.

Credit: Courtesy of the researchers.

In one of the devices produced by the team, the voxels are attached end-to-end in a long row to form a meter-long, snake-like structure. The body is made up of four segments, each consisting of five voxels, with an actuator in the center that can pull a wire attached to each of the two voxels on either side, contracting them and causing the structure to bend. The whole structure of 20 units is then covered with a rib-like supporting structure, and then a tight-fitting waterproof neoprene skin. The researchers deployed the structure in an MIT tow tank to show its efficiency in the water, and demonstrated that it was indeed capable of generating forward thrust sufficient to propel itself forward using undulating motions.

“There have been many snake-like robots before,” Gershenfeld says. “But they’re generally made of bespoke components, as opposed to these simple building blocks that are scalable.”

For example, Parra Rubio says, a snake-like robot built by NASA was made up of thousands of unique pieces, whereas for this group’s snake, “we show that there are some 60 pieces.” And compared to the two years spent designing and building the MIT RoboTuna, this device was assembled in about two days, he says.

The individual voxels are mostly hollow structures made up of cast plastic pieces with narrow struts in complex shapes. Credit: Courtesy of the researchers

The other device they demonstrated is a wing-like shape, or hydrofoil, made up of an array of the same voxels but able to change its profile shape and therefore control the lift-to-drag ratio and other properties of the wing. Such wing-like shapes could be used for a variety of purposes, ranging from generating power from waves to helping to improve the efficiency of ship hulls — a pressing demand, as shipping is a significant source of carbon emissions.

The wing shape, unlike the snake, is covered in an array of scale-like overlapping tiles, designed to press down on each other to maintain a waterproof seal even as the wing changes its curvature. One possible application might be in some kind of addition to a ship’s hull profile that could reduce the formation of drag-inducing eddies and thus improve its overall efficiency, a possibility that the team is exploring with collaborators in the shipping industry.

The team also created a wing-like hydrofoil. Credit: Courtesy of the researchers

Ultimately, the concept might be applied to a whale-like submersible craft, using its morphable body shape to create propulsion. Such a craft that could evade bad weather by staying below the surface, but without the noise and turbulence of conventional propulsion. The concept could also be applied to parts of other vessels, such as racing yachts, where having a keel or a rudder that could curve gently during a turn instead of remaining straight could provide an extra edge. “Instead of being rigid or just having a flap, if you can actually curve the way fish do, you can morph your way around the turn much more efficiently,” Gershenfeld says.

The research team included Dixia Fan of the Westlake University in China; Benjamin Jenett SM ’15, PhD ’ 20 of Discrete Lattice Industries; Jose del Aguila Ferrandis, Amira Abdel-Rahman and David Preiss of MIT; and Filippos Tourlomousis of the Demokritos Research Center of Greece. The work was supported by the U.S. Army Research Lab, CBA Consortia funding, and the MIT Sea Grant Program.

• PAPER – Modular Morphing Lattices for Large-Scale Underwater Continuum Robotic Structures. Alfonso Parra Rubio, Dixia Fan, Benjamin Jenett, José del Águila Ferrandis, Filippos Tourlomousis, Amira Abdel-Rahman, David Preiss, Michael Triantafyllou, and Neil Gershenfeld. Soft Robotics (2023).

Prof. Hae-Won Park (left), Ph.D. Student Yong Um (centre), Ph.D. Student Seungwoo Hong (right). Credits: KAIST

We had the chance to interview Hae-Won Park, Seungwoo Hong and Yong Um, authors of the paper “Agile and versatile climbing on ferromagnetic surfaces with a quadrupedal robot”, recently published in Science Robotics.

What is the topic of the research in your paper?
The main topic of our work is that the robot we have developed can move agilely, not only on flat ground but also on vertical walls and ceilings made of ferromagnetic materials. Also, it has the ability to perform dexterous maneuvers such as crossing gaps, overcoming obstacles, and transitioning upon corners.

Could you tell us about the implications of your research and why it is an interesting area for study?
Such agile and dexterous locomotion capabilities will be able to expand the robot’s operational workspace and approach places that are difficult or dangerous for human operators to access directly. For example, inspection and welding operations in heavy industries such as shipbuilding, steel bridges, and storage tanks.

Could you explain your methodology? What were your main findings?
Our magnet foot can switch the on/off state in a short period of time (5 ms) and in an energy-efficient way, thanks to the novel geometry design of EPM. At the same time, the magnet foot can provide large holding forces in both shear and normal directions due to the MRE footpad. Also, our actuators can provide balanced speed/torque characteristics, high-bandwidth torque control capability, and the ability to mediate high impulsive force. To control vertical and inverted locomotion as well as various versatile motions, we have utilized a control framework (model predictive control) that can generate reliable and robust reaction forces to track desired body motions in 3D space while preventing slippage or tipping-over occurs. We found that all the elements mentioned earlier are imperative to perform dynamic maneuvers against gravity.

What further work are you planning in this area?
So far, the robot is able to move on smooth surfaces with moderate curvature. To enable the robot to move on irregularly shaped surfaces, we are working on designing a compliantly-integrated multiple miniaturized EPMs with MRE footpads that can increase the effective contact area to provide robust adhesion. Also, a vision system with high-level navigation algorithms will be included to enable the robot to move autonomously in the near future.

About the authors

Hae-Won Park received the B.S. and M.S. degrees from Yonsei University, Seoul, South Korea, in 2005 and 2007, respectively, and the Ph.D. degree from the University of Michigan, Ann Arbor, MI, USA, in 2012, all in mechanical engineering. He is an Associate Professor of mechanical engineering with the Korea Advanced Institute of Science and Technology, Daejeon, South Korea. His research interests include the intersection of control, dynamics, and mechanical design of robotic systems, with special emphasis on legged locomotion robots. Dr. Park is the recipient of the 2018 National Science Foundation (NSF) CAREER Award and NSF most prestigious awards in support of early-career faculty.

Seungwoo Hong received the B.S. degree from Shanghai Jiao Tong University, Shanghai, China, in July 2014, and the M.S. degree from Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in August 2017, all in mechanical engineering. He is currently a Ph.D. candidate with the Department of Mechanical Engineering, KAIST, Daejeon, Korea. His current research interests include model-based optimization, motion planning and control of legged robotic systems.

Yong Um received the B.S. degree in mechanical engineering from the Korea Advanced Institute of Science and Technology, Daejeon, South Korea, in 2020. He is currently working toward the Ph.D. degree in mechanical engineering in Korea Advanced Institute of Science and Technology. His research interests include mechanical system and magnetic device design for legged robot.

• PAPER – Agile and versatile climbing on ferromagnetic surfaces with a quadrupedal robot. Seungwoo Hong, Yong Um, Jaejun Park, and Hae-Won Park. Science Robotics 7.73 (2022): eadd1017.

Elvis Nava is a fellow at ETH’ Zurich’s AI center as well as a doctoral student at the Institute of Neuroinformatics and in the Soft Robotics Lab. (Photograph: Daniel Winkler / ETH Zurich)

By Christoph Elhardt

In ETH Zurich’s Soft Robotics Lab, a white robot hand reaches for a beer can, lifts it up and moves it to a glass at the other end of the table. There, the hand carefully tilts the can to the right and pours the sparkling, gold-coloured liquid into the glass without spilling it. Cheers!

Computer scientist Elvis Nava is the person controlling the robot hand developed by ETH start-up Faive Robotics. The 26-year-old doctoral student’s own hand hovers over a surface equipped with sensors and a camera. The robot hand follows Nava’s hand movement. When he spreads his fingers, the robot does the same. And when he points at something, the robot hand follows suit.

But for Nava, this is only the beginning: “We hope that in future, the robot will be able to do something without our having to explain exactly how,” he says. He wants to teach machines to carry out written and oral commands. His goal is to make them so intelligent that they can quickly acquire new abilities, understand people and help them with different tasks.

Functions that currently require specific instructions from programmers will then be controlled by simple commands such as “pour me a beer” or “hand me the apple”. To achieve this goal, Nava received a doctoral fellowship from ETH Zurich’s AI Center in 2021: this program promotes talents that bridges different research disciplines to develop new AI applications. In addition, the Italian – who grew up in Bergamo – is doing his doctorate at Benjamin Grewe’s professorship of neuroinformatics and in Robert Katzschmann’s lab for soft robotics.

Developed by the ETH start-​up Faive Robotics, the robot hand imitates the movements of a human hand. (Video: Faive Robotics)

Combining sensory stimuli

But how do you get a machine to carry out commands? What does this combination of artificial intelligence and robotics look like? To answer these questions, it is crucial to understand the human brain.

We perceive our environment by combining different sensory stimuli. Usually, our brain effortlessly integrates images, sounds, smells, tastes and haptic stimuli into a coherent overall impression. This ability enables us to quickly adapt to new situations. We intuitively know how to apply acquired knowledge to unfamiliar tasks.

“Computers and robots often lack this ability,” Nava says. Thanks to machine learning, computer programs today may write texts, have conversations or paint pictures, and robots may move quickly and independently through difficult terrain, but the underlying learning algorithms are usually based on only one data source. They are – to use a computer science term – not multimodal.

For Nava, this is precisely what stands in the way of more intelligent robots: “Algorithms are often trained for just one set of functions, using large data sets that are available online. While this enables language processing models to use the word ‘cat’ in a grammatically correct way, they don’t know what a cat looks like. And robots can move effectively but usually lack the capacity for speech and image recognition.”

“Every couple of years, our discipline changes the way we think about what it means to be a researcher,” Elvis Nava says. (Video: ETH AI Center)

Robots have to go to preschool

This is why Nava is developing learning algorithms for robots that teach them exactly that: to combine information from different sources. “When I tell a robot arm to ‘hand me the apple on the table,’ it has to connect the word ‘apple’ to the visual features of an apple. What’s more, it has to recognise the apple on the table and know how to grab it.”

But how does the Nava teach the robot arm to do all that? In simple terms, he sends it to a two-stage training camp. First, the robot acquires general abilities such as speech and image recognition as well as simple hand movements in a kind of preschool.

Open-source models that have been trained using giant text, image and video data sets are already available for these abilities. Researchers feed, say, an image recognition algorithm with thousands of images labelled ‘dog’ or ‘cat.’ Then, the algorithm learns independently what features – in this case pixel structures – constitute an image of a cat or a dog.

A new learning algorithm for robots

Nava’s job is to combine the best available models into a learning algorithm, which has to translate different data, images, texts or spatial information into a uniform command language for the robot arm. “In the model, the same vector represents both the word ‘beer’ and images labelled ‘beer’,” Nava says. That way, the robot knows what to reach for when it receives the command “pour me a beer”.

Researchers who deal with artificial intelligence on a deeper level have known for a while that integrating different data sources and models holds a lot of promise. However, the corresponding models have only recently become available and publicly accessible. What’s more, there is now enough computing power to get them up and running in tandem as well.

When Nava talks about these things, they sound simple and intuitive. But that’s deceptive: “You have to know the newest models really well, but that’s not enough; sometimes getting them up and running in tandem is an art rather than a science,” he says. It’s tricky problems like these that especially interest Nava. He can work on them for hours, continuously trying out new solutions.

Nava spends the majority of his time coding. (Photograph: Elvis Nava)

Nava evaluates his learning algorithm. The results of the experiment in a nutshell. (Photograph: Elvis Nava)

Special training: Imitating humans

Once the robot arm has completed preschool and has learnt to understand speech, recognise images and carry out simple movements, Nava sends it to special training. There, the machine learns to, say, imitate the movements of a human hand when pouring a glass of beer. “As this involves very specific sequences of movements, existing models no longer suffice,” Nava says.

Instead, he shows his learning algorithm a video of a hand pouring a glass of beer. Based on just a few examples, the robot then tries to imitate these movements, drawing on what it has learnt in preschool. Without prior knowledge, it simply wouldn’t be able to imitate such a complex sequence of movements.

“If the robot manages to pour the beer without spilling, we tell it ‘well done’ and it memorises the sequence of movements,” Nava says. This method is known as reinforcement learning in technical jargon.

Elvis Nava teaches robots to carry out oral commands such as “pour me a beer”. (Photograph: Daniel Winkler / ETH Zürich)

Foundations for robotic helpers

With this two-stage learning strategy, Nava hopes to get a little closer to realising the dream of creating an intelligent machine. How far it will take him, he does not yet know. “It’s unclear whether this approach will enable robots to carry out tasks we haven’t shown them before.”

It is much more probable that we will see robotic helpers that carry out oral commands and fulfil tasks they are already familiar with or that closely resemble them. Nava avoids making predictions as to how long it will take before these applications can be used in areas such as the care sector or construction.

Developments in the field of artificial intelligence are too fast and unpredictable. In fact, Nava would be quite happy if the robot would just hand him the beer he will politely request after his dissertation defence.

Growing up in Rhode Island (the Ocean State), I lived very close to the water. Over the years, I have seen the effects of sea level rise and rapid erosion. Entire houses and beaches have slowly been consumed by the tide. I have witnessed first hand how climate change is rapidly changing the ocean ecosystem. Sometimes I feel overwhelmed by the inexorability of climate change. What can we do in the face of such a global, almost incomprehensible dilemma? The only way I can overcome this perception is by committing to doing something with my life to help, even if it’s in a small way. I think with such a big issue, the only way forward is by starting small, identifying one niche I can work in, and seeing how I can shape my research around solving that challenge.

One major challenge is rapid global ocean temperature rise. When scientists look to make climate associations using temperature data, they generally use fixed temperature loggers attached to buoys or on the ocean floor. Unfortunately, this approach discounts the area between the ocean’s surface and floor. Variable ocean conditions create microclimates, pockets of the ocean that are unaffected by general climate trends. Scientists have shown that most organisms experience climate change via these microclimates. Fish are greatly affected by this rapid increase in temperature as they can only lay eggs in a minimal range of temperatures. Microclimates are changing temperature with celerity. Hence, many species cannot adapt quickly enough to survive. At this rate, 60% of fish species could go extinct by 2100.

Of course, fish are not the only organisms affected by the rapid increase in temperature. Coral in the Great Barrier Reef can only survive in a minimal temperature threshold, and as temperature increases, reefs are experiencing mass coral bleaching. AIMS, the Australian Institute for Marine Science, the government agency that monitors the Great Barrier Reef, utilizes divers pulled behind boats to record reef observations and collect data. Unfortunately, this has led to some casualties due to shark attacks. They have begun deploying large, almost seven feet in length, ocean gliders that can mitigate this risk. These robots come with a hefty price tag of $125,000 to $500,000. They are also too large to navigate portions of the reef.

Our solution in the Soft Robotics Lab at Worcester Polytechnic Institute is building a free-swimming (tetherless), biologically inspired robotic fish, funded in part by the National Science Foundation Future of Robots in the Workplace Research and Development Program. Our goal is for the robot to navigate the complex environment of the Great Barrier Reef and record dense three-dimensional temperature data throughout the water column. Moreover, we will use non-hazardous and affordable material for the fish’s body. Since our motivation is to create a tool to use in climate research, a robot that is cheap and easy to manufacture will increase its effectiveness. Our approach is in stark contrast to traditional autonomous underwater vehicles that utilize propellers that are noisy and incongruous to underwater life. We chose to mimic the motion of real fish to reduce the environmental impact of our robot and enable close observation of other real fish.

We are, of course, not the first people to build a robot fish. In 1994, MIT produced the RoboTuna, a fully rigid fish robot, and since then, there have been many different iterations of fish robots. Some have been made of fully rigid materials like the RoboTuna and used motors that run the caudal tail (rear fin) actuation that powers the fish. However, this does not replicate the fluid motion achieved by real fish as they swim. A possible solution would be to use soft materials. Designs using soft materials, up to this point, utilize a silicone, pneumatically or hydraulically actuated tail. Unfortunately, these robots cannot operate in rough environments since any cuts or abrasions to the silicone could cause a leak in the system and lead to a total failure in the actuation of the tail. Other robots have combined the more durable rigid materials, actuated with cables, and then attached a soft silicone end that bends with the force of the water. All these previous robots are difficult to manufacture and require institutional knowledge to recreate.

MIT Robotuna and MIT SOFI robots

We have fabricated a 3D printed, cable-actuated wave spring tail made from soft materials that can drive a small robot fish. The wave spring gives the robot its biologically inspired shape, but it can bend fluidly like the silicone-based robots and real fish. The wave springis entirely 3D printed from a flexible material that is affordable and easy to use. This material and method creates a very soft yet durable robot, withstands harsh treatment, and runs for hundreds of thousands of cycles without any degradation to any of the robot’s systems. The robot sets itself apart by being very easy to assemble, with only a handful of parts, most of which can be 3D printed.

The wave spring itself has a biologically inspired design. Reef fish are morphologically diverse but share a similar body shape which we emulate with a tapered oval design. The wave spring itself is composed of a mesh of diamond-shaped cells that can compress and bend. To restrict our robot to only lateral bending, we added supports down the dorsal and ventral edges of the wave spring.

Using this design, we have successfully created a robot fish. The robot is able to swim freely in a fish tank, swimming pool, and in a lake. While testing the fish in these environments, we found that the speed and performance of our robot was comparable to other fish robots operating under similar parameters. In order to waterproof the robot (to protect the electronics required for tetherless swimming), we had to add a latex skin. This does increase the manufacturing complexity of the design, so we will look to improve not only the robot’s performance, but also its design to ensure a simplistic yet high functioning robot.

Most importantly, we will add the sensors required to collect data like temperature, which is imperative to a better understanding of the oceans’ rapidly changing microclimates. It’s crucial that we remain focused on this goal, as it drives not only the robot’s design, but our motivation for why we do this work. Climate change is the foremost crisis facing our world. I encourage everyone to connect their interests and work, no matter the field, in some way to this issue as we are the only ones who can do something about it.

With just a random disturbance that energizes the cubes, they selectively self-assemble into a larger block. Photos courtesy of MIT CSAIL.

By Rachel Gordon | MIT CSAIL

While automated manufacturing is ubiquitous today, it was once a nascent field birthed by inventors such as Oliver Evans, who is credited with creating the first fully automated industrial process, in flour mill he built and gradually automated in the late 1700s. The processes for creating automated structures or machines are still very top-down, requiring humans, factories, or robots to do the assembling and making. 

However, the way nature does assembly is ubiquitously bottom-up; animals and plants are self-assembled at a cellular level, relying on proteins to self-fold into target geometries that encode all the different functions that keep us ticking. For a more bio-inspired, bottom-up approach to assembly, then, human-architected materials need to do better on their own. Making them scalable, selective, and reprogrammable in a way that could mimic nature’s versatility means some teething problems, though. 

Now, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have attempted to get over these growing pains with a new method: introducing magnetically reprogrammable materials that they coat different parts with — like robotic cubes — to let them self-assemble. Key to their process is a way to make these magnetic programs highly selective about what they connect with, enabling robust self-assembly into specific shapes and chosen configurations. 

The soft magnetic material coating the researchers used, sourced from inexpensive refrigerator magnets, endows each of the cubes they built with a magnetic signature on each of its faces. The signatures ensure that each face is selectively attractive to only one other face from all the other cubes, in both translation and rotation. All of the cubes — which run for about 23 cents — can be magnetically programmed at a very fine resolution. Once they’re tossed into a water tank (they used eight cubes for a demo), with a totally random disturbance — you could even just shake them in a box — they’ll bump into each other. If they meet the wrong mate, they’ll drop off, but if they find their suitable mate, they’ll attach. 

An analogy would be to think of a set of furniture parts that you need to assemble into a chair. Traditionally, you’d need a set of instructions to manually assemble parts into a chair (a top-down approach), but using the researchers’ method, these same parts, once programmed magnetically, would self-assemble into the chair using just a random disturbance that makes them collide. Without the signatures they generate, however, the chair would assemble with its legs in the wrong places.

“This work is a step forward in terms of the resolution, cost, and efficacy with which we can self-assemble particular structures,” says Martin Nisser, a PhD student in MIT’s Department of Electrical Engineering and Computer Science (EECS), an affiliate of CSAIL, and the lead author on a new paper about the system. “Prior work in self-assembly has typically required individual parts to be geometrically dissimilar, just like puzzle pieces, which requires individual fabrication of all the parts. Using magnetic programs, however, we can bulk-manufacture homogeneous parts and program them to acquire specific target structures, and importantly, reprogram them to acquire new shapes later on without having to refabricate the parts anew.” 

Using the team’s magnetic plotting machine, one can stick a cube back in the plotter and reprogram it. Every time the plotter touches the material, it creates either a “north”- or “south”-oriented magnetic pixel on the cube’s soft magnetic coating, letting the cubes be repurposed to assemble new target shapes when required. Before plotting, a search algorithm checks each signature for mutual compatibility with all previously programmed signatures to ensure they are selective enough for successful self-assembly.

With self-assembly, you can go the passive or active route. With active assembly, robotic parts modulate their behavior online to locate, position, and bond to their neighbors, and each module needs to be embedded with hardware for the computation, sensing, and actuation required to self-assemble themselves. What’s more, a human or computer is needed in the loop to actively control the actuators embedded in each part to make it move. While active assembly has been successful in reconfiguring a variety of robotic systems, the cost and complexity of the electronics and actuators have been a significant barrier to scaling self-assembling hardware up in numbers and down in size. 

With passive methods like these researchers’, there’s no need for embedded actuation and control.

Once programmed and set free under a random disturbance that gives them the energy to collide with one another, they’re on their own to shapeshift, without any guiding intelligence.  

If you want a structure built from hundreds or thousands of parts, like a ladder or bridge, for example, you wouldn’t want to manufacture a million uniquely different parts, or to have to re-manufacture them when you need a second structure assembled.

The trick the team used toward this goal lies in the mathematical description of the magnetic signatures, which describes each signature as a 2D matrix of pixels. These matrices ensure that any magnetically programmed parts that shouldn’t connect will interact to produce just as many pixels in attraction as those in repulsion, letting them remain agnostic to all non-mating parts in both translation and rotation. 

While the system is currently good enough to do self-assembly using a handful of cubes, the team wants to further develop the mathematical descriptions of the signatures. In particular, they want to leverage design heuristics that would enable assembly with very large numbers of cubes, while avoiding computationally expensive search algorithms. 

“Self-assembly processes are ubiquitous in nature, leading to the incredibly complex and beautiful life we see all around us,” says Hod Lipson, the James and Sally Scapa Professor of Innovation at Columbia University, who was not involved in the paper. “But the underpinnings of self-assembly have baffled engineers: How do two proteins destined to join find each other in a soup of billions of other proteins? Lacking the answer, we have been able to self-assemble only relatively simple structures so far, and resort to top-down manufacturing for the rest. This paper goes a long way to answer this question, proposing a new way in which self-assembling building blocks can find each other. Hopefully, this will allow us to begin climbing the ladder of self-assembled complexity.”

Nisser wrote the paper alongside recent EECS graduates Yashaswini Makaram ’21 and Faraz Faruqi SM ’22, both of whom are former CSAIL affiliates; Ryo Suzuki, assistant professor of computer science at the University of Calgary; and MIT associate professor of EECS Stefanie Mueller, who is a CSAIL affiliate. They will present their research at the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022).

• PAPER – Selective Self-Assembly using Re-Programmable Magnetic Pixels. Martin Nisser, Yashaswini Makaram, Faraz Faruqi1 Ryo Suzuki, and Stefanie Mueller.

MIT chemical engineers have shown that specialized particles can oscillate together, demonstrating a phenomenon known as emergent behavior. Image: Courtesy of the researchers

By Anne Trafton | MIT News Office

Taking advantage of a phenomenon known as emergent behavior in the microscale, MIT engineers have designed simple microparticles that can collectively generate complex behavior, much the same way that a colony of ants can dig tunnels or collect food.

Working together, the microparticles can generate a beating clock that oscillates at a very low frequency. These oscillations can then be harnessed to power tiny robotic devices, the researchers showed.

“In addition to being interesting from a physics point of view, this behavior can also be translated into an on-board oscillatory electrical signal, which can be very powerful in microrobotic autonomy. There are a lot of electrical components that require such an oscillatory input,” says Jingfan Yang, a recent MIT PhD recipient and one of the lead authors of the new study.

The particles used to create the new oscillator perform a simple chemical reaction that allows the particles to interact with each other through the formation and bursting of tiny gas bubbles. Under the right conditions, these interactions create an oscillator that behaves similar to a ticking clock, beating at intervals of a few seconds.

“We’re trying to look for very simple rules or features that you can encode into relatively simple microrobotic machines, to get them to collectively do very sophisticated tasks,” says Michael Strano, the Carbon P. Dubbs Professor of Chemical Engineering at MIT.

Strano is the senior author of the new paper, which appears in Nature Communications. Along with Yang, Thomas Berrueta, a Northwestern University graduate student advised by Professor Todd Murphey, is a lead author of the study.

Collective behavior

Demonstrations of emergent behavior can be seen throughout the natural world, where colonies of insects such as ants and bees accomplish feats that a single member of the group would never be able to achieve.

“Ants have minuscule brains and they do very simple cognitive tasks, but collectively they can do amazing things. They can forage for food and build these elaborate tunnel structures,” Strano says. “Physicists and engineers like myself want to understand these rules because it means we can make tiny things that collectively do complex tasks.”

In this study, the researchers wanted to design particles that could generate rhythmic movements, or oscillations, with a very low frequency. Until now, building low-frequency micro-oscillators has required sophisticated electronics that are expensive and difficult to design, or specialized materials with complex chemistries.

The simple particles that the researchers designed for this study are discs as small as 100 microns in diameter. The discs, made from a polymer called SU-8, have a platinum patch that can catalyze the breakdown of hydrogen peroxide into water and oxygen.

When the particles are placed at the surface of a droplet of hydrogen peroxide on a flat surface, they tend to travel to the top of the droplet. At this liquid-air interface, they interact with any other particles found there. Each particle produces its own tiny bubble of oxygen, and when two particles come close enough that their bubbles interact, the bubbles pop, propelling the particles away from each other. Then, they begin forming new bubbles, and the cycle repeats over and over.

“One particle by itself stays still and doesn’t do anything interesting, but through teamwork, they can do something pretty amazing and useful, which is actually a difficult thing to achieve at the microscale,” Yang says.

MIT chemical engineers showed that specialized particles can oscillate together, demonstrating a phenomenon known as emergent behavior. At left, two particles oscillate together, and at right, eight particles. Video courtesy of the researchers.

The researchers found that two particles could make a very reliable oscillator, but as more particles were added, the rhythm would get thrown off. However, if they added one particle that was slightly different from the others, that particle could act as a “leader” that reorganized the other particles back into a rhythmic oscillator.

This leader particle is the same size as the other particles but has a slightly larger platinum patch, which enables it to create a larger oxygen bubble. This allows this particle to move to the center of the group, where it coordinates the oscillations of all of the other particles. Using this approach, the researchers found they could create oscillators containing up to at least 11 particles.

Depending on the number of particles, this oscillator beats at a frequency of about 0.1 to 0.3 hertz, which is on the order of the low-frequency oscillators that govern biological functions such as walking and the beating of the heart.

Oscillating current

The researchers also showed that they could use the rhythmic beating of these particles to generate an oscillating electric current. To do that, they swapped out the platinum catalyst for a fuel cell made of platinum and ruthenium or gold. The mechanical oscillation of the particles rhythmically alters the resistance from one end of the fuel cell to the other, which converts the voltage generated by the fuel cell to an oscillating current.

“Like a dripping faucet, catalytic microdiscs floating at a liquid interface use a chemical reaction to drive the periodic growth and release of gas bubbles. The study shows how these oscillatory dynamics can be harnessed for mechanical actuation and electrochemical signaling relevant to microrobotics,” says Kyle Bishop, a professor of chemical engineering at Columbia University, who was not involved in the study.

Generating an oscillating current instead of a constant one could be useful for applications such as powering tiny robots that can walk. The MIT researchers used this approach to show that they could power a microactuator, which was previously used as legs on a tiny walking robot developed by researchers at Cornell University. The original version was powered by a laser that had to be alternately pointed at each set of legs, to manually oscillate the current. The MIT team showed that the on-board oscillating current generated by their particles could drive the cyclic actuation of the microrobotic leg, using a wire to transfer the current from the particles to the actuator.

“It shows that this mechanical oscillation can become an electrical oscillation, and then that electrical oscillation can actually power activities that a robot would do,” Strano says.

One possible application for this kind of system would be to control swarms of tiny autonomous robots that could be used as sensors to monitor water pollution.

The research was funded in part by the U.S. Army Research Office, the U.S. Department of Energy, and the National Science Foundation.

A battery-free, wireless underwater camera developed at MIT could have many uses, including climate modeling. “We are missing data from over 95 percent of the ocean. This technology could help us build more accurate climate models and better understand how climate change impacts the underwater world,” says Associate Professor Fadel Adib. Image: Adam Glanzman

By Adam Zewe | MIT News Office

Scientists estimate that more than 95 percent of Earth’s oceans have never been observed, which means we have seen less of our planet’s ocean than we have the far side of the moon or the surface of Mars.

The high cost of powering an underwater camera for a long time, by tethering it to a research vessel or sending a ship to recharge its batteries, is a steep challenge preventing widespread undersea exploration.

MIT researchers have taken a major step to overcome this problem by developing a battery-free, wireless underwater camera that is about 100,000 times more energy-efficient than other undersea cameras. The device takes color photos, even in dark underwater environments, and transmits image data wirelessly through the water.

The autonomous camera is powered by sound. It converts mechanical energy from sound waves traveling through water into electrical energy that powers its imaging and communications equipment. After capturing and encoding image data, the camera also uses sound waves to transmit data to a receiver that reconstructs the image. 

Because it doesn’t need a power source, the camera could run for weeks on end before retrieval, enabling scientists to search remote parts of the ocean for new species. It could also be used to capture images of ocean pollution or monitor the health and growth of fish raised in aquaculture farms.

“One of the most exciting applications of this camera for me personally is in the context of climate monitoring. We are building climate models, but we are missing data from over 95 percent of the ocean. This technology could help us build more accurate climate models and better understand how climate change impacts the underwater world,” says Fadel Adib, associate professor in the Department of Electrical Engineering and Computer Science and director of the Signal Kinetics group in the MIT Media Lab, and senior author of a new paper on the system.

Joining Adib on the paper are co-lead authors and Signal Kinetics group research assistants Sayed Saad Afzal, Waleed Akbar, and Osvy Rodriguez, as well as research scientist Unsoo Ha, and former group researchers Mario Doumet and Reza Ghaffarivardavagh. The paper is published in Nature Communications.

Going battery-free

To build a camera that could operate autonomously for long periods, the researchers needed a device that could harvest energy underwater on its own while consuming very little power.

The camera acquires energy using transducers made from piezoelectric materials that are placed around its exterior. Piezoelectric materials produce an electric signal when a mechanical force is applied to them. When a sound wave traveling through the water hits the transducers, they vibrate and convert that mechanical energy into electrical energy.

Those sound waves could come from any source, like a passing ship or marine life. The camera stores harvested energy until it has built up enough to power the electronics that take photos and communicate data.

To keep power consumption as a low as possible, the researchers used off-the-shelf, ultra-low-power imaging sensors. But these sensors only capture grayscale images. And since most underwater environments lack a light source, they needed to develop a low-power flash, too.

“We were trying to minimize the hardware as much as possible, and that creates new constraints on how to build the system, send information, and perform image reconstruction. It took a fair amount of creativity to figure out how to do this,” Adib says.

They solved both problems simultaneously using red, green, and blue LEDs. When the camera captures an image, it shines a red LED and then uses image sensors to take the photo. It repeats the same process with green and blue LEDs.

Even though the image looks black and white, the red, green, and blue colored light is reflected in the white part of each photo, Akbar explains. When the image data are combined in post-processing, the color image can be reconstructed.

“When we were kids in art class, we were taught that we could make all colors using three basic colors. The same rules follow for color images we see on our computers. We just need red, green, and blue — these three channels — to construct color images,” he says.

Fadel Adib (left) associate professor in the Department of Electrical Engineering and Computer Science and director of the Signal Kinetics group in the MIT Media Lab, and Research Assistant Waleed Akbar display the battery-free wireless underwater camera that their group developed. Image: Adam Glanzman

Sending data with sound

Once image data are captured, they are encoded as bits (1s and 0s) and sent to a receiver one bit at a time using a process called underwater backscatter. The receiver transmits sound waves through the water to the camera, which acts as a mirror to reflect those waves. The camera either reflects a wave back to the receiver or changes its mirror to an absorber so that it does not reflect back.

A hydrophone next to the transmitter senses if a signal is reflected back from the camera. If it receives a signal, that is a bit-1, and if there is no signal, that is a bit-0. The system uses this binary information to reconstruct and post-process the image.

“This whole process, since it just requires a single switch to convert the device from a nonreflective state to a reflective state, consumes five orders of magnitude less power than typical underwater communications systems,” Afzal says.

The researchers tested the camera in several underwater environments. In one, they captured color images of plastic bottles floating in a New Hampshire pond. They were also able to take such high-quality photos of an African starfish that tiny tubercles along its arms were clearly visible. The device was also effective at repeatedly imaging the underwater plant Aponogeton ulvaceus in a dark environment over the course of a week to monitor its growth.

Now that they have demonstrated a working prototype, the researchers plan to enhance the device so it is practical for deployment in real-world settings. They want to increase the camera’s memory so it could capture photos in real-time, stream images, or even shoot underwater video.

They also want to extend the camera’s range. They successfully transmitted data 40 meters from the receiver, but pushing that range wider would enable the camera to be used in more underwater settings.

“This will open up great opportunities for research both in low-power IoT devices as well as underwater monitoring and research,” says Haitham Al-Hassanieh, an assistant professor of electrical and computer engineering at the University of Illinois Urbana-Champaign, who was not involved with this research.

This research is supported, in part, by the Office of Naval Research, the Sloan Research Fellowship, the National Science Foundation, the MIT Media Lab, and the Doherty Chair in Ocean Utilization.

• PAPER – Battery-free wireless imaging of underwater environments. Afzal, S.S., Akbar, W., Rodriguez, O. et al. Nat Commun 13, 5546 (2022). https://doi.org/10.1038

Robots that move about by themselves must be able to adapt to the dynamic and challenging conditions in a supermarket. Hernández Corbato says: “My research focuses on using artificial intelligence to make machines smarter and more reliable by teaching them symbolic knowledge. The goal is to develop robotic ‘brains’ for intelligent robots that can be trusted to work alongside people, because they can explain their decisions.”

A supermarket is typically a place where unexpected things happen all the time. Not only are there thousands of products with different shapes and looks, there are also people walking in and out. How can an independently operating machine handle this safely, efficiently and intelligently? By activating symbolic knowledge that we humans also use, says Hernández. “We recognize a tray with four legs underneath as a symbol: ‘table’. We don’t need a photo for it. When we encode such ‘symbol language’ and make it suitable for robots, they can perform more complex tasks.”

One look with its camera eyes and the robot knows it is facing an object on which plates and cups can be placed. Based on this, it can decide what to do. The acceleration that this produces, should enable robots to perform multiple actions at the same time: navigate, pick up and move objects, and ultimately communicate with people.

Symbolic knowledge

For Hernández, the AI for Retail Lab research program of supermarket chain Ahold Delhaize brings together everything that fascinates him about artificial intelligence. Retail requires robots to use a broad diversity of skills: to perceive the environment, navigate around it, manipulate objects or collaborate with humans. For him, it’s all about the question of which algorithms are needed to make a machine respond just as intelligently as a human brain. As a specialist in software for autonomously operating robots, he already won the Amazon Picking Challenge in 2016 with a team from TU Delft. At this occasion a robotic arm placed products from a container in their place on a shelf.

“Retail requires robots to use a broad diversity of skills: to perceive the environment, navigate around it, manipulate objects or collaborate with humans.”

– Carlos Hernández Corbato, researcher

The ‘supermarket robot’ is even more challenging. It requires the leap from a static factory environment to the dynamics of a store. The traditional way, in which robots learn from the data they collect, is too cumbersome for that. The robot would already get stuck in stock management. Programming a customized robot treatment for every orange, bottle, soup can, milk carton or cucumber would be too much work. “We want to inject symbolic knowledge into the robot’s operating system all at once,” says Hernández. “If that knowledge is available, the robot can continuously adapt to his changing environment. For example, by downloading a different hand movement.”

The robot must be able to independently choose a different algorithm if it encounters a problem along the way. So that he can pick up a can that falls from his hands, or change the grip of his hand slightly when picking up an unknown object. The technicians have already set up a test shop where robot ‘Tiago’ can practice with it. In about five years’ time, it should deliver a machine with a mobile base, two arms and two camera eyes, which independently refills supermarket shelves 24 hours a day. And it must be able to do that under all circumstances, day and night.

Sensor broken

The latter does not only apply to the supermarket robot. In fact, every robot should have a next generation operating system to better cope with changing circumstances. Hernández Corbato:  “Beyond integrating different robot skills, cognitive skills for robots need to enable them to reason about those skills, to understand how they can use them, and what are the consequences of their own actions. In sum, we need to endow robots (or any intelligent autonomous system build) with self-awareness so that we can trust them.”

“We need to endow robots (or any intelligent autonomous system build) with self-awareness so that we can trust them.”

– Carlos Hernández Corbato, researcher

It is the core idea behind the European project Metacontrol for ROS2 systems (MROS) that the Cognitive Robotics department recently completed. The AI technique that Hernández used for this is called the metacontrol method. It describes the properties and skills of the robot in a structured way, so that the robot can use the knowledge to adapt and overcome problems.

As part of this research, he developed multiple prototypes of these next generation robots together with Bosch Corporate Research, Universidad Rey Juan Carlos, Universidad Politecnica de Madrid and IT University in Copenhagen.

Does it perform better than traditional robots? “Yes, he navigated more safely and, thanks to its symbolic knowledge, was able to adapt to the circumstances. When one sensor broke, it switched to another independently,” says Hernández enthusiastically. “That is where we want to go to: a robot with sufficient intelligence to deal with failures.”

The post Shelf-stocking robots with independent movement appeared first on RoboHouse.

Research team : Maxence LEVEZIEL, Wissem HAOUAS, Michaël GAUTHIER, Guillaume J. LAURENT, Redwan DAHMOUCHE.

By Redwan Dahmouche

Speed and precision are two major issues in robotics and in Industry of the Future (also known as Industry 4.0). Within this framework, RoMoCo research team of AS2M department at FEMTO-ST Institute has developed MiGriBot, a miniature robot able to perform 720 pick-and-place operations of sub-millimeter objects per minute. The results of this research work have been published in Science Robotics.

These performances are made possible thanks to its architecture, that allows it to grip and manipulate micro-objects barely visible to the naked eye (from 40 micrometers to several hundred micrometers). In fact, where other microrobots have a rigid end-effector, MiGriBot is based on a principle with an articulated end. This articulated end allows to drive a microgripper without any wire or embedded actuator. The second advantage of this robot is that all its degrees of mobility, including the ones from the microgripper on the articulated end, are operated from the base of the robot, making its mobile parts very lightweight. Finally, its robotic structure occupies a surface of only 20x20mm2. This level of compactness is achieved by using Silicon for the rigid elements, a polymer (Polydimethylsiloxane – PDMS) as flexible joints and piezoelectric actuators equipped with position sensors. MiGriBot is therefore lighter, more compact and faster than existing robotic micro-manipulators.

MiGriBot holding a cylindrical ruby with a diameter of 700µm and a thickness of 200µm.

While the fastest industrial pick-and-place robots do not exceed 250 cycles per minute, the combination of all the features of this robot: soft joints, small footprint, integrated gripping, lightweight structure and closed-loop control of the fast actuators allows MiGriBot to reach 720 pick-and-place operations per minute with about one micrometer accuracy. In the blink of an eye, MiGRiBot will have manipulated 5 micro-objects, which means that it will have approached, grabbed, moved and released an object 5 times successively.

This robot will be used to assemble Micro-Electro-Mechanical and Optical Systems (MEMS/MOEMS) used in the electronics industry, where the production throughput is increasingly high. Thanks to its speed and compactness, more than 2000 robots can be placed in 1m2 to perform more than one million operations per second. Increasing work rates will improve the productivity and competitiveness of manufacturers, which may encourage the relocation of production to Europe, America and countries with high labor costs. Applications in watch industry, medical instrumentation, aerospace, and other fields are also possible.

MiGriBot holding another parallel structure showing the capacity of the microrobot to manipulate heavy objects.

This work was funded by the ANR MiniSoRo project (ANR-19-CE10-0004) and by Grand Besançon Métropole.

Research team: Maxence LEVEZIEL, Wissem HAOUAS, Michaël GAUTHIER, Guillaume J. LAURENT, Redwan DAHMOUCHE (associate professor at université de Franche-Comté, project leader and head of the research team: redwan.dahmouche@univ-fcomte.fr).

• PAPER – MiGriBot: A miniature parallel robot with integrated gripping for high-throughput micromanipulation. Maxence Leveziel, Wissem Haouas, Guillaume J. Laurent, Michaël Gauthier, and Redwan Dahmouche. Science Robotics, 7(69).

The team at Retrocausal built a computer vision platform that allows any manufacturer to rapidly set up an activity recognition pipeline to assist with manual assemblies. Using any standard camera pointed at an assembly station, the system can learn the assembly procedure and give real-time feedback.

Deci:

At Deci, they are tackling the problem of taking a Machine Learning model optimized for a single hardware platform and adapting it to work on a different hardware platform.

When developing an ML model running on the edge, a time-consuming development cycle is made to optimize the model to run at peak capacity. However, over time hardware platforms get updated and companies need to make the decision of whether or not to update their hardware and dedicate engineering resources to manually tweak a multitude of settings.

Deci allows companies to upload their models to the cloud, then automatically optimize them for running on a variety of different hardware platforms that they own in their facilities.

StegAI:

Dr. Eric Wengrowski co-founded StegAI after publishing a paper in CVPR 2019 on embedding “fingerprints” that are embedded in images and videos that are invisible to the human eye, but detectable by their software.

Their product modulates the pixels in an image in such a way that even after compressing, resizing, or printing out the images, their deep learning algorithms will still be able to detect the origin of the image.

This powerful tool is used by social media firms to protect against potential copyright infringement when users upload content to their platforms

Ant leading other ant to new nest, known as tandem running. Image credit: Norasmah Basari and Nigel R Franks

The team built the robot to mimic the behaviour of rock ants that use one-to-one tuition, in which an ant that has discovered a much better new nest can teach the route there to another individual.

The findings, published in the Journal of Experimental Biology, confirm that most of the important elements of teaching in these ants are now understood because the teaching ant can be replaced by a machine.

Key to this process of teaching is tandem running where one ant literally leads another ant quite slowly along a route to the new nest. The pupil ant learns the route sufficiently well that it can find its own way back home and then lead a tandem-run with another ant to the new nest, and so on.

Prof Nigel Franks of Bristol’s School of Biological Sciences said: “Teaching is so important in our own lives that we spend a great deal of time either instructing others or being taught ourselves. This should cause us to wonder whether teaching actually occurs among non-human animals. And, in fact, the first case in which teaching was demonstrated rigorously in any other animal was in an ant.” The team wanted to determine what was necessary and sufficient in such teaching. If they could build a robot that successfully replaced the teacher, this should show that they largely understood all the essential elements in this process.

Prof Nigel Franks showing Sir David Attenborough the gantry during the opening of the new Life Sciences Building in 2014. Image credit: University of Bristol

The researchers built a large arena so there was an appreciable distance between the ants’ old nest, which was deliberately made to be of low quality, and a new much better one that ants could be led to by a robot. A gantry was placed atop the arena to move back and forth with a small sliding robot attached to it, so that the scientists could direct the robot to move along either straight or wavy routes. Attractive scent glands, from a worker ant, were attached to the robot to give it the pheromones of an ant teacher.

Prof Franks explained: “We waited for an ant to leave the old nest and put the robot pin, adorned with attractive pheromones, directly ahead of it. The pinhead was programmed to move towards the new nest either on a straight path or on a beautifully sinuous one. We had to allow for the robot to be interrupted in its journey, by us, so that we could wait for the following ant to catch up after it had looked around to learn landmarks.”

Diagram of ant pheromone glands. Image credit: Norasmah Basari

“When the follower ant had been led by the robot to the new nest, we allowed it to examine the new nest and then, in its own time, begin its homeward journey. We then used the gantry automatically to track the path of the returning ant.”

The team found that the robot had indeed taught the route successfully to the apprentice ant. The ants knew their way back to the old nest whether they had taken a winding path or a straight one.

Prof Franks explained: “A straight path might be quicker but a winding path would provide more time in which the following ant could better learn landmarks so that it could find its way home as efficiently as if it had been on a straight path.

“Crucially, we could compare the performance of the ants that the robot had taught with ones that we carried to the site of the new nest and that had not had an opportunity to learn the route. The taught ants found their way home much more quickly and successfully.”

The experiments were conducted by undergraduates Jacob Podesta, who is now a PhD student at York, and Edward Jarvis, who was also a Masters student at Professor Nigel Franks’s Lab. The gantry programming was accomplished by Dr. Alan Worley and all the statistical analyses were driven by Dr. Ana Sendova-Franks.

Their approach should make it possible to interrogate further exactly what is involved in successful teaching.

• PAPER – Robotic communication with ants. Nigel R. Franks, Jacob A. Podesta, Edward C. Jarvis, Alan Worley, and Ana B. Sendova-Franks. Journal of Experimental Biology 1 August 2022; 225 (15)

Prof Brendan Englot, from Stevens Institute of Technology, discusses the challenges in perception and decision-making for underwater robots – especially in the field. He discusses ongoing research using the BlueROV platform and autonomous driving simulators.

Brendan Englot

Brendan Englot received his S.B., S.M., and Ph.D. degrees in mechanical engineering from the Massachusetts Institute of Technology in 2007, 2009, and 2012, respectively. He is currently an Associate Professor with the Department of Mechanical Engineering at Stevens Institute of Technology in Hoboken, New Jersey. At Stevens, he also serves as interim director of the Stevens Institute for Artificial Intelligence. He is interested in perception, planning, optimization, and control that enable mobile robots to achieve robust autonomy in complex physical environments, and his recent work has considered sensing tasks motivated by underwater surveillance and inspection applications, and path planning with multiple objectives, unreliable sensors, and imprecise maps.

Links

• Download mp3 (48.0 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

This image shows 3D-printed crystalline lattice structures with air-filled channels, known as “fluidic sensors,” embedded into the structures (the indents on the middle of lattices are the outlet holes of the sensors.) These air channels let the researchers measure how much force the lattices experience when they are compressed or flattened. Image: Courtesy of the researchers, edited by MIT News

By Adam Zewe | MIT News Office

MIT researchers have developed a method for 3D printing materials with tunable mechanical properties, that sense how they are moving and interacting with the environment. The researchers create these sensing structures using just one material and a single run on a 3D printer.

To accomplish this, the researchers began with 3D-printed lattice materials and incorporated networks of air-filled channels into the structure during the printing process. By measuring how the pressure changes within these channels when the structure is squeezed, bent, or stretched, engineers can receive feedback on how the material is moving.

The method opens opportunities for embedding sensors within architected materials, a class of materials whose mechanical properties are programmed through form and composition. Controlling the geometry of features in architected materials alters their mechanical properties, such as stiffness or toughness. For instance, in cellular structures like the lattices the researchers print, a denser network of cells makes a stiffer structure.

This technique could someday be used to create flexible soft robots with embedded sensors that enable the robots to understand their posture and movements. It might also be used to produce wearable smart devices that provide feedback on how a person is moving or interacting with their environment.

“The idea with this work is that we can take any material that can be 3D-printed and have a simple way to route channels throughout it so we can get sensorization with structure. And if you use really complex materials, then you can have motion, perception, and structure all in one,” says co-lead author Lillian Chin, a graduate student in the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL).

Joining Chin on the paper are co-lead author Ryan Truby, a former CSAIL postdoc who is now as assistant professor at Northwestern University; Annan Zhang, a CSAIL graduate student; and senior author Daniela Rus, the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science and director of CSAIL. The paper is published today in Science Advances.

Architected materials

The researchers focused their efforts on lattices, a type of “architected material,” which exhibits customizable mechanical properties based solely on its geometry. For instance, changing the size or shape of cells in the lattice makes the material more or less flexible.

While architected materials can exhibit unique properties, integrating sensors within them is challenging given the materials’ often sparse, complex shapes. Placing sensors on the outside of the material is typically a simpler strategy than embedding sensors within the material. However, when sensors are placed on the outside, the feedback they provide may not provide a complete description of how the material is deforming or moving.

Instead, the researchers used 3D printing to incorporate air-filled channels directly into the struts that form the lattice. When the structure is moved or squeezed, those channels deform and the volume of air inside changes. The researchers can measure the corresponding change in pressure with an off-the-shelf pressure sensor, which gives feedback on how the material is deforming.

Because they are incorporated into the material, these “fluidic sensors” offer advantages over conventional sensor materials.

This image shows a soft robotic finger made from two cylinders comprised of a new class of materials known as handed shearing auxetics (HSAs), which bend and rotate. Air-filled channels embedded within the HSA structure connect to pressure sensors (pile of chips in the foreground), which actively measure the pressure change of these “fluidic sensors.” Image: Courtesy of the researchers

“Sensorizing” structures

The researchers incorporate channels into the structure using digital light processing 3D printing. In this method, the structure is drawn out of a pool of resin and hardened into a precise shape using projected light. An image is projected onto the wet resin and areas struck by the light are cured.

But as the process continues, the resin remains stuck inside the sensor channels. The researchers had to remove excess resin before it was cured, using a mix of pressurized air, vacuum, and intricate cleaning.

They used this process to create several lattice structures and demonstrated how the air-filled channels generated clear feedback when the structures were squeezed and bent.

“Importantly, we only use one material to 3D print our sensorized structures. We bypass the limitations of other multimaterial 3D printing and fabrication methods that are typically considered for patterning similar materials,” says Truby.

Building off these results, they also incorporated sensors into a new class of materials developed for motorized soft robots known as handed shearing auxetics, or HSAs. HSAs can be twisted and stretched simultaneously, which enables them to be used as effective soft robotic actuators. But they are difficult to “sensorize” because of their complex forms.

They 3D printed an HSA soft robot capable of several movements, including bending, twisting, and elongating. They ran the robot through a series of movements for more than 18 hours and used the sensor data to train a neural network that could accurately predict the robot’s motion. 

Chin was impressed by the results — the fluidic sensors were so accurate she had difficulty distinguishing between the signals the researchers sent to the motors and the data that came back from the sensors.

“Materials scientists have been working hard to optimize architected materials for functionality. This seems like a simple, yet really powerful idea to connect what those researchers have been doing with this realm of perception. As soon as we add sensing, then roboticists like me can come in and use this as an active material, not just a passive one,” she says.

“Sensorizing soft robots with continuous skin-like sensors has been an open challenge in the field. This new method provides accurate proprioceptive capabilities for soft robots and opens the door for exploring the world through touch,” says Rus.

In the future, the researchers look forward to finding new applications for this technique, such as creating novel human-machine interfaces or soft devices that have sensing capabilities within the internal structure. Chin is also interested in utilizing machine learning to push the boundaries of tactile sensing for robotics.

“The use of additive manufacturing for directly building robots is attractive. It allows for the complexity I believe is required for generally adaptive systems,” says Robert Shepherd, associate professor at the Sibley School of Mechanical and Aerospace Engineering at Cornell University, who was not involved with this work. “By using the same 3D printing process to build the form, mechanism, and sensing arrays, their process will significantly contribute to researcher’s aiming to build complex robots simply.”

This research was supported, in part, by the National Science Foundation, the Schmidt Science Fellows Program in partnership with the Rhodes Trust, an NSF Graduate Fellowship, and the Fannie and John Hertz Foundation.

A sustainable and scalable gas fermentation technology transforms CO2 from industrial emissions into a single cell protein for animal nutrition. © Valdis Skudre, Shutterstock

By Andrew Dunne

Gardening is proven to be healthful and joyful, but as more of us discover the joys of working in the garden for the first time, some basic knowledge about plants, landscaping and soil is required to get started. What, where and when should you plant, for instance?

These were some of the core questions co-founder of the start-up Draw Me A Garden (DMAG), Florent De Salaberry, realised were standing in the way of more people digging in to the subject.

IKEA-like

Many people want to garden, but lots of us just don’t have the expertise or confidence to begin,’ said the French tech entrepreneur.

DMAG is an app and website service which offers tailored 3D-plans for garden design. It helps budding gardeners to transform any plot into a beautiful, sustainable garden with ease.

The inspiration behind the company’s name comes from the children’s book ‘Le Petit Prince’ in which the prince requests the narrator to ‘draw me a sheep’ to start a conversation and build a relationship.

De Salaberry says “Draw Me A Garden” uses digital tools in a similar way to help people build a relationship with nature in their gardens.

The DMAG service helps customers envisage their dream garden by providing creative ideas, planting tips and, most important of all, delivering all the plants to their door.

Giving customers ownership of their creations is what distinguishes DMAG from traditional landscaping, argues De Salaberry. ‘We know that if you just pay people to landscape your garden, not only is that really expensive but it’s also hard to feel pride in it,’ he said.

‘DMAG is about making gardening easy and affordable, and providing the resources to enable customers to be at the heart of their own projects.’

Garden varieties

Customers locate their garden online via a satellite map. Next, they list any pre-existing features such as a terrace or a child’s play area, then select a preferred garden style, such as for example English cottage garden or Mediterranean.

“Many people want to garden, but lots of us just don’t have the expertise or confidence to begin.”

– Florent de Salaberry, Draw Me A Garden

Behind the scenes, DMAG’s algorithm whirrs away using these inputs together with local knowledge (soil type, elevation, sun direction) to map out the perfect garden design. Customers can visualise the design using 3D mapping tools on the DMAG website.

A qualified landscaper supports the design process and the customer receives a number of planning options to mull over.

Green thumbs

Results come back almost instantaneously. ‘The idea was always to enable customers to do this wherever or whenever they wanted and it takes just a few seconds to get the first design back,’ said De Salaberry.

Once further small refinements are made, a 3D view is rendered, and customers can sit back and wait for all plants and growing instructions to be delivered.

A typical delivery might consist of between 200 – 300 plants. These come with biodegradable cardboard scaffolds cut to the exact garden size and instructions to help the gardeners plant them out.

So far, the DMAG team have supplied to gardeners of all kinds in France, Belgium and Luxembourg, with average expenditure of around €1 500.

De Salaberry likens his turnkey garden concept to how IKEA has revolutionised kitchen design.

As they look to scale-up this work in new EU countries and the US, they hope many more people will soon be asking them to start their gardening journey and “draw me a garden.”

Glade runner

If DMAG can help gardeners create the ideal future garden space, then the TrimBot2020 might be the answer to help maintain it.

The brainchild of computer vision and robotics’ expert, Professor Bob Fisher of the University of Edinburgh, TrimBot2020 is one of the first robot gardening devices that promises to do more than simply mow the lawn.

The TrimBot2020 © TrimBot2020 Consortium, 2020

Based on a modified commercially available robot lawnmower, the autonomous vehicle prunes roses, trims hedges and shapes topiary, all while auto-navigating garden terrain.

To achieve this, the robot uses a ring of cameras to draw a 3D map of the garden, some robotic snippers and hefty dose of computer processing power.

‘There are ten cameras which work together to build up a 3D model of the garden, just like our eyes do,’ said Fisher.

Together, these cameras help the robot gain a 360-degree view of the complex terrain of the garden. The robot also matches what it sees to a hand drawn map supplied by the users.

Upon command, the TrimBot springs into life by rolling up to the bush and scanning it to build up a computer-vision model of that particular plant.

‘Once it has an idea of where all the stems are, its robotic arm comes out with the cutter and it starts snipping away,’ said Fisher.

Robochop

For the TrimBot team, the commercial target market is horticultural businesses responsible for maintaining parks, gardens, and recreational areas.

In such cases, they believe the robot can take on pruning duties while the human gardener does something more challenging.

While the commercial future of TrimBot is yet to be determined, the real benefits may yet come through incorporating the technology into the “brains” of next-generation of garden robots.

‘Outdoor robotics is notoriously hard,’ said Fisher. Typical challenges include constant lighting changes, the many different shades of green and variations in the terrain.

Current robot lawnmowers usually require users to mark out an exact area to mow and to position a robot in the right place to start. TrimBot’s technology should enable robots of tomorrow to work that out themselves.

‘With the TrimBot project we’ve really demonstrated what might be possible in the future,’ said Fisher.

Research in this article was funded by the EU.

This article was originally published in Horizon, the EU Research and Innovation magazine.

Uncrewed aircraft in the sky above the headquarters of the Port of Antwerp-Bruges. © Helicus – Geert Vanhandenhove, Rik Van Boxem, 2022

By Gareth Willmer

In a city in the future, a fire breaks out in a skyscraper. An alarm is triggered and a swarm of drones swoops in, surrounds the building and uses antennas to locate people inside, enabling firefighters to go straight to the stricken individuals. Just in the nick of time – no deaths are recorded.

Elsewhere in the city, drones fly back and forth delivering tissue samples from hospitals to specialist labs for analysis, while another rushes a defibrillator to someone who has suffered a suspected cardiac arrest on a football pitch. The patient lives, with the saved minutes proving critical.

At the time of writing, drones have already been used in search-and-rescue situations to save more than 880 people worldwide, according to drone company DJI. Drones are also being used for medical purposes, such as to transport medicines and samples, and take vaccines to remote areas.

Drones for such uses are still a relatively new development, meaning there is plenty of room to make them more effective and improve supporting infrastructure. This is particularly true when it comes to urban environments, where navigation is complex and requires safety regulations.

Flying firefighters

The IDEAL DRONE project developed a system to aid in firefighting and other emergencies to demonstrate the potential for using swarms of uncrewed aerial vehicles (UAVs) in such situations. Equipped with antennas, the drones use a radio-frequency system to detect the location of ‘nodes’ – or tags – worn by people inside a building.

“By knowing how many people are inside the building and where they are located, it will optimise the search-and-rescue operation.”

– Prof Gian Paolo Cimellaro, IDEAL DRONE

Making use of an Italian aircraft hangar, the tests involved pilots on the ground flying three drones around the outside of a building. The idea is that the drones triangulate the position of people inside where their signals intersect, as well as detecting information about their health condition. The details can then be mapped to optimise and accelerate rescue operations, and enhance safety for firefighters by allowing them to avoid searching all over a burning building without knowing where people are.

‘You create a sort of temporary network from outside the building through which you can detect the people inside,’ said Professor Gian Paolo Cimellaro, an engineer at the Polytechnic University of Turin and project lead on IDEAL DRONE.

‘By knowing how many people are inside the building and where they are located, it will optimise the search-and-rescue operation.’

He added: ‘A unique characteristic of this project is that it allows indoor tracking without communication networks such as Wi-Fi or GPS, which might not be available if you are in an emergency like a disaster or post-earthquake situation.’

There are some challenges in terms of accuracy and battery life, while another obvious drawback is that people in the building need to already be wearing trackers.

However, said Prof Cimellaro, current thinking is that this can be unintrusive if tags are incorporated in existing technology that people often already carry such as smartwatches, mobile phones or ID cards. They can also be used by organisations that mandate their use for staff working in hazardous environments, such as factories or offshore oil rigs.

Looking beyond the challenges, Prof Cimellaro thinks such systems could be a reality within five years, with drones holding significant future promise for avoiding ‘putting human lives in danger’.

Medical networks

Another area in which drones can be used to save lives is medical emergencies. This is the focus of the SAFIR-Med project.

Belgian medical drone operator Helicus has established a command-and-control (C2C) centre in Antwerp to coordinate drone flights. The idea is that the C2C automatically creates flight plans using artificial intelligence, navigating within a digital twin – or virtual representation – of the real world. These plans are then relayed to the relevant air traffic authorities for flight authorisation.

‘We foresee drone cargo ports on the rooftops of hospitals, integrated as much as possible with the hospital’s logistical system so that transport can be on demand,’ added Geert Vanhandenhove, manager of flight operations at Helicus.

So far, SAFIR-Med has successfully carried out remote virtual demonstrations, simulations, flights controlled from the C2C at test sites, and other tests such as that of a ‘detect-and-avoid’ system to help drones take evasive action when others are flying in the vicinity.

The next step will be to validate the concepts in real-life demonstrations in several countries, including Belgium, Germany and the Netherlands. The trials envisage scenarios including transfers of medical equipment and tissue samples between hospitals and labs, delivery of a defibrillator to treat a cardiac patient outside a hospital, and transport of a physician to an emergency site by passenger drone.

“We foresee drone cargo ports on the rooftops of hospitals.”

– Geert Vanhandenhove, SAFIR-Med

Additional simulations in Greece and the Czech Republic will show the potential for extending such systems across Europe.

SAFIR-Med is part of a wider initiative known as U-space. It’s co-funded by the Single European Sky Air Traffic Management Research (SESAR) Joint Undertaking which is a public-private effort for safer drone operations under the Digital European Sky.

Making rules

Much of the technology is already there for such uses of drones, says Vanhandenhove. However, he highlights that there are regulatory challenges involved in drone flights in cities, especially with larger models flying beyond visual line of sight (BVLOS). This includes authorisations for demonstrations within SAFIR-Med itself.

‘The fact that this is the first time this is being done is posing significant hurdles,’ he said. ‘It will depend on the authorisations granted as to which scenarios can be executed.’

But regulations are set to open up over time, with European Commission rules facilitating a framework for use of BVLOS UAVs in low-level airspace due to come into force next January.

Vanhandenhove emphasises that the development of more robust drone infrastructure will be a gradual process of learning and improvement. Eventually, he hopes that through well-coordinated systems with authorities, emergency flights can be mobilised in seconds in smart cities of the future. ‘For us, it’s very important that we can get an authorisation in sub-minute time,’ he said.

He believes commercial flights could even begin within a couple of years, though it may not be until post-2025 that widely integrated, robust uncrewed medical systems come into play in cities. ‘It’s about making the logistics of delivering whatever medical treatment faster and more efficient, and taking out as much as possible the constraints and limitations that we have on the route,’ said Vanhandenhove.

Research in this article was funded via the EU’s European Research Council.

This article was originally published in Horizon, the EU Research and Innovation magazine.

Image credit: rtbilder / Shutterstock.com

By Conn Hastings, science writer

Honeybees use a sophisticated dance to tell their sisters about the location of nearby flowers. This phenomenon forms the inspiration for a form of robot-robot communication that does not rely on digital networks. A recent study presents a simple technique whereby robots view and interpret each other’s movements or a gesture from a human to communicate a geographical location. This approach could prove invaluable when network coverage is unreliable or absent, such as in disaster zones.

Where are those flowers and how far away are they? This is the crux of the ‘waggle dance’ performed by honeybees to alert others to the location of nectar-rich flowers. A new study in Frontiers in Robotics and AI has taken inspiration from this technique to devise a way for robots to communicate. The first robot traces a shape on the floor, and the shape’s orientation and the time it takes to trace it tell the second robot the required direction and distance of travel. The technique could prove invaluable in situations where robot labor is required but network communications are unreliable, such as in a disaster zone or in space.

Honeybees excel at non-verbal communication

If you have ever found yourself in a noisy environment, such as a factory floor, you may have noticed that humans are adept at communicating using gestures. Well, we aren’t the only ones. In fact, honeybees take non-verbal communication to a whole new level.

By wiggling their backside while parading through the hive, they can let other honeybees know about the location of food. The direction of this ‘waggle dance’ lets other bees know the direction of the food with respect to the hive and the sun, and the duration of the dance lets them know how far away it is. It is a simple but effective way to convey complex geographical coordinates.

Applying the dance to robots

This ingenious method of communication inspired the researchers behind this latest study to apply it to the world of robotics. Robot cooperation allows multiple robots to coordinate and complete complex tasks. Typically, robots communicate using digital networks, but what happens when these are unreliable, such as during an emergency or in remote locations? Moreover, how can humans communicate with robots in such a scenario?

To address this, the researchers designed a visual communication system for robots with on-board cameras, using algorithms that allow the robots to interpret what they see. They tested the system using a simple task, where a package in a warehouse needs to be moved. The system allows a human to communicate with a ‘messenger robot’, which supervises and instructs a ‘handling robot’ that performs the task.

Robot dancing in practice

In this situation, the human can communicate with the messenger robot using gestures, such as a raised hand with a closed fist. The robot can recognize the gesture using its on-board camera and skeletal tracking algorithms. Once the human has shown the messenger robot where the package is, it conveys this information to the handling robot.

This involves positioning itself in front of the handling robot and tracing a specific shape on the ground. The orientation of the shape indicates the required direction of travel, while the length of time it takes to trace it indicates the distance. This robot dance would make a worker bee proud, but did it work?

• Read original article
• Download original article (pdf)

The researchers put it to the test using a computer simulation, and with real robots and human volunteers. The robots interpreted the gestures correctly 90% and 93.3% of the time, respectively, highlighting the potential of the technique.

“This technique could be useful in places where communication network coverage is insufficient and intermittent, such as robot search-and-rescue operations in disaster zones or in robots that undertake space walks,” said Prof Abhra Roy Chowdhury of the Indian Institute of Science, senior author on the study. “This method depends on robot vision through a simple camera, and therefore it is compatible with robots of various sizes and configurations and is scalable,” added Kaustubh Joshi of the University of Maryland, first author on the study.

Video credit: K Joshi and AR Chowdury

This article was originally published on the Frontiers blog.

At ICRA 2022, the researchers weren’t the only ones working with cutting-edge technology. We spoke to the exhibitors to get real-life demos of their products.

Tangram Vision

Tangram Vision is a hardware-agnostic sensor fusion platform. It streamlines the development and deployment of critical sensor infrastructure like calibration, fusion, and monitoring for any number of cameras, depth sensors, LiDAR, radar, and IMU. Their co-founder, Adam Rodnitzky, walks us through their sensor fusion platform.

FLX Solutions

Matt Bilsky, Founder and CEO of FLX Solutions gives us a live demo of their robot, the FLX BOT. Matt Bilsky applied his Ph.D. in Mechanical Engineering to create a novel, highly compact robot that is designed to reach and inspect parts of a building that a human cannot reach. The FLX BOT is one inch in diameter and is made of modular links that can be attached one after the other to extend the reach and degrees of freedom of the robot.

Exyn Technologies

Exyn Technologies specializes in aerial robotics, drone swarms, multi-modal sensor fusion, 3D mapping, obstacle avoidance, and autonomous navigation & planning.

Exyn’s focus is on developing software for aerial robots so they can operate in GPS-denied environments, without human control, prior information, or pre-existing infrastructure (e.g. no motion capture system).

Pollen Robotics

Pollen Robotics developed the robot, Reachy, an open-source humanoid robot with a quirky appearance. Reachy’s primary userbase is researchers studying fields such as teleoperation. Using mixed autonomy, Reachy is able to be teleoperated by humans, while still having onboard intelligence to autonomously infer the actions you want it to do.

Inspired by fireflies, MIT researchers have created soft actuators that can emit light in different colors or patterns. Credits: Courtesy of the researchers

By Adam Zewe | MIT News Office

Fireflies that light up dusky backyards on warm summer evenings use their luminescence for communication — to attract a mate, ward off predators, or lure prey.

These glimmering bugs also sparked the inspiration of scientists at MIT. Taking a cue from nature, they built electroluminescent soft artificial muscles for flying, insect-scale robots. The tiny artificial muscles that control the robots’ wings emit colored light during flight.

This electroluminescence could enable the robots to communicate with each other. If sent on a search-and-rescue mission into a collapsed building, for instance, a robot that finds survivors could use lights to signal others and call for help.

The ability to emit light also brings these microscale robots, which weigh barely more than a paper clip, one step closer to flying on their own outside the lab. These robots are so lightweight that they can’t carry sensors, so researchers must track them using bulky infrared cameras that don’t work well outdoors. Now, they’ve shown that they can track the robots precisely using the light they emit and just three smartphone cameras.

“If you think of large-scale robots, they can communicate using a lot of different tools — Bluetooth, wireless, all those sorts of things. But for a tiny, power-constrained robot, we are forced to think about new modes of communication. This is a major step toward flying these robots in outdoor environments where we don’t have a well-tuned, state-of-the-art motion tracking system,” says Kevin Chen, who is the D. Reid Weedon, Jr. Assistant Professor in the Department of Electrical Engineering and Computer Science (EECS), the head of the Soft and Micro Robotics Laboratory in the Research Laboratory of Electronics (RLE), and the senior author of the paper.

He and his collaborators accomplished this by embedding miniscule electroluminescent particles into the artificial muscles. The process adds just 2.5 percent more weight without impacting the flight performance of the robot.

Joining Chen on the paper are EECS graduate students Suhan Kim, the lead author, and Yi-Hsuan Hsiao; Yu Fan Chen SM ’14, PhD ’17; and Jie Mao, an associate professor at Ningxia University. The research was published this month in IEEE Robotics and Automation Letters.

A light-up actuator

These researchers previously demonstrated a new fabrication technique to build soft actuators, or artificial muscles, that flap the wings of the robot. These durable actuators are made by alternating ultrathin layers of elastomer and carbon nanotube electrode in a stack and then rolling it into a squishy cylinder. When a voltage is applied to that cylinder, the electrodes squeeze the elastomer, and the mechanical strain flaps the wing.

To fabricate a glowing actuator, the team incorporated electroluminescent zinc sulphate particles into the elastomer but had to overcome several challenges along the way.

First, the researchers had to create an electrode that would not block light. They built it using highly transparent carbon nanotubes, which are only a few nanometers thick and enable light to pass through.

However, the zinc particles only light up in the presence of a very strong and high-frequency electric field. This electric field excites the electrons in the zinc particles, which then emit subatomic particles of light known as photons. The researchers use high voltage to create a strong electric field in the soft actuator, and then drive the robot at a high frequency, which enables the particles to light up brightly.

“Traditionally, electroluminescent materials are very energetically costly, but in a sense, we get that electroluminescence for free because we just use the electric field at the frequency we need for flying. We don’t need new actuation, new wires, or anything. It only takes about 3 percent more energy to shine out light,” Kevin Chen says.

As they prototyped the actuator, they found that adding zinc particles reduced its quality, causing it to break down more easily. To get around this, Kim mixed zinc particles into the top elastomer layer only. He made that layer a few micrometers thicker to accommodate for any reduction in output power.

While this made the actuator 2.5 percent heavier, it emitted light without impacting flight performance.

“We put a lot of care into maintaining the quality of the elastomer layers between the electrodes. Adding these particles was almost like adding dust to our elastomer layer. It took many different approaches and a lot of testing, but we came up with a way to ensure the quality of the actuator,” Kim says.

Adjusting the chemical combination of the zinc particles changes the light color. The researchers made green, orange, and blue particles for the actuators they built; each actuator shines one solid color.

They also tweaked the fabrication process so the actuators could emit multicolored and patterned light. The researchers placed a tiny mask over the top layer, added zinc particles, then cured the actuator. They repeated this process three times with different masks and colored particles to create a light pattern that spelled M-I-T.

These artificial muscles, which control the wings of featherweight flying robots, light up while the robot is in flight, which provides a low-cost way to track the robots and also could enable them to communicate. Credits: Courtesy of the researchers

Following the fireflies

Once they had finetuned the fabrication process, they tested the mechanical properties of the actuators and used a luminescence meter to measure the intensity of the light.

From there, they ran flight tests using a specially designed motion-tracking system. Each electroluminescent actuator served as an active marker that could be tracked using iPhone cameras. The cameras detect each light color, and a computer program they developed tracks the position and attitude of the robots to within 2 millimeters of state-of-the-art infrared motion capture systems.

“We are very proud of how good the tracking result is, compared to the state-of-the-art. We were using cheap hardware, compared to the tens of thousands of dollars these large motion-tracking systems cost, and the tracking results were very close,” Kevin Chen says.

In the future, they plan to enhance that motion tracking system so it can track robots in real-time. The team is working to incorporate control signals so the robots could turn their light on and off during flight and communicate more like real fireflies. They are also studying how electroluminescence could even improve some properties of these soft artificial muscles, Kevin Chen says.

“This work is really interesting because it minimizes the overhead (weight and power) for light generation without compromising flight performance,” says Kaushik Jayaram, an assistant professor in Department of Mechanical Engineering at the University of Colorado at Boulder, who was not involved with this research. “The wingbeat synchronized flash generation demonstrated in this work will make it easier for motion tracking and flight control of multiple microrobots in low-light environments both indoors and outdoors.”

“While the light production, the reminiscence of biological fireflies, and the potential use of communication presented in this work are extremely interesting, I believe the true momentum is that this latest development could turn out to be a milestone toward the demonstration of these robots outside controlled laboratory conditions,” adds Pakpong Chirarattananon, an associate professor in the Department of Biomedical Engineering at the City University of Hong Kong, who also was not involved with this work. “The illuminated actuators potentially act as active markers for external cameras to provide real-time feedback for flight stabilization to replace the current motion capture system. The electroluminescence would allow less sophisticated equipment to be used and the robots to be tracked from distance, perhaps via another larger mobile robot, for real-world deployment. That would be a remarkable breakthrough. I would be thrilled to see what the authors accomplish next.”

This work was supported by the Research Laboratory of Electronics at MIT.

• PAPER – FireFly: An Insect-Scale Aerial Robot Powered by Electroluminescent Soft Artificial Muscles. Suhan Kim, Yi-Hsuan Hsiao, YuFan Chen, Jie Mao, and YuFeng Chen. IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 6950-6957, July 2022, doi: 10.1109/LRA.2022.3179486.

Researchers Matthias Hofer, Michael Muehlebach and Raffaello D’Andrea have developed the one-wheel Cubli, a three-dimensional pendulum system that can balance on its pivot using a single reaction wheel. How is it possible to stabilize the two tilt angles of the system with only a single reaction wheel?

The key is to design the system such that the inertia in one direction is higher than in the other direction by attaching two masses far away from the center. As a consequence, the system moves faster in the direction with the lower inertia and slower in the direction with the higher inertia. The controller can leverage this property and stabilize both directions simultaneously.

This work was carried out at the Institute for Dynamic Systems and Control, ETH Zurich, Switzerland.

Almost a decade has passed since the first Cubli

The Cubli robot started with a simple idea: Can we build a 15cm sided cube that can jump up, balance on its corner, and walk across our desk using off-the-shelf motors, batteries, and electronic components? The educational article Cubli – A cube that can jump up, balance, and walk across your desk shows all the design principles and prototypes that led to the development of the robot.

Cubli, from ETH Zurich.

Ritu Raman, the d’Arbeloff Career Development Assistant Professor of Mechanical Engineering, focuses on building with biology, using living cells. Photo: David Sella

By Daniel de Wolff | MIT Industrial Liaison Program

It would seem that engineering is in Ritu Raman’s blood. Her mother is a chemical engineer, her father is a mechanical engineer, and her grandfather is a civil engineer. A common thread among her childhood experiences was witnessing firsthand the beneficial impact that engineering careers could have on communities. One of her earliest memories is watching her parents build communication towers to connect the rural villages of Kenya to the global infrastructure. She recalls the excitement she felt watching the emergence of a physical manifestation of innovation that would have a lasting positive impact on the community.  

Raman is, as she puts it, “a mechanical engineer through and through.” She earned her BS, MS, and PhD in mechanical engineering. Her postdoc at MIT was funded by a L’Oréal USA for Women in Science Fellowship and a Ford Foundation Fellowship from the National Academies of Sciences Engineering and Medicine.

Today, Ritu Raman leads the Raman Lab and is an assistant professor in the Department of Mechanical Engineering. But Raman is not tied to traditional notions of what mechanical engineers should be building or the materials typically associated with the field. “As a mechanical engineer, I’ve pushed back against the idea that people in my field only build cars and rockets from metals, polymers, and ceramics. I’m interested in building with biology, with living cells,” she says.

Our machines, from our phones to our cars, are designed with very specific purposes. And they aren’t cheap. But a dropped phone or a crashed car could mean the end of it, or at the very least an expensive repair bill. For the most part, that isn’t the case with our bodies. Biological materials have an unparalleled ability to sense, process, and respond to their environment in real-time. “As humans, if we cut our skin or if we fall, we’re able to heal,” says Raman. “So, I started wondering, ‘Why aren’t engineers building with the materials that have these dynamically responsive capabilities?’”

These days, Raman is focused on building actuators (devices that provide movement) powered by neurons and skeletal muscle that can teach us more about how we move and how we navigate the world. Specifically, she’s creating millimeter-scale models of skeletal muscle controlled by the motor neurons that help us plan and execute movement as well as the sensory neurons that tell us how to respond to dynamic changes in our environment.

Eventually, her actuators may guide the way to building better robots. Today, even our most advanced robots are a far cry from being able to reproduce human motion — our ability to run, leap, pivot on a dime, and change direction. But bioengineered muscle made in Raman’s lab has the potential to create robots that are more dynamically responsive to their environments.

Dr. Sihao Sun discusses his award-winning research in the area of controlling the flight of a drone when faced with a sudden rotor failure.

Typical research in this area addressed the case where one of the four rotors in a quadrotor suddenly, spontaneously stops working. This previous research does not take into full account real-life scenarios where rotor failure is common. This includes collisions with other drones, walls, birds, and operating in degraded GPS environments.

Dr. Sihao Sun

Dr. Sihao Sun is a postdoctoral research assistant at the Robotics and Perception Group (RPG) in University of Zurich directed by Prof. Davide Scaramuzza. Currently, he is working on control and perception for aerial robots (drones).

In December 2020, he received his PhD degree in Aerospace Engineering from the Control and Simulation Group of Delft University of Technology. His works on quadrotor fault-tolerant flight control have been featured by reputable media, such as IEEE Spectrum.

Links

• Download mp3 (16.9 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

Underwater Autonomous Vehicles face challenging environments where GPS Navigation is rarely possible. John McConnell discusses his research, presented at ICRA 2022, into fusing overhead imagery with traditional SLAM algorithms. This research results in a more robust localization and mapping, with reduced drift commonly seen in SLAM algorithms.

Satellite imagery can be obtained for free or low cost through Google or Mapbox, creating an easily deployable framework for companies in industry to implement.

Links

• Download mp3 (12.2 MB)
• Sonar SLAM opensource repo
• Research Paper
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

Sebastian Scherer from CMU’s Airlab gives us a behind-the-scenes demo at ICRA of their Autonomous Flight Control AI. Their approach aims to cooperate with human pilots and act the way they would.

The team took this approach to create a more natural, less intrusive process for co-habiting human and AI pilots at a single airport. They describe it as a Turing Test, where ideally the human pilot will be unable to distinguish an AI from a person operating the plane.

Their communication system works parallel with a 6-camera hardware package based on the Nvidia AGX Dev Kit. This kit measures the angular speed of objects flying across the videos.

In this world, high angular velocity means low risk — since the object is flying at a fast speed perpendicular to the camera plane.

Low angular velocity indicates high risk since the object could be flying directly at the plane, headed for a collision.

Links

• Download mp3 (19.3 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

Yu Sun, Professor of Computer Science and Engineering at the University of South Florida, created and organized the Robotic Grasping and Manipulation Competition. Yu talks about the impact robots will have in domestic environments, the disparity between industry and academia showcased by competitions, and the commercialization of research.

Yu Sun

Yu Sun is a Professor in the Department of Computer Science and Engineering at the University of South Florida (Assistant Professor 2009-2015, Associate Professor 2015-2020, Associate Chair of Graduate Affairs 2018-2020). He was a Visiting Associate Professor at Stanford University from 2016 to 2017, and received his Ph.D. degree in Computer Science from the University of Utah in 2007. Then he had his Postdoctoral training at Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA (2007-2008) and the University of Utah (2008-2009).

He initiated the IEEE RAS Technical Committee on Robotic Hands, Grasping, and Manipulation and served as its first co-Chair. Yu Sun also served on several editorial boards as an Associate Editor and Senior Editor, including IEEE Transactions on Robotics, IEEE Robotics and Automation Letters (RA-L), ICRA, and IROS.

Links

• Download mp3 (49.0 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

Bill Smart, Professor of Mechanical Engineering and Robotics at Oregon State University, helped start competitions as part of ICRA. In this episode, Bill dives into the high-level decisions involved with creating a meaningful competition. The conversation explores how competitions are there to showcase research, potential ideas for future competitions, the exciting phase of robotics we are currently in, and the intersection of robotics, ethics, and law.

Bill Smart

Dr. Smart does research in the areas of robotics and machine learning. In robotics, Smart is particularly interested in improving the interactions between people and robots; enabling robots to be self-sufficient for weeks and months at a time; and determining how they can be used as personal assistants for people with severe motor disabilities. In machine learning, Smart is interested in developing strategies for teaching robots to act effectively (or even optimally), based on long-term interactions with the world and given intermittent and at times incorrect feedback on their performance.

Links

• Download mp3 (29.1 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

At ICRA 2022, Competitions are a core part of the conference. We shine a spotlight on influential competitions in Robotics. In this episode, Dr Liam Paull talks about the Duckietown Competition, where robots drive around Rubber Ducky passengers in an autonomous driving track.

Dr. Liam Paull

Liam Paull is an assistant professor at l’Université de Montréal and the head of the Montreal Robotics and Embodied AI Lab (REAL). His lab focuses on robotics problems including building representations of the world (such as for simultaneous localization and mapping), modeling of uncertainty, and building better workflows to teach robotic agents new tasks (such as through simulation or demonstration). Previous to this, Liam was a research scientist at CSAIL MIT where he led the TRI funded autonomous car project. He was also a postdoc in the marine robotics lab at MIT where he worked on SLAM for underwater robots. He obtained his PhD from the University of New Brunswick in 2013 where he worked on robust and adaptive planning for underwater vehicles. He is a co-founder and director of the Duckietown Foundation, which is dedicated to making engaging robotics learning experiences accessible to everyone. The Duckietown class was originally taught at MIT but now the platform is used at numerous institutions worldwide.

Links

• Download mp3 (20.1 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

Deep reinforcement learning (DRL) is transitioning from a research field focused on game playing to a technology with real-world applications. Notable examples include DeepMind’s work on controlling a nuclear reactor or on improving Youtube video compression, or Tesla attempting to use a method inspired by MuZero for autonomous vehicle behavior planning. But the exciting potential for real world applications of RL should also come with a healthy dose of caution – for example RL policies are well known to be vulnerable to exploitation, and methods for safe and robust policy development are an active area of research.

At the same time as the emergence of powerful RL systems in the real world, the public and researchers are expressing an increased appetite for fair, aligned, and safe machine learning systems. The focus of these research efforts to date has been to account for shortcomings of datasets or supervised learning practices that can harm individuals. However the unique ability of RL systems to leverage temporal feedback in learning complicates the types of risks and safety concerns that can arise.

This post expands on our recent whitepaper and research paper, where we aim to illustrate the different modalities harms can take when augmented with the temporal axis of RL. To combat these novel societal risks, we also propose a new kind of documentation for dynamic Machine Learning systems which aims to assess and monitor these risks both before and after deployment.

What’s Special About RL? A Taxonomy of Feedback

Reinforcement learning systems are often spotlighted for their ability to act in an environment, rather than passively make predictions. Other supervised machine learning systems, such as computer vision, consume data and return a prediction that can be used by some decision making rule. In contrast, the appeal of RL is in its ability to not only (a) directly model the impact of actions, but also to (b) improve policy performance automatically. These key properties of acting upon an environment, and learning within that environment can be understood as by considering the different types of feedback that come into play when an RL agent acts within an environment. We classify these feedback forms in a taxonomy of (1) Control, (2) Behavioral, and (3) Exogenous feedback. The first two notions of feedback, Control and Behavioral, are directly within the formal mathematical definition of an RL agent while Exogenous feedback is induced as the agent interacts with the broader world.

1. Control Feedback

First is control feedback – in the control systems engineering sense – where the action taken depends on the current measurements of the state of the system. RL agents choose actions based on an observed state according to a policy, which generates environmental feedback. For example, a thermostat turns on a furnace according to the current temperature measurement. Control feedback gives an agent the ability to react to unforeseen events (e.g. a sudden snap of cold weather) autonomously.

Figure 1: Control Feedback.

2. Behavioral Feedback

Next in our taxonomy of RL feedback is ‘behavioral feedback’: the trial and error learning that enables an agent to improve its policy through interaction with the environment. This could be considered the defining feature of RL, as compared to e.g. ‘classical’ control theory. Policies in RL can be defined by a set of parameters that determine the actions the agent takes in the future. Because these parameters are updated through behavioral feedback, these are actually a reflection of the data collected from executions of past policy versions. RL agents are not fully ‘memoryless’ in this respect–the current policy depends on stored experience, and impacts newly collected data, which in turn impacts future versions of the agent. To continue the thermostat example – a ‘smart home’ thermostat might analyze historical temperature measurements and adapt its control parameters in accordance with seasonal shifts in temperature, for instance to have a more aggressive control scheme during winter months.

Figure 2: Behavioral Feedback.

3. Exogenous Feedback

Finally, we can consider a third form of feedback external to the specified RL environment, which we call Exogenous (or ‘exo’) feedback. While RL benchmarking tasks may be static environments, every action in the real world impacts the dynamics of both the target deployment environment, as well as adjacent environments. For example, a news recommendation system that is optimized for clickthrough may change the way editors write headlines towards attention-grabbing  clickbait. In this RL formulation, the set of articles to be recommended would be considered part of the environment and expected to remain static, but exposure incentives cause a shift over time.

To continue the thermostat example, as a ‘smart thermostat’ continues to adapt its behavior over time, the behavior of other adjacent systems in a household might change in response – for instance other appliances might consume more electricity due to increased heat levels, which could impact electricity costs. Household occupants might also change their clothing and behavior patterns due to different temperature profiles during the day. In turn, these secondary effects could also influence the temperature which the thermostat monitors, leading to a longer timescale feedback loop.

Negative costs of these external effects will not be specified in the agent-centric reward function, leaving these external environments to be manipulated or exploited. Exo-feedback is by definition difficult for a designer to predict. Instead, we propose that it should be addressed by documenting the evolution of the agent, the targeted environment, and adjacent environments.

Figure 3: Exogenous (exo) Feedback.

How can RL systems fail?

Let’s consider how two key properties can lead to failure modes specific to RL systems: direct action selection (via control feedback) and autonomous data collection (via behavioral feedback).

First is decision-time safety. One current practice in RL research to create safe decisions is to augment the agent’s reward function with a penalty term for certain harmful or undesirable states and actions. For example, in a robotics domain we might penalize certain actions (such as extremely large torques) or state-action tuples (such as carrying a glass of water over sensitive equipment). However it is difficult to anticipate where on a pathway an agent may encounter a crucial action, such that failure would result in an unsafe event. This aspect of how reward functions interact with optimizers is especially problematic for deep learning systems, where numerical guarantees are challenging.

Figure 4: Decision time failure illustration.

As an RL agent collects new data and the policy adapts, there is a complex interplay between current parameters, stored data, and the environment that governs evolution of the system. Changing any one of these three sources of information will change the future behavior of the agent, and moreover these three components are deeply intertwined. This uncertainty makes it difficult to back out the cause of failures or successes.

In domains where many behaviors can possibly be expressed, the RL specification leaves a lot of factors constraining behavior unsaid. For a robot learning locomotion over an uneven environment, it would be useful to know what signals in the system indicate it will learn to find an easier route rather than a more complex gait. In complex situations with less well-defined reward functions, these intended or unintended behaviors will encompass a much broader range of capabilities, which may or may not have been accounted for by the designer.

Figure 5: Behavior estimation failure illustration.

While these failure modes are closely related to control and behavioral feedback, Exo-feedback does not map as clearly to one type of error and introduces risks that do not fit into simple categories. Understanding exo-feedback requires that stakeholders in the broader communities (machine learning, application domains, sociology, etc.) work together on real world RL deployments.

Risks with real-world RL

Here, we discuss four types of design choices an RL designer must make, and how these choices can have an impact upon the socio-technical failures that an agent might exhibit once deployed.

Scoping the Horizon

Determining the timescale on which aRL agent can plan impacts the possible and actual behavior of that agent. In the lab, it may be common to tune the horizon length until the desired behavior is achieved. But in real world systems, optimizations will externalize costs depending on the defined horizon. For example, an RL agent controlling an autonomous vehicle will have very different goals and behaviors if the task is to stay in a lane,  navigate a contested intersection, or route across a city to a destination. This is true even if the objective (e.g. “minimize travel time”) remains the same.

Figure 6: Scoping the horizon example with an autonomous vehicle.

Defining Rewards

A second design choice is that of actually specifying the reward function to be maximized. This immediately raises the well-known risk of RL systems, reward hacking, where the designer and agent negotiate behaviors based on specified reward functions. In a deployed RL system, this often results in unexpected exploitative behavior – from bizarre video game agents to causing errors in robotics simulators. For example, if an agent is presented with the problem of navigating a maze to reach the far side, a mis-specified reward might result in the agent avoiding the task entirely to minimize the time taken.

Figure 7: Defining rewards example with maze navigation.

Pruning Information

A common practice in RL research is to redefine the environment to fit one’s needs – RL designers make numerous explicit and implicit assumptions to model tasks in a way that makes them amenable to virtual RL agents. In highly structured domains, such as video games, this can be rather benign.However, in the real world redefining the environment amounts to changing the ways information can flow between the world and the RL agent. This can dramatically change the meaning of the reward function and offload risk to external systems. For example, an autonomous vehicle with sensors focused only on the road surface shifts the burden from AV designers to pedestrians. In this case, the designer is pruning out information about the surrounding environment that is actually crucial to robustly safe integration within society.

Figure 8: Information shaping example with an autonomous vehicle.

Training Multiple Agents

There is growing interest in the problem of multi-agent RL, but as an emerging research area, little is known about how learning systems interact within dynamic environments. When the relative concentration of autonomous agents increases within an environment, the terms these agents optimize for can actually re-wire norms and values encoded in that specific application domain. An example would be the changes in behavior that will come if the majority of vehicles are autonomous and communicating (or not) with each other. In this case, if the agents have autonomy to optimize toward a goal of minimizing transit time (for example), they could crowd out the remaining human drivers and heavily disrupt accepted societal norms of transit.

Figure 9: The risks of multi-agency example on autonomous vehicles.

Making sense of applied RL: Reward Reporting

In our recent whitepaper and research paper, we proposed Reward Reports, a new form of ML documentation that foregrounds the societal risks posed by sequential data-driven optimization systems, whether explicitly constructed as an RL agent or implicitly construed via data-driven optimization and feedback. Building on proposals to document datasets and models, we focus on reward functions: the objective that guides optimization decisions in feedback-laden systems. Reward Reports comprise questions that highlight the promises and risks entailed in defining what is being optimized in an AI system, and are intended as living documents that dissolve the distinction between ex-ante (design) specification and ex-post (after the fact) harm. As a result, Reward Reports provide a framework for ongoing deliberation and accountability before and after a system is deployed.

Our proposed template for a Reward Reports consists of several sections, arranged to help the reporter themselves understand and document the system. A Reward Report begins with (1) system details that contain the information context for deploying the model. From there, the report documents (2) the optimization intent, which questions the goals of the system and why RL or ML may be a useful tool. The designer then documents (3) how the system may affect different stakeholders in the institutional interface. The next two sections contain technical details on (4) the system implementation and (5) evaluation. Reward reports conclude with (6) plans for system maintenance as additional system dynamics are uncovered.

The most important feature of a Reward Report is that it allows documentation to evolve over time, in step with the temporal evolution of an online, deployed RL system! This is most evident in the change-log, which is we locate at the end of our Reward Report template:

Figure 10: Reward Reports contents.

What would this look like in practice?

As part of our research, we have developed a reward report LaTeX template, as well as several example reward reports that aim to illustrate the kinds of issues that could be managed by this form of documentation. These examples include the temporal evolution of the MovieLens recommender system, the DeepMind MuZero game playing system, and a hypothetical deployment of an RL autonomous vehicle policy for managing merging traffic, based on the Project Flow simulator.

However, these are just examples that we hope will serve to inspire the RL community–as more RL systems are deployed in real-world applications, we hope the research community will build on our ideas for Reward Reports and refine the specific content that should be included. To this end, we hope that you will join us at our (un)-workshop.

Work with us on Reward Reports: An (Un)Workshop!

We are hosting an “un-workshop” at the upcoming conference on Reinforcement Learning and Decision Making (RLDM) on June 11th from 1:00-5:00pm EST at Brown University, Providence, RI. We call this an un-workshop because we are looking for the attendees to help create the content! We will provide templates, ideas, and discussion as our attendees build out example reports. We are excited to develop the ideas behind Reward Reports with real-world practitioners and cutting-edge researchers.

For more information on the workshop, visit the website or contact the organizers at geese-org@lists.berkeley.edu.

This post is based on the following papers:

• Choices, Risks, and Reward Reports: Charting Public Policy for Reinforcement Learning Systems by Thomas Krendl Gilbert, Sarah Dean, Tom Zick, Nathan Lambert. Center for Long Term Cybersecurity Whitepaper Series 2022.
• Reward Reports for Reinforcement Learning by Thomas Krendl Gilbert, Sarah Dean, Nathan Lambert, Tom Zick and Aaron Snoswell. ArXiv Preprint 2022.

In this episode, Audrow Nash speaks to Afreez Gan, who is the founder and CEO of MangDang; MangDang is a Chinese startup that makes Minipupper, an open source robot dog that uses the Robot Operating System (ROS). Minipupper was inspired by and built with lessons learned from the Stanford Pupper, an open source robot dog. In this interview, Afreez speaks about Minipupper’s hardware, software, and documentation, about manufacturing Minipupper, and about his eventual goal of building a home robot dog.

Episode Links

• Download the episode
• Afreez’s LinkedIn
• MangDang’s website
• Minipupper’s Kickstarter
• MangDang’s Github

Podcast info

• Podcast website
• Apple Podcasts
• Spotify
• RSS
• Full episodes
• Clips

3D-printed vase deformed by human finger touch.

Future generations of robots will work very differently from those that assemble entire vehicles or solder electronics onto circuit boards at lightning speed on factory floors today. They will leave the factory halls and start working with people, handing them a tool at the right moment or assisting them in assembling heavy components. They will appear in agriculture, helping harvest the fields or process the fruits. And they will increasingly be found in living rooms, supporting and entertaining people there or simply making them feel less alone.

Of course, these robots will also look different from the enormous metallic contraptions found in today’s industrial plants. Their appearance will change along with their new functions. Whenever they come into contact with people, they will be gentle and soft so that they won’t hurt anyone – and “soft” here actually means that they are made of conformable materials; that their surface is elastic, flexible and stretchable. Of course, at the same time they are equipped with comprehensive sensor technology that immediately registers every touch and every approach, in order to be able to react appropriately. Today, the development of these soft electronics and robotics mostly relies on synthetic materials such as silicone elastomers – a rubber with very good elastic properties but of fossil origin. This also means that if soft robots become as omnipresent in the future as smartphones are today, the tech-waste grows substantially once again. This raises the question: Where are the biodegradable alternatives? And if they exist, how can we make truly empowering robots that move, sense, and react to their environment?

Martin Kaltenbrunner, head of the Soft Materials Laboratory at the Johannes Kepler University in Linz, Austria, investigates such sustainable alternatives for our future technologies. His team focuses on technologies that interface the human body and, therefore, are equally soft and conformable. Wearable electronics, stretchable energy supplies, and biomimetic robots are just a few examples. And by addressing sustainability, his team adds a new twist to these soft technologies.

The scientific breakthrough was achieved in 2020, when Kaltenbrunner and his team discovered a frugal way to make biodegradable gels (biogels) extremely resilient and durable but still vanish when disposed. Based on the abundant biopolymer gelatin, their material had similar properties and performance than the non-degradable silicone rubbers, paving the way for its use in soft robots.

Now, the doctoral students Andreas Heiden and David Preninger have built a system to 3D print this biogel into complex shapes. They have printed finger-like robots that use intricate sensor networks to sense their own deformation and also objects in their surroundings. Together with Florian Hartmann, materials engineer at EPFL, they have published their research in the renowned journal Science Robotics.

Nature as a source of inspiration

Soft robotics greatly benefits from nature as a source of inspiration, introducing innate means of safe interaction between robotic appliances and living organisms. Kaltenbrunner soon realized that despite soft robotics being largely inspired by nature, an inherent “feature” of nature’s creations was missing: biodegradability. Once a soft robot has reached its end-of-life, there is often no simple solution to recycle its components or treating waste in an environmentally friendly manner.

Introducing biodegradable materials to soft robotics seems to be the logical solution, but existing materials were just not durable enough or were difficult to process. Kaltenbrunner and his team were driven to engineer bioderived materials such as gels based on the biopolymer gelatin, that can match the performance of conventional synthetic elastomers, yet degrade fully after their intended use – leaving essentially no trace of having existed. About two years ago, published in the journal Nature Materials, they optimized such gels to be of use in on-skin electronics, and for soft robots. Based on naturally occurring materials as degradable building blocks that are durable, it is a broadly applicable gelatin-based biogel that unites the challenging needs of resilient yet sustainable (soft) robots in a single platform. It is highly stretchable and elastic—and its thermoplastic. An attribute that lets the material melt when heated and makes it perfectly suitable for 3D-printing.

Complex shapes through 3D printing

The deformable structure of soft robots poses challenges in fabrication and assembly. Unlike conventional robots that are screwed together from individual parts, soft robots are manufactured as monolithic blocks. To this end, 3D printing is a versatile fabrication strategy that also allows to produce complex objects. Heiden and Preninger designed a custom system based on Fused Deposition Modeling (FDM) to print their biogel. FDM is one of the most common 3d printing methods nowadays and based on fusing melted polymers that become solid again when cooled. For printing the biogel, the material is melted in a medical syringe and squeezed through the tip, which leads to to the deposition of a biogel “thread” that solidifies quickly after the extrusion. In this manner several two-dimensional layers are drawn subsequently, stacked on each other, to form the three-dimensional object.

XYZ calibration cube and gummybear model printed from gelatin-based biogel ink.

But what happens if a print fails? Usually, you would throw away your print and restart. Having a biodegradable solution, you do not even have to worry about waste production. Alongside this eco-friendly fabrication approach using biodegradable materials, we introduced an additional reuse cycle where the biogel is reprinted up to 5 times, maintaining more than 70% of the initial performance metrics. Utilizing such approaches for a circular economy will enable more sustainable solutions for materials that are less degradable.

Omnidirectional actuators with perception

By extending the fabrication of biodegradable gels to 3D-printing, the researchers were able to produce versatile soft actuators in various complex shapes and even include integrated sensor networks to let them interact with their environment.  In their Science robotics publication, they demonstrated a finger like actuator that is powered by pressurized air and can bend in any direction, similar to an elephant trunk or an octopus tentacle. A combination of three inflatable chambers within the actuator and the use of a cotton-textile reinforcement make this motion possible.

Fully actuated 3-chamber actuator with integrated sensor network detects human finger touch.

Additionally, the actuator features a distributed sensor network that is based on light transmission through transparent materials. These sensors acquire information on the actuator’s own bending state, and also on impact with objects in its surrounding. Without having eyes, this robot is able to detect an obstacle and remove it with from its vicinity. The function of this single robotic element demonstrates that both motion and sensing can be achieved with sustainable material and fabrication solutions, without making large compromise on performance. And once they are no longer of use, they can be simply disposed. Immersing it in water triggers the swelling and dissolution of the biogel and, in the presence of enzymes, complete decomposition.

• PAPER – 3D printing of resilient biogels for omnidirectional and exteroceptive soft actuators. A. Heiden, D. Preninger, L. Lehner, M. Baumgartner, M. Drack, E. Woritzka, D. Schiller, R. Gerstmayr, F. Hartmann, and M. Kaltenbrunner. Science Robotics, Vol 7, Issue 63.

Project BladeBug. Photo credits: RIMA Network

The Inspection and Maintenance (I&M) Industry represents a large economic activity spanning across multiple sectors such as energy, oil & gas, water supply, transport, civil engineering, and infrastructure. RIMA project aims at bringing together Digital Innovation Hubs and Facilitators operating under a common network that allow them to join forces and competences in promoting I&M robotics in Europe.

The BIS Research projects’ analysis of the Inspection and Maintenance Robot Industry forecasts that the I&M market will grow at a significant CAGR of 12.73% on the basis of value from 2020 to 2025. In 2019, Europe dominated the 40% of the global inspection and maintenance robot market (BIS322A, Mar 2020). Although the European Union hosts most of the I&M robotics offer – being France, Germany, and Spain (and U.K. until 2021 Brexit) the leading manufacturing countries, there is still a bottleneck connecting this offer to the market and high potential applications. Therefore, it is the aim of RIMA Network to bridge that gap and improve the connection between key stakeholders.

The RIMA Network

RIMA (Robotics for Inspection and MAintenance) started in 2019 as a European Union’s Horizon2020 initiative for bringing together Digital Innovation Hubs (DIHs) and Facilitators operating under a common network to join forces and competences in promoting I&M robotics. The project consortium consists of 23 partners including Universities, Industry associations and research institutes.

The RIMA project received € 16 Million for establishing a network of Digital Innovation Hubs (DIH) and industry associations to support the uptake of robotics – and help European small and medium-sized companies (SMEs) to develop novel solutions for different industry sectors.

The Digital Innovation Hubs offer key acceleration services for companies. Linked to organisations involved in research and innovation activities, they offer expertise in robotics, equipment, and services applicable for, in this case, I&M operations. The Facilitator organisations have the connections and network to reach a broad set of industries in their domains of expertise.

The RIMA Network consists of 13 Digital Innovation Hubs (DIH) with a vast experience in the field of robotics and is now accepting new DIHs with the purpose to expand the network. All the information and contact points are available on the RIMA Network community platform.

Training courses

The RIMA training framework focuses on increasing the use of digital skills and knowledge of robotics within the I&M domain, amongst a wide range of people – from students, through the current workforce or managers who need to understand the importance and effect of digitisation, and start-up technology development companies that need specific information and support for getting their technology to market.

RIMA offers special trainings on the field of I&M in topics such as financing, current and upcoming robotics technologies, optimisation, innovation management, and working with robots or with virtual reality (VR) and simulation.

Information on existing and future training courses is accessible in the RIMA community and soon in the RIMA YouTube Channel.

Cross-border experiments

RIMA has supported financially and technically 50 cross-border experiments involving European Small and Medium-sized enterprises through two Open Call rounds – the first one in 2019 and the second one in 2020 – with a total amount of 8.1M€.

The successful candidates have received equity-free funding for Technology transfer and development in I&M robotic applications. We will present in future articles some examples of success stories funded through RIMA such as INOWATT, Pipeye RIE, TANGO, ROBUCLEAN, RPIMUS, ACIDRONES, and RAHIP. Stay tuned!

Project GreenBee. Photo credits: RIMA Network

RIMA Network at the European Robotics Forum 2022

The European Robotics Forum (ERF) is one of the most influential events for the robotics and artificial intelligence community in Europe. This year the event takes place from the 28th to the 30th of June in Rotterdam, The Netherlands.

In the session of the 29th of June, “DIH as driver for innovative experimentation: The RIMA Network case”, the RIMA project will offer a holistic approach on the field of DIH-supported experimentation in robotics to allow the audience to gather best practices, inspiration, robotic at I&M literacy, connection with the EU DIH Network, and value propositions to incorporate to their service catalogue.

More information can be found on the ERF2022 website or you can contact RIMA’s Communication lead Juan Antonio Pavón Losada.

Join the RIMA Network community

The RIMA Network offers a global online community that brings together technology providers, service providers, asset owners and operators, with Digital Innovation Hubs and Facilitators. The RIMA Network community platform is a place to collaborate, share information on I&M robotics and keep up to date on RIMA activities. You can join the RIMA community for free here.

Dear robotics graduate students and newcomers to robotics,

If you are what I imagine delving into robotics to be like today, the majority of your time is spent as follows:

• navigating Slack channels while tuning into some online lectures,
• trying to figure out whether you should be reading more papers, coding more, or if you are just a slow reader/coder/etc,
• and if you are a grad student, in particular, having a never-ending cycle of self-doubt questions that seem super important, such as “what is a research question anyway? And how do you find one that no one has tackled before — and not because it’s a dumb question? Oh wait… is my question dumb? Can I find out without asking my prof?”

Much of the questions I had as a grad student stemmed from my lack of knowledge about what is considered to be normal by academia, the robotics community, and more narrowly the subdomain of robotics I belonged to. The friends/colleagues/people I met along the way helped me fill that much-needed knowledge gap about the norm because many of us were on the same journey with similar struggles/questions.

As an assistant professor who spent the majority of my professorship in the COVID-19 pandemic mode, I worry that my students’ grad school journey has not offered the same kind of shared experience and camaraderie with people in the domain that I am now seeing the huge benefit of.

The upcoming IEEE International Conference on Robotics and Automation 2022 will be the first robotics conference that many of you attend in-person since the pandemic (I’m in this category). For many of you, it may be your first time attending an academic conference. For even more of you, this may be your first virtual attendance at ICRA.

ICRA is a multi-track, full-week of robotics festivity that draws in thousands. It can pass by you in a blink.

So, in a series of short blog posts (because, who has the time these days), I am going to highlight a few things in the form of a millennial’s guide to ICRA.

I’m assuming that you, the reader, may be as impatient a reader as I am, who likes information presented in a short, snappy, and organized way. The more bullet points the better.

So let’s get started.

** Full disclosure, I’m one of the two publicity co-chairs for the ICRA conference. If you want to be on the grounds of ICRA as a student science communicator, reach out to us. **

Dr. Carolyn Matl, Research Scientist at Toyota Research Institute, explains why Interactive Perception and soft tactile sensors are critical for manipulating challenging objects such as liquids, grains, and dough. She also dives into “StRETcH” a Soft to Resistive Elastic Tactile Hand, a variable stiffness soft tactile end-effector, presented by her research group.

Carolyn Matl

Carolyn Matl is a research scientist at the Toyota Research Institute, where she works on robotic perception and manipulation with the Mobile Manipulation Team. She received her B.S.E in Electrical Engineering from Princeton University in 2016, and her Ph.D. in Electrical Engineering and Computer Sciences at the University of California, Berkeley in 2021. At Berkeley, she was awarded the NSF Graduate Research Fellowship and was advised by Ruzena Bajcsy. Her dissertation work focused on developing and leveraging non-traditional sensors for robotic manipulation of complicated objects and substances like liquids and doughs.

Carolyn Matl’s Related Research Videos

Links

• StRETcH: a Soft to Resistive Elastic Tactile Hand
• Carolyn Matl’s personal website
• Download mp3 (48.7 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

The GelSight Fin Ray gripper holds a glass Mason jar with its tactile sensing. Photo courtesy of MIT CSAIL.

By Rachel Gordon | MIT CSAIL

The notion of a large metallic robot that speaks in monotone and moves in lumbering, deliberate steps is somewhat hard to shake. But practitioners in the field of soft robotics have an entirely different image in mind — autonomous devices composed of compliant parts that are gentle to the touch, more closely resembling human fingers than R2-D2 or Robby the Robot.

That model is now being pursued by Professor Edward Adelson and his Perceptual Science Group at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). In a recent project, Adelson and Sandra Liu — a mechanical engineering PhD student at CSAIL — have developed a robotic gripper using novel “GelSight Fin Ray” fingers that, like the human hand, is supple enough to manipulate objects. What sets this work apart from other efforts in the field is that Liu and Adelson have endowed their gripper with touch sensors that can meet or exceed the sensitivity of human skin.

Their work was presented last week at the 2022 IEEE 5th International Conference on Soft Robotics.

The fin ray has become a popular item in soft robotics owing to a discovery made in 1997 by the German biologist Leif Kniese. He noticed that when he pushed against a fish’s tail with his finger, the ray would bend toward the applied force, almost embracing his finger, rather than tilting away. The design has become popular, but it lacks tactile sensitivity. “It’s versatile because it can passively adapt to different shapes and therefore grasp a variety of objects,” Liu explains. “But in order to go beyond what others in the field had already done, we set out to incorporate a rich tactile sensor into our gripper.”

The gripper consists of two flexible fin ray fingers that conform to the shape of the object they come in contact with. The fingers themselves are assembled from flexible plastic materials made on a 3D printer, which is pretty standard in the field. However, the fingers typically used in soft robotic grippers have supportive cross-struts running through the length of their interiors, whereas Liu and Adelson hollowed out the interior region so they could create room for their camera and other sensory components.

The camera is mounted to a semirigid backing on one end of the hollowed-out cavity, which is, itself, illuminated by LEDs. The camera faces a layer of “sensory” pads composed of silicone gel (known as “GelSight”) that is glued to a thin layer of acrylic material. The acrylic sheet, in turn, is attached to the plastic finger piece at the opposite end of the inner cavity. Upon touching an object, the finger will seamlessly fold around it, melding to the object’s contours. By determining exactly how the silicone and acrylic sheets are deformed during this interaction, the camera — along with accompanying computational algorithms — can assess the general shape of the object, its surface roughness, its orientation in space, and the force being applied by (and imparted to) each finger.

Liu and Adelson tested out their gripper in an experiment during which just one of the two fingers was “sensorized.” Their device successfully handled such items as a mini-screwdriver, a plastic strawberry, an acrylic paint tube, a Ball Mason jar, and a wine glass. While the gripper was holding the fake strawberry, for instance, the internal sensor was able to detect the “seeds” on its surface. The fingers grabbed the paint tube without squeezing so hard as to breach the container and spill its contents.

The GelSight sensor could even make out the lettering on the Mason jar, and did so in a rather clever way. The overall shape of the jar was ascertained first by seeing how the acrylic sheet was bent when wrapped around it. That pattern was then subtracted, by a computer algorithm, from the deformation of the silicone pad, and what was left was the more subtle deformation due just to the letters.

Glass objects are challenging for vision-based robots because of the refraction of the light. Tactile sensors are immune to such optical ambiguity. When the gripper picked up the wine glass, it could feel the orientation of the stem and could make sure the glass was pointing straight up before it was slowly lowered. When the base touched the tabletop, the gel pad sensed the contact. Proper placement occurred in seven out of 10 trials and, thankfully, no glass was harmed during the filming of this experiment.

Wenzhen Yuan, an assistant professor in the Robotics Institute at Carnegie Mellon University who was not invovled with the research, says, “Sensing with soft robots has been a big challenge, because it is difficult to set up sensors — which are traditionally rigid — on soft bodies,” Yuan says. “This paper provides a neat solution to that problem. The authors used a very smart design to make their vision-based sensor work for the compliant gripper, in this way generating very good results when robots grasp objects or interact with the external environment. The technology has lots of potential to be widely used for robotic grippers in real-world environments.”

Liu and Adelson can foresee many possible applications for the GelSight Fin Ray, but they are first contemplating some improvements. By hollowing out the finger to clear space for their sensory system, they introduced a structural instability, a tendency to twist, that they believe can be counteracted through better design. They want to make GelSight sensors that are compatible with soft robots devised by other research teams. And they also plan to develop a three-fingered gripper that could be useful in such tasks as picking up pieces of fruit and evaluating their ripeness.

Tactile sensing, in their approach, is based on inexpensive components: a camera, some gel, and some LEDs. Liu hopes that with a technology like GelSight, “it may be possible to come up with sensors that are both practical and affordable.” That, at least, is one goal that she and others in the lab are striving toward.

The Toyota Research Institute and the U.S. Office of Naval Research provided funds to support this work.

• PAPER – GelSight Fin Ray: Incorporating Tactile Sensing into a Soft Compliant Robotic Gripper. Sandra Q. Liu, and Edward H. Adelson. 2022 IEEE 5th International Conference on Soft Robotics (RoboSoft).

The robot seen here can’t see the human arm during the entire dressing process, yet it manages to successfully get a jacket sleeve pulled onto the arm. Photo courtesy of MIT CSAIL.

By Steve Nadis | MIT CSAIL

Robots are already adept at certain things, such as lifting objects that are too heavy or cumbersome for people to manage. Another application they’re well suited for is the precision assembly of items like watches that have large numbers of tiny parts — some so small they can barely be seen with the naked eye.

“Much harder are tasks that require situational awareness, involving almost instantaneous adaptations to changing circumstances in the environment,” explains Theodoros Stouraitis, a visiting scientist in the Interactive Robotics Group at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).

“Things become even more complicated when a robot has to interact with a human and work together to safely and successfully complete a task,” adds Shen Li, a PhD candidate in the MIT Department of Aeronautics and Astronautics.

Li and Stouraitis — along with Michael Gienger of the Honda Research Institute Europe, Professor Sethu Vijayakumar of the University of Edinburgh, and Professor Julie A. Shah of MIT, who directs the Interactive Robotics Group — have selected a problem that offers, quite literally, an armful of challenges: designing a robot that can help people get dressed. Last year, Li and Shah and two other MIT researchers completed a project involving robot-assisted dressing without sleeves. In a new work, described in a paper that appears in an April 2022 issue of IEEE Robotics and Automation, Li, Stouraitis, Gienger, Vijayakumar, and Shah explain the headway they’ve made on a more demanding problem — robot-assisted dressing with sleeved clothes. 

The big difference in the latter case is due to “visual occlusion,” Li says. “The robot cannot see the human arm during the entire dressing process.” In particular, it cannot always see the elbow or determine its precise position or bearing. That, in turn, affects the amount of force the robot has to apply to pull the article of clothing — such as a long-sleeve shirt — from the hand to the shoulder.

To deal with obstructed vision in trying to dress a human, an algorithm takes a robot’s measurement of the force applied to a jacket sleeve as input and then estimates the elbow’s position. Image: MIT CSAIL

To deal with the issue of obstructed vision, the team has developed a “state estimation algorithm” that allows them to make reasonably precise educated guesses as to where, at any given moment, the elbow is and how the arm is inclined — whether it is extended straight out or bent at the elbow, pointing upwards, downwards, or sideways — even when it’s completely obscured by clothing. At each instance of time, the algorithm takes the robot’s measurement of the force applied to the cloth as input and then estimates the elbow’s position — not exactly, but placing it within a box or volume that encompasses all possible positions. 

That knowledge, in turn, tells the robot how to move, Stouraitis says. “If the arm is straight, then the robot will follow a straight line; if the arm is bent, the robot will have to curve around the elbow.” Getting a reliable picture is important, he adds. “If the elbow estimation is wrong, the robot could decide on a motion that would create an excessive, and unsafe, force.” 

The algorithm includes a dynamic model that predicts how the arm will move in the future, and each prediction is corrected by a measurement of the force that’s being exerted on the cloth at a particular time. While other researchers have made state estimation predictions of this sort, what distinguishes this new work is that the MIT investigators and their partners can set a clear upper limit on the uncertainty and guarantee that the elbow will be somewhere within a prescribed box.   

The model for predicting arm movements and elbow position and the model for measuring the force applied by the robot both incorporate machine learning techniques. The data used to train the machine learning systems were obtained from people wearing “Xsens” suits with built-sensors that accurately track and record body movements. After the robot was trained, it was able to infer the elbow pose when putting a jacket on a human subject, a man who moved his arm in various ways during the procedure — sometimes in response to the robot’s tugging on the jacket and sometimes engaging in random motions of his own accord.

This work was strictly focused on estimation — determining the location of the elbow and the arm pose as accurately as possible — but Shah’s team has already moved on to the next phase: developing a robot that can continually adjust its movements in response to shifts in the arm and elbow orientation. 

In the future, they plan to address the issue of “personalization” — developing a robot that can account for the idiosyncratic ways in which different people move. In a similar vein, they envision robots versatile enough to work with a diverse range of cloth materials, each of which may respond somewhat differently to pulling.

Although the researchers in this group are definitely interested in robot-assisted dressing, they recognize the technology’s potential for far broader utility. “We didn’t specialize this algorithm in any way to make it work only for robot dressing,” Li notes. “Our algorithm solves the general state estimation problem and could therefore lend itself to many possible applications. The key to it all is having the ability to guess, or anticipate, the unobservable state.” Such an algorithm could, for instance, guide a robot to recognize the intentions of its human partner as it works collaboratively to move blocks around in an orderly manner or set a dinner table. 

Here’s a conceivable scenario for the not-too-distant future: A robot could set the table for dinner and maybe even clear up the blocks your child left on the dining room floor, stacking them neatly in the corner of the room. It could then help you get your dinner jacket on to make yourself more presentable before the meal. It might even carry the platters to the table and serve appropriate portions to the diners. One thing the robot would not do would be to eat up all the food before you and others make it to the table.  Fortunately, that’s one “app” — as in application rather than appetite — that is not on the drawing board.

This research was supported by the U.S. Office of Naval Research, the Alan Turing Institute, and the Honda Research Institute Europe.

• PAPER – Set-based State Estimation with Probabilistic Consistency Guarantee under Epistemic Uncertainty. Shen Li, Theodoros Stouraitis, Michael Gienger, Sethu Vijayakumar, and Julie A. Shah. Published at IEEE Robotics and Automation Letters, 2022. https://doi.org/10.48550/arXiv.2110.09584

Robotic hand with a 3D-printed tactile fingertip on the little (pinky) finger. The white rigid back to the fingertip is covered with the black flexible 3D-printed skin.

Machines can beat the world’s best chess player, but they cannot handle a chess piece as well as an infant. This lack of robot dexterity is partly because artificial grippers lack the fine tactile sense of the human fingertip, which is used to guide our hands as we pick up and handle objects.

Two papers published in the Journal of the Royal Society Interface give the first in-depth comparison of an artificial fingertip with neural recordings of the human sense of touch. The research was led by Professor of Robotics & AI (Artificial Intelligence), Nathan Lepora, from the University of Bristol’s Department of Engineering Maths and based at the Bristol Robotics Laboratory.

“Our work helps uncover how the complex internal structure of human skin creates our human sense of touch. This is an exciting development in the field of soft robotics – being able to 3D-print tactile skin could create robots that are more dexterous or significantly improve the performance of prosthetic hands by giving them an in-built sense of touch,” said Professor Lepora.

Cut-through section on the 3D-printed tactile skin. The white plastic is a rigid mount for the flexible black rubber skin. Both parts are made together on an advanced 3D-printer. The ‘pins’ on the inside of the skin replicate dermal papillae that are formed inside human skin.

Professor Lepora and colleagues created the sense of touch in the artificial fingertip using a 3D-printed mesh of pin-like papillae on the underside of the compliant skin, which mimic the dermal papillae found between the outer epidermal and inner dermal layers of human tactile skin. The papillae are made on advanced 3D-printers that can mix together soft and hard materials to create complicated structures like those found in biology.

“We found our 3D-printed tactile fingertip can produce artificial nerve signals that look like recordings from real, tactile neurons. Human tactile nerves transmit signals from various nerve endings called mechanoreceptors, which can signal the pressure and shape of a contact. Classic work by Phillips and Johnson in 1981 first plotted electrical recordings from these nerves to study ‘tactile spatial resolution’ using a set of standard ridged shapes used by psychologists. In our work, we tested our 3D-printed artificial fingertip as it ‘felt’ those same ridged shapes and discovered a startlingly close match to the neural data,” said Professor Lepora.

“For me, the most exciting moment was when we looked at our artificial nerve recordings from the 3D-printed fingertip and they looked like the real recordings from over 40 years ago! Those recordings are very complex with hills and dips over edges and ridges, and we saw the same pattern in our artificial tactile data,” said Professor Lepora.

While the research found a remarkably close match between the artificial fingertip and human nerve signals, it was not as sensitive to fine detail. Professor Lepora suspects this is because the 3D-printed skin is thicker than real skin and his team is now exploring how to 3D-print structures on the microscopic scale of human skin.

“Our aim is to make artificial skin as good – or even better – than real skin,” said Professor Lepora.

PAPERS

• Artificial SA-I, RA-I and RA-II/vibrotactile afferents for tactile sensing of texture. Nicholas Pestell and Nathan F. Lepora. Journal of The Royal Society Interface, 19, 20210603. http://doi.org/10.1098/rsif.2021.0603
• Artificial SA-I and RA-I afferents for tactile sensing of ridges and gratings. Nicholas Pestell, Thom Griffith and Nathan F. Lepora. Journal of The Royal Society Interface, 19, 20210822. https://doi.org/10.1098/rsif.2021.0822

The basics

A PhD (doctoral degree) in engineering or a DEng (Doctorate of Engineering) is the highest degree that you can earn in engineering. This is generally a degree that people only earn one of, if they earn one at all. Unlike a bachelor’s degree or a master’s degree, a PhD studying a topic relevant to robotics should be free and students should receive a modest stipend for their living expenses. There are very few stand-alone robotics PhDs programs, so people generally join robotics labs through PhD programs in electrical engineering, computer science, or mechanical engineering. 

Joining a lab

In some programs, students are matched with a lab when they are accepted to the university. This matching is not random: If a university works this way, a professor has to have a space in their lab, see the application, and decide that the student would be a good fit for their lab. Essentially, the professor “hires” the student to join their lab. 

Other programs accept cohorts of students who take courses in the first few years and pick professors to work with by some deadline in the program. The mechanism through which students and professors pair up is usually rotations: Students perform a small research project in each of several labs and then join one of the labs they rotated in. 

The advisor

Regardless of how a student gets matched up with their advisor, the advisor has a lot of power to make their graduate school experience a positive one or a negative one. Someone who is a great advisor for one student may not be a great advisor for another. If you are choosing an advisor, it pays to pay attention to the culture in a lab, and whether you personally feel supported by that environment and the type of mentorship that your advisor offers. In almost every case, this is more important for your success in the PhD program than the specifics of the project you will work on or the prestige of the project, collaborators, or lab. 

Qualifiers

PhD programs typically have qualifiers at some point in the first three years. Some programs use a test-based qualifier system, either creating a specific qualifier test or using tests from final exams of required courses. In some programs, you are tested by a panel of faculty who ask the student questions about course material that they are expected to have learned by this point. In other programs, the student performs a research project and then presents it to a panel of faculty. 

Some universities view the qualifiers as a hurdle that almost all of the admitted PhD students should be able to pass, and some universities view them as a method to weed out students from the PhD program. If you are considering applying to PhD programs, it is worth paying attention to this cultural difference between programs, and not taking it too personally if you do not pass the qualifiers at a school that weeds out half of their students. After all, you were qualified enough to be accepted. It is also important to remember, if you join either kind of program, that if you do not pass your qualifiers, usually what happens is that you leave the program with a free master’s degree. Your time in the program will not be wasted!

The author testing a robot on a steep dune face on a research field trip at Oceano Dunes.

Research

Some advisors will start students on a research project as soon as they join the lab, typically by attaching them to an existing project so that they can get a little mentorship before starting their own project. Some advisors will wait until the student is finished with qualifiers. Either way, it is worth knowing that a PhD student’s relationship to their PhD project is likely different from any project they have ever been involved with before. 

For any other research project, there is another person – the advisor, an older graduate student, a post doc – who has designed the project or at least worked with the student to come up with parameters for success. The scope of previous research projects would typically be one semester or one summer, resulting in one or two papers at most. In contrast, a PhD student’s research project is expected to span multiple years (at least three), should result in multiple publications, and is designed primarily by the student. It is not just that the student has ownership over their project, but that they are responsible for it in a way that they have never been responsible for such a large open-ended project before. It is also their primary responsibility – not one project alongside many others. This can be overwhelming for a lot of students, which is why it is impolite to ask a PhD student how much longer they expect their PhD to take. 

The committee

The “committee” is a group of professors that work in a related area to the student’s. Their advisor is on the committee, but it must include other professors as well. Typically, students need to have a mix of professors from their school and at least one other institution. These professors provide ongoing mentorship on the thesis project. They are the primary audience for the thesis proposal and defense, and will ultimately decide what is sufficient for the student to do in order to graduate. If you are a student choosing your committee, keep in mind that you will benefit greatly from having supportive professors on your committee, just like you will benefit from having a supportive advisor. 

Proposing and defending the thesis

When students are expected to propose a thesis project varies widely by program. In some programs, students propose a topic as part of their qualifier process. In others, students have years after finishing their qualifiers to propose a topic – and can propose as little as a semester before they defend! 

The proposal and defense both typically take the form of a presentation followed by questions from the committee and the audience. In the proposal, the student outlines the project they plan to do, and presents criteria that they and their committee should agree on as the required minimum for them to do in order to graduate. The defense makes the case that the student has hit those requirements. 

After the student presents, the committee will ask them some questions, will confer, and then will either tell the student that they passed or failed. It is very uncommon for a PhD student to fail their defense, and it is generally considered a failure on the part of the advisor rather than the student if this happens, because the advisor shouldn’t have let the student present an unfinished thesis. After the defense, there may be some corrections to the written thesis document or even a few extra experiments, but typically the student does not need to present their thesis again in order to graduate.

The bottom line

A PhD is a long training process to teach students how to become independent researchers. Students will take classes and perform research, and will also likely teach or develop coursework. If this is something you’re thinking about, it’s important to learn about what you might be getting yourself into – and if it’s a journey one of your loved ones is starting on, you should know that it’s not just more school!

By Alexander Badri-Sprowitz, Alborz Aghamaleki Sarvestani, Metin Sitti and Linda Behringer

If a Tyrannosaurus Rex living 66 million years ago featured a similar leg structure as an ostrich running in the savanna today, then we can assume bird legs stood the test of time – a good example of evolutionary selection.

Graceful, elegant, powerful – flightless birds like the ostrich are a mechanical wonder. Ostriches, some of which weigh over 100kg, run through the savanna at up to 55km/h. The ostrich’s outstanding locomotor performance is thought to be enabled by the animal’s leg structure. Unlike humans, birds fold their feet back when pulling their legs up towards their bodies. Why do the animals do this? Why is this foot movement pattern energy-efficient for walking and running? And can the bird’s leg structure with all its bones, muscles, and tendons be transferred to walking robots?

Alexander Badri-Spröwitz has spent more than five years on these questions. At the Max Planck Institute for Intelligent Systems (MPI-IS), he leads the Dynamic Locomotion Group. His team works at the interface between biology and robotics in the field of biomechanics and neurocontrol. The dynamic locomotion of animals and robots is the group’s main focus.

Together with his doctoral student Alborz Aghamaleki Sarvestani, Badri-Spröwitz has constructed a robot leg that, like its natural model, is energy-efficient: BirdBot needs fewer motors than other machines and could, theoretically, scale to large size. On March 16th, Badri-Spröwitz, Aghamaleki Sarvestani, the roboticist Metin Sitti, a director at MPI-IS, and biology professor Monica A. Daley of the University of California, Irvine, published their research in the renowned journal Science Robotics.

Compliant spring-tendon network made of muscles and tendons

When walking, humans pull their feet up and bend their knees, but feet and toes point forward almost unchanged. It is known that Birds are different — in the swing phase, they fold their feet backward. But what is the function of this motion? Badri-Spröwitz and his team attribute this movement to a mechanical coupling. “It’s not the nervous system, it’s not electrical impulses, it’s not muscle activity,” Badri-Spröwitz explains. “We hypothesized a new function of the foot-leg coupling through a network of muscles and tendons that extends across multiple joints”. These multi-joint muscle-tendon coordinate foot folding in the swing phase. In our robot, we have implemented the coupled mechanics in the leg and foot, which enables energy-efficient and robust robot walking. Our results demonstrating this mechanism in a robot lead us to believe that similar efficiency benefits also hold true for birds,” he explains.

The coupling of the leg and foot joints and the forces and movements involved could be the reason why a large animal like an ostrich can not only run fast but also stand without tiring, the researchers speculate. A person weighing over 100kg can also stand well and for a long time, but only with the knees ‘locked’ in an extended position. If the person were to squat slightly, it becomes strenuous after a few minutes. The bird, however, does not seem to mind its bent leg structure; many birds even stand upright while sleeping. A robotic bird’s leg should be able to do the same: no motor power should be needed to keep the structure standing upright.

Robot walks on treadmill

To test their hypothesis, the researchers built a robotic leg modeled after the leg of a flightless bird. They constructed their artificial bird leg so that its foot features no motor, but instead a joint equipped with a spring and cable mechanism. The foot is mechanically coupled to the rest of the leg’s joints through cables and pulleys. Each leg contains only two motors— the hip joints motor, which swings the leg back and forth, and a small motor that flexes the knee joint to pull the leg up. After assembly, the researchers walked BirdBot on a treadmill to observe the robot’s foot folding and unfolding. “The foot and leg joints don’t need actuation in the stance phase,” says Aghamaleki Sarvestani. “Springs power these joints, and the multi-joint spring-tendon mechanism coordinates joint movements. When the leg is pulled into swing phase, the foot disengages the leg’s spring – or the muscle-tendon spring, as we believe it happens in animals,” Badri-Spröwitz adds. A video shows BirdBot walking in the research group’s laboratory.

Zero effort when standing, and when flexing the leg and knee

When standing, the leg expends zero energy. “Previously, our robots had to work against the spring or with a motor either when standing or when pulling the leg up, to prevent the leg from colliding with the ground during leg swing. This energy input is not necessary in BirdBot’s legs,” says Badri-Spröwitz and Aghamaleki Sarvestani adds: “Overall, the new robot requires a mere quarter of the energy of its predecessor.”

The treadmill is now switched back on, the robot starts running, and with each leg swing, the foot disengages the leg’s spring. To disengage, the large foot movement slacks the cable and the remaining leg joints swing loosely. This transition of states, between standing and leg swing, is provided in most robots by a motor at the joint. And a sensor sends a signal to a controller, which turns the robot’s motors on and off. “Previously, motors were switched depending on whether the leg was in the swing or stance phase. Now the foot takes over this function in the walking machine, mechanically switching between stance and swing. We only need one motor at the hip joint and one motor to bend the knee in the swing phase. We leave leg spring engagement and disengagement to the bird-inspired mechanics. This is robust, fast, and energy-efficient,” says Badri-Spröwitz.

Motion sequence of BirdBot’s leg; left is touch-down, then stance (the three first snapshots on the left), then mid-swing with the leg’s characteristic leg flexing posture, and back to touch-down on the right. Springs and spring-tendons are shown on top.

Monica Daley observed in several of her earlier biology studies that the bird’s leg structure not only saves energy during walking and standing but is also adapted by nature so that the animal hardly stumbles and injures itself. In experiments with guineafowls running over hidden potholes, she quantified the birds’ remarkable locomotion robustness. A morphological intelligence is built into the system that allows the animal to act quickly – without having to think about it. Daley had shown that the animals control their legs during locomotion not only with the help of the nervous system. If an obstacle unexpectedly lies in the way, it is not always the animal’s sense of touch or sight that comes into play.

“The structure with its multi-jointed muscle-tendons and its unique foot movement can explain why even heavy, large birds run so quickly, robustly, and energy-efficient. If I assume that everything in the bird is based on sensing and action, and the animal steps onto an unexpected obstacle, the animal might not be able to react quickly enough. Perception and sensing, even the transmission of the stimuli, and the reaction cost time,” Daley says.

Yet Daley’s work on running birds over 20 years demonstrates that birds respond more rapidly than the nervous system allows, indicating mechanical contributions to control. Now that the team developed BirdBot, which is a physical model that directly demonstrates how these mechanisms work, it all makes more sense: the leg switches mechanically if there is a bump in the ground. The switch happens immediately and without time delay. Like birds, the robot features high locomotion robustness.

Whether it’s on the scale of a Tyrannosaurus Rex or a small quail, or a small or large robotic leg. Theoretically, meter-high legs can now be implemented to carry robots with the weight of several tons, that walk around with little power input.

The knowledge gained through BirdBot developed at the Dynamic Locomotion Group and the University of California, Irvine, leads to new insights about animals, which are adapted by evolution. Robots allow testing and sometimes confirming hypotheses from Biology, and advancing both fields.

• PAPER – BirdBot achieves energy-efficient gait with minimal control using avian-inspired leg clutching. Alexander Badri-Spröwitz, Alborz Aghamaleki Sarvestani, Metin Sitti and Monica A. Daley. Science Robotics, Vol 7, Issue 64. DOI: 10.1126/scirobotics.abg4055

Congratulations to the winners of the best paper award of the International Association for Automation and Robotics in Construction 2021. The team around Cynthia Brosque, Elena Galbally, Prof. Martin Fischer, and Prof. Oussama Khatib did excellent groundwork for construction robotics. With permission, Silicon Valley Robotics is reposting the first parts of the paper below. It is also available on the IAARC website.

GOLDBECK US Inc. helped define construction robotics use-cases that generate real-world value on their job sites. In the first phase of the project, they collected data to understand the tasks on site holistically. Subsequently, simulations on how robots perform the job were created. Using this information, the researchers of Stanford University | CIFE – Center for Integrated Facility Engineering then developed suitable robotic prototypes. GOLDBECK supplied physical building components to test the prototypes under realistic conditions.

GOLDBECK is now hunting for robotics companies to help us transition this research into a real-world robotic application!

Abstract

Due to their unstructured and dynamic nature, construction sites present many challenges for robotic automation of tasks. Integrating human-robot collaboration (HRC) is critical for task success and implementation feasibility. This is particularly important for contact-rich tasks and other complex scenarios which require a level of reasoning that cannot be accomplished by a fully autonomous robot. Currently, many solutions rely on precise teleoperation that requires one operator per robot. Alternatively, one operator may oversee several semi-autonomous robots. However, the operators do not have the sensory feedback needed to adequately leverage their expertise and craftsmanship. Haptic interfaces allow for intuitive human-robot collaboration by providing rich contact feedback. This paper presents two human-robot collaboration solutions for welding and joint sealing through the use of a haptic device. Our approach allows for seamless transitions between autonomous robot capabilities and human intervention with rich contact feedback. Additionally, this work opens the door to intuitive programming of new tasks through haptic human demonstration.

1 Introduction

In recent years, progress in mobility, manipulation skills, and AI reasoning have started to enable the use of robots in space, underwater, homes, agriculture, and construction [1]. A particularly important area of interest is the automation of dangerous, strenuous, and laborintensive tasks [2].

Construction sites are especially challenging environments for autonomous robots because of their highly unpredictable and unstructured nature [3, 4]. Hence, fully autonomous robots that replace human labor are not the most feasible or ideal solution. The majority of current approaches rely on a human operator that oversees a single task autonomous robot. The operator receives only visual feedback and is limited in the type of input and recovery from failure he can provide due to the lack of an intuitive interface to do so. [4] attributed this lack in technical flexibility of construction robots to the fact that early construction solutions imitated systems initially developed for industrial fabrication [5].

Some tasks are structured enough to be autonomously performed by a robot with little human input, but many require a more flexible approach that incorporates a higher degree of human reasoning and intuition [6]. Given this reality, a method to design construction robots should be flexible enough to allow varying levels of human-robot collaboration depending on the task.

Haptic devices (Fig.1) provide an effective interface for collaboration by allowing the human to (1) feel the contact forces between the robot’s end-effector and the environment [7], and (2) easily intervene and control the robot motion in scenarios that the autonomous behaviors are not able to handle successfully [8]. Additionally, data from these haptic interventions can be collected and used to learn new autonomous skills. Remote robot control using a haptic interface has been tested in fields such as surgery [9] and underwater exploration [1], but has not yet been widely implemented in construction.

In previous work, the authors explored human-robot collaboration solutions to five hazardous and repetitive construction tasks: installing drywall, painting, bolting, welding, and sealing precast concrete slab joints [10]. Our industry partner, Goldbeck, was interested in automating these assembly and finishing tasks that require on-site, repetitive manual effort, ergonomically challenging positions, and working from dangerous heights. [10] outlines a method for designing collaborative robotic solutions with haptic feedback and to assess their feasibility in simulation.

In this paper, we focus on two of the previously explored tasks (steel welding and sealing precast concrete joints) and apply the aforementioned method to design more flexible collaborative solutions. Different from [10], we propose relying primarily on the robot’s functional autonomy and using haptics as an effective and intuitive way to intervene in unexplored or failure scenarios. Force data from the recovery strategy employed by the operator can be recorded and used to learn from demonstration and augment the robot’s autonomy. Over time, the robot will require less human intervention. This higher degree of autonomy could allow a single operator to supervise many robots at once, overcoming the problems of teleoperation in which one operator per robot is needed.

2 Related Work

While factories have typically separated workers from robots due to safety concerns, human-robot collaboration cannot be overlooked in construction, as robots and humans share one workspace [2]. This requires devising solutions that allow us to effectively combine the workers’ expertise with the robots’ autonomous skills.

Construction literature has studied the use of teleoperation devices [11, 12, 7], particularly focusing on construction machinery, such as excavators. These solutions often involve cameras for visual feedback and GPS sensors for navigation, which can be sufficient to accomplish low dexterity tasks with increased operator safety. However, [7] states that complex tasks involving contact greatly benefit from additional sensory feedback such as tactile information. Furthermore, teleoperation solutions rely heavily on the operator’s guidance and do not fully exploit the autonomous capabilities of the robot.

A different set of collaborative solutions currently used onsite have semi-autonomous robots with a human supervisor that oversees the tasks such as drywall installation, concrete drilling, and layout [13]. The supervisor can provide simple inputs to the robot, such as when to start or stop the operation, while the robot handles the rest of the task. This approach makes better use of modern robotics capabilities and allows a single operator to manage several robots. However, the interfaces used to provide inputs to the robot are often too simplistic to allow recovery from failure.

In the event of a robot failure during task execution, joysticks and control pendants do not always provide enough feedback for the operator to intervene in an effective way that enables timely task completion. Additionally, there is currently no streamlined way to learn from the operator’s intervention and use this data to improve the robot’s autonomous capabilities.

By allowing the operator to feel the contact between the robot and its environment, haptic devices increase the range of scenarios in which the operator can aid in failure recovery [14]. Additionally, we can easily record both force and position data during the operator’s intervention. These human demonstrations of recovery strategies can allow the robot to learn new skills [15] and augment its functional autonomy.

Haptic devices have been used by the construction industry in combination with virtual reality for task training purposes [16]. The technology has allowed workers to train in a safe environment with realistic task conditions. However, haptics are still a novel technology in construction applications and field use has not been reported.

Current algorithms for haptic control of robots [17] can handle large communication delays, making them effective interfaces for remote intervention at long distances. In [1] an operator haptically controls an underwater ocean exploration robot from a distance of 100m.

Finally, [18] provides an example that integrates two modalities of robot control: autonomous robot behavior and expert human-guided motion interactions. In this study, a group of mobile robot arms successfully installed drywall boards in simulation with flexible human intervention.

This body of prior work illustrates how keeping the human in the loop with adequate feedback can facilitate successful task automation in complex, unstructured environments. Moreover, it highlights the value of haptics as a way to provide a flexible and effective interface for human-robot collaboration as well as teaching robots new autonomous skills.

Full Text of Paper

Over the past decade, camera technology has made gradual, and significant improvements thanks to the mobile phone industry. This has accelerated multiple industries, including Robotics. Today, Davide Scaramuzza discusses a step-change in camera innovation that has the potential to dramatically accelerate vision-based robotics applications.

Davide Scaramuzza deep dives on Event Cameras, which operate fundamentally different from traditional cameras. Instead of sampling every pixel on an imaging sensor at a fixed frequency, the “pixels” on an event camera all operate independently, and each responds to changes in illumination. This technology unlocks a multitude of benefits, including extremely highspeed imaging, removal of the concept of “framerate”, removal of data corruption due to having the sun in the sensor, reduced data throughput, and low power consumption. Tune in for more.

Davide Scaramuzza

Davide Scaramuzza is a Professor of Robotics and Perception at both departments of Informatics (University of Zurich) and Neuroinformatics (joint between the University of Zurich and ETH Zurich), where he directs the Robotics and Perception Group. His research lies at the intersection of robotics, computer vision, and machine learning, using standard cameras and event cameras, and aims to enable autonomous, agile, navigation of micro drones in search-and-rescue applications.

Links

• Download mp3 (49.3 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

Researchers from MIT and Harvard suggest that applying theories from cognitive science and educational psychology to the area of human-robot interaction can help humans build more accurate mental models of their robot collaborators, which could boost performance and improve safety in cooperative workspaces. Image: MIT News, iStockphoto

By Adam Zewe | MIT News Office

Scientists who study human-robot interaction often focus on understanding human intentions from a robot’s perspective, so the robot learns to cooperate with people more effectively. But human-robot interaction is a two-way street, and the human also needs to learn how the robot behaves.

Thanks to decades of cognitive science and educational psychology research, scientists have a pretty good handle on how humans learn new concepts. So, researchers at MIT and Harvard University collaborated to apply well-established theories of human concept learning to challenges in human-robot interaction.

They examined past studies that focused on humans trying to teach robots new behaviors. The researchers identified opportunities where these studies could have incorporated elements from two complementary cognitive science theories into their methodologies. They used examples from these works to show how the theories can help humans form conceptual models of robots more quickly, accurately, and flexibly, which could improve their understanding of a robot’s behavior.

Humans who build more accurate mental models of a robot are often better collaborators, which is especially important when humans and robots work together in high-stakes situations like manufacturing and health care, says Serena Booth, a graduate student in the Interactive Robotics Group of the Computer Science and Artificial Intelligence Laboratory (CSAIL), and lead author of the paper.

“Whether or not we try to help people build conceptual models of robots, they will build them anyway. And those conceptual models could be wrong. This can put people in serious danger. It is important that we use everything we can to give that person the best mental model they can build,” says Booth.

Booth and her advisor, Julie Shah, an MIT professor of aeronautics and astronautics and the director of the Interactive Robotics Group, co-authored this paper in collaboration with researchers from Harvard. Elena Glassman ’08, MNG ’11, PhD ’16, an assistant professor of computer science at Harvard’s John A. Paulson School of Engineering and Applied Sciences, with expertise in theories of learning and human-computer interaction, was the primary advisor on the project. Harvard co-authors also include graduate student Sanjana Sharma and research assistant Sarah Chung. The research will be presented at the IEEE Conference on Human-Robot Interaction.

A theoretical approach

The researchers analyzed 35 research papers on human-robot teaching using two key theories. The “analogical transfer theory” suggests that humans learn by analogy. When a human interacts with a new domain or concept, they implicitly look for something familiar they can use to understand the new entity.

The “variation theory of learning” argues that strategic variation can reveal concepts that might be difficult for a person to discern otherwise. It suggests that humans go through a four-step process when they interact with a new concept: repetition, contrast, generalization, and variation.

While many research papers incorporated partial elements of one theory, this was most likely due to happenstance, Booth says. Had the researchers consulted these theories at the outset of their work, they may have been able to design more effective experiments.

For instance, when teaching humans to interact with a robot, researchers often show people many examples of the robot performing the same task. But for people to build an accurate mental model of that robot, variation theory suggests that they need to see an array of examples of the robot performing the task in different environments, and they also need to see it make mistakes.

“It is very rare in the human-robot interaction literature because it is counterintuitive, but people also need to see negative examples to understand what the robot is not,” Booth says.

These cognitive science theories could also improve physical robot design. If a robotic arm resembles a human arm but moves in ways that are different from human motion, people will struggle to build accurate mental models of the robot, Booth explains. As suggested by analogical transfer theory, because people map what they know — a human arm — to the robotic arm, if the movement doesn’t match, people can be confused and have difficulty learning to interact with the robot.

Enhancing explanations

Booth and her collaborators also studied how theories of human-concept learning could improve the explanations that seek to help people build trust in unfamiliar, new robots.

“In explainability, we have a really big problem of confirmation bias. There are not usually standards around what an explanation is and how a person should use it. As researchers, we often design an explanation method, it looks good to us, and we ship it,” she says.

Instead, they suggest that researchers use theories from human concept learning to think about how people will use explanations, which are often generated by robots to clearly communicate the policies they use to make decisions. By providing a curriculum that helps the user understand what an explanation method means and when to use it, but also where it does not apply, they will develop a stronger understanding of a robot’s behavior, Booth says.

Based on their analysis, they make a number recommendations about how research on human-robot teaching can be improved. For one, they suggest that researchers incorporate analogical transfer theory by guiding people to make appropriate comparisons when they learn to work with a new robot. Providing guidance can ensure that people use fitting analogies so they aren’t surprised or confused by the robot’s actions, Booth says.

They also suggest that including positive and negative examples of robot behavior, and exposing users to how strategic variations of parameters in a robot’s “policy” affect its behavior, eventually across strategically varied environments, can help humans learn better and faster. The robot’s policy is a mathematical function that assigns probabilities to each action the robot can take.

“We’ve been running user studies for years, but we’ve been shooting from the hip in terms of our own intuition as far as what would or would not be helpful to show the human. The next step would be to be more rigorous about grounding this work in theories of human cognition,” Glassman says.

Now that this initial literature review using cognitive science theories is complete, Booth plans to test their recommendations by rebuilding some of the experiments she studied and seeing if the theories actually improve human learning.

This work is supported, in part, by the National Science Foundation.

MIT PhD student Martin Nisser tests self-reconfiguring robot blocks, or ElectroVoxels, in microgravity. Photo: Steve Boxall/ZeroG

By Rachel Gordon | MIT CSAIL

If faced with the choice of sending a swarm of full-sized, distinct robots to space, or a large crew of smaller robotic modules, you might want to enlist the latter. Modular robots, like those depicted in films such as “Big Hero 6,” hold a special type of promise for their self-assembling and reconfiguring abilities. But for all of the ambitious desire for fast, reliable deployment in domains extending to space exploration, search and rescue, and shape-shifting, modular robots built to date are still a little clunky. They’re typically built from a menagerie of large, expensive motors to facilitate movement, calling for a much-needed focus on more scalable architectures — both up in quantity and down in size.

Scientists from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) called on electromagnetism — electromagnetic fields generated by the movement of electric current — to avoid the usual stuffing of bulky and expensive actuators into individual blocks. Instead, they embedded small, easily manufactured, inexpensive electromagnets into the edges of the cubes that repel and attract, allowing the robots to spin and move around each other and rapidly change shape.

The “ElectroVoxels” have a side length of about 60 millimeters, and the magnets consist of ferrite core (they look like little black tubes) wrapped with copper wire, totaling a whopping cost of just 60 cents. Inside each cube are tiny printed circuit boards and electronics that send current through the right electromagnet in the right direction.

Unlike traditional hinges that require mechanical attachments between two elements, ElectroVoxels are completely wireless, making it much easier to maintain and manufacture for a large-scale system.

ElectroVoxels are robotic cubes that can reconfigure using electromagnets. The cubes don’t need motors or propellant to move, and can operate in microgravity.

To better visualize what a bunch of blocks would look like while interacting, the scientists used a software planner that visualizes reconfigurations and computes the underlying electromagnetic assignments. A user can manipulate up to a thousand cubes with just a few clicks, or use predefined scripts that encode multiple, consecutive rotations. The system really lets the user drive the fate of the blocks, within reason — you can change the speed, highlight the magnets, and display necessary moves to avoid collisions. You can instruct the blocks to take on different shapes (like a chair to a couch, because who needs both?)

The cheap little blocks are particularly auspicious for microgravity environments, where any structure that you want to launch to orbit needs to fit inside the rocket used to launch it. After initial tests on an air table, ElextroVoxels found true weightlessness when tested in a microgravity flight, with the overall impetus of better space exploration tools like propellant-free reconfiguration or changing the inertia properties of a spacecraft.

By leveraging propellant-free actuation, for example, there’s no need to launch extra fuel for reconfiguration, which addresses many of the challenges associated with launch mass and volume. The hope, then, is that this reconfigurability method could aid myriad future space endeavors: augmentation and replacement of space structures over multiple launches, temporary structures to help with spacecraft inspection and astronaut assistance, and (future iterations) of the cubes acting as self-sorting storage containers.

“ElectroVoxels show how to engineer a fully reconfigurable system, and exposes our scientific community to the challenges that need to be tackled to have a fully functional modular robotic system in orbit,” says Dario Izzo, head of the Advanced Concepts Team at the European Space Agency. “This research demonstrates how electromagnetically actuated pivoting cubes are simple to build, operate, and maintain, enabling a flexible, modular and reconfigurable system that can serve as an inspiration to design intelligent components of future exploration missions.”

To make the blocks move, they have to follow a sequence, like little homogeneous Tetris pieces. In this case, there are three steps to the polarization sequence: launch, travel, and catch, with each phase having a traveling cube (for moving), an origin one (where the traveling cube launches), and destination (which catches the traveling cube). Users of the software can specify which cube to pivot in what direction, and the algorithm will automatically compute the sequence and address of electromagnetic assignments required to make that happen (repel, attract, or turn off).

For future work, moving from space to Earth is the natural next step for ElectroVoxels, which would require doing more detailed modeling and optimization of these electromagnets to do reconfiguration against gravity here.

“When building a large, complex structure, you don’t want to be constrained by the availability and expertise of people assembling it, the size of your transportation vehicle, or the adverse environmental conditions of the assembly site. While these axioms hold true on Earth, they compound severely for building things in space,” says MIT CSAIL PhD student Martin Nisser, the lead author on a paper about ElectroVoxels. “If you could have structures that assemble themselves from simple, homogeneous modules, you could eliminate a lot of these problems. So while the potential benefits in space are particularly great, the paradox is that the favorable dynamics provided by microgravity mean some of those problems are actually also easier to solve — in space, even tiny forces can make big things move. By applying this technology to solve real near-term problems in space, we can hopefully incubate the technology for future use on earth too.”

Nisser wrote the paper alongside Leon Cheng and Yashaswini Makaram of MIT CSAIL; Ryo Suzuki, assistant professor of computer science at the University of Calgary; and MIT Professor Stefanie Mueller. They will present the work at the 2022 International Conference on Robotics and Automation. The work was supported, in part, by The MIT Space Exploration Initiative.

• PAPER – ElectroVoxel: Electromagnetically Actuated Pivoting for Scalable Modular Self-Reconfigurable Robots. Martin Nisser, Leon Cheng, Yashaswini Makaram, Ryo Suzuki and Stefanie Mueller. Proceedings of ICRA ’22.

In this episode, Audrow Nash speaks to Tim Chung, Program Manager in the Tactical Technology Office at the Defense Advanced Research Projects Agency (DARPA), on the DARPA Subterranean (SubT) Challenge. The SubT Challenge is a robotics challenge that aims to develop innovative technologies that would augment operations underground. In this conversation, they talk about the motivation of the SubT Challenge, its systems (hardware) and virtual (simulation) challenges, how the resulting technology has been shared with the world, on benchmarking the robotics behaviors in simulation, on the challenge of scoping the SubT Challenge, and on building the final environment for the systems challenge.

Episode Links

• Download the episode
• The SubT Challenge’s Website
• Tim’s LinkedIn
• DARPAtv’s YouTube Channel
• SubT’s Repository on OSRF

Podcast info

• Podcast website
• Apple Podcasts
• Spotify
• Podcast RSS feed
• Full episodes
• Clips

Photo by RICAIP.eu

By Tereza Šamanová

In the Czech Republic, the initial network of Digital Innovation Hubs (DIHs) in 2016 was very small – it included only two pioneer hubs raised from Horizon 2020 European projects focused on support of digital manufacturing (DIGIMAT located in Kuřim was the first one) and high-performance computing (the National Supercomputing Centre IT4Innovations located in Ostrava was the first DIH registered in CZ). Since then, the Czech network has enlarged up to twelve DIHs and their specialisation varies from manufacturing, cybersecurity, artificial intelligence, robotics, high-performance computing up to sectoral focus such as agriculture and food industry, health or smart cities and smart regions development.

Nowadays you can find at least one DIH in each region of the Czech Republic:

Source: https://s3platform.jrc.ec.europa.eu

How are the Czech DIHs built and structured? They are based often on a leading competence centre, research organisation, technical university, science park, technology centre or the most active innovations-supporting NGOs who have created an efficient consortium of other partners around them, always following the selected field of specialisation. The DIHs in Czechia, as well as all over the Europe, have not been built “on a green meadow” – as we say in Czech, but based on a record of projects, activities and common references that have created the optimal mixture of the DIH`s expertise and capacities. The scheme developed by TNO in 2015 illustrates the typical position of a DIH as an “orchestrator” and cross-point of the regional activities focused on development and implementation of digital innovation:

Photo by DIH-HERO.EU

In 2021, based on the Digital Europe Programme, there were – among the existing DIHs and other players – a selection of an initial network of European Digital Innovation Hubs – DIHs with European relevance, serving not only clients coming from their region and/or country, but providing also cross-border services in the field of their specialisation.

The basic services of DIHs and European DIHs do not differ too much, their principal difference lies in the European and cross-border nature of their services. Therefore, when it comes to the arrangement of services of a standard DIH, you can imagine the mixture of different ingredients that together make a Digital Innovation Hub as a “satellite” published in 2019 by the European Commission:

Source: https://digital-strategy.ec.europa.eu/en/activities/edihs

How do DIHs in the Czech Republic cope with the mission to help SMEs and public organisations with their digitalisation?

• Explaining that advanced digitalisation means working with data and sophisticated interconnection of people, devices, and technologies, not simply transferring paper or analogue-based data to digital ones.
• Organising common events – regional roadshows, awareness raising activities and brainstorming among Czech DIHs to update information at national level on advanced digitalisation, and sharing examples of best practices.
• Sharing information about projects and recommending them to the appropriate beneficiaries with the objective of creating the most added value for the national innovative ecosystem.
• Attracting traditional non-tech SMEs and smaller public organisations and communities (i.e., farmers and/or service providers) to communicate with DIHs so they can offer them a starting plan of digitalisation in an appropriate way.
• Including all the new players (those who are interested in creating a DIH or cooperating with an existing one) inside the Czech DIH community and Central European Platform for Digital Innovations – CEEInno, organising an annual overview of Czech Digital Innovation Hubs and regular meetings for Czech DIHs, always with the support of our Ministry of Industry and Trade as the governmental manager of DIHs.

Photo by CEEInno

How can Czech DIHs become useful for foreign partners, especially operating in the field of robotics or interested in some of the sectors, i.e., agriculture?

• Identifying relevant partners for your project in the Czech Republic (CZ) and bordering countries. Foreign partners will be able to cooperate not only with the Czech Digital Innovation Hubs, but also with the DIHs and other players of the surrounding countries within our Central European Platform for Digital Innovations, CEEInno.
• Providing all the DIH services to clients not only from CZ, we are strengthening our forces in the near future under the Digital Europe Programme and within the European Digital Innovation Hubs initial network.
• Helping to contact Czech SMEs and public organisations, through members and cooperating organisations of the CEEInno Platform.
• Organising, co-organising or hosting your project if you intend to attract the Czech participants.

Photo by RICAIP.EU

Among relevant projects of the Czech DIHs focused on robotics and agro-robotics, there could be pointed out the following:

• RICAIP (Research and Innovation Centre on Advanced Industrial Production), which as an international distributed research centre of excellence focusing on research in robotics and artificial intelligence. The project includes Czech and German partners, being hosted at CIIRC CTU and National Centre for Industry 4.0 as one of the Czech DIHs. Its aim is to strengthen research in industrial production and to connect robotic and AI testbeds in Prague, Brno and Saarbrücken.
• Smart Agri Hub CZ-SK (AgriHub made in CzechoSlovakia), joining eight Czech and Slovak partners on common project aiming to promote digital innovation in the agri-food domain and to facilitate set-up and realisation of Innovation Experiments (IEs) in the region. Activities such as innovative hackathons, matchmaking activities for partners from different sectors and awareness raising events will take place within forthcoming year, the international community (not only from CZ and SK) is invited to participate as well.

If you would like to know more about the Czech Digital Innovation ecosystem you can contact me, Tereza Šamanová (LinkedIn, email), DIHNET Ambassador and informal coordinator of the “Czech DIHs Community” in the DIHNET community platform.

Innovative technologies are under development to reduce plastic litter at sea by at least 50% © MOHAMED ABDULRAHEEM, Shutterstock

By Gareth Willmer

‘It’s the scale of it – it’s a global problem. You can guarantee that any beach you walk on, you’ll find pieces of plastic,’ said James Comerford, a senior researcher in materials and nanotechnology at SINTEF, an independent research organisation in Oslo, Norway.

Plastics are estimated to comprise 85% of marine litter, with 11 million metric tonnes entering the oceans annually and the volume potentially tripling by 2040. Some have predicted that, by weight, there will be more plastics than fish in the seas by 2050.

In light of the alarming outlook, innovative approaches are required to tackle the problem. This is exactly what the EU Mission “Restore our Ocean and Waters by 2030” is targeting, with the ambition of reducing plastic litter at sea by at least 50%, cutting microplastics released into the environment by 30%, and halving agricultural nutrient losses as well as the use of chemical pesticides.

To reduce pollution, the Mission is launching a ‘lighthouse’ in the Mediterranean Sea that will act as a hub to develop, demonstrate and deploy solutions far and wide across the world by getting all the relevant players on board. Its role is to connect and structure activities, disseminate and upscale solutions and mobilise relevant actors.

Its initial focus is on plastic pollution. Projects such as In-No-Plastic and AQUA-LIT are exploring ways to reduce the contribution of people and sea-based industries to plastic pollution, while the Maelstrom project looks at where marine debris is distributed and how best to remove it from the seabed and water. It is also exploring economically viable ways to recover and recycle marine plastic debris, such as circular product design for fishing gear.

The wide-reaching In-No-Plastic project, led by Comerford as the project coordinator, is developing a range of technologies that deal not only with easily visible, large pieces of plastics – or macroplastics – but also the insidious threat of tiny microplastics measuring less than 5 millimetres, and even smaller nanoplastics.

‘Macroplastics are going to need different cleaning technologies to microplastics, so we’re looking at the whole spectrum,’ said Comerford.

Several separate technologies that are currently under development can be deployed in tandem to clean up the water. A couple of them help to deal with microplastics by clumping them into more manageable sizes, one using biodegradable chemical substances called flocculants that cause particles to coagulate, and the other – known as SepaRaptor – using ultrasonic waves that push the particles into clusters.

These can be combined with another technology that uses a screen to sift out plastic debris.

On the macroplastics side of things is SEEker, a four-wheeled plastic-waste-collection robot being trained using artificial intelligence to identify and pick up litter from beaches and put it in a bin carried on its back. The robot will also have a loading station near the beach, where it can dispose of waste and recharge.

‘It’ll be entirely autonomous,’ said Comerford. ‘Because there’s so much litter and because it’s everywhere, you need something focusing on it all the time. To have solely a human influence is really time-consuming.’

Mobile application

Another technology, which includes features that could be key to tackling the issue of plastic pollution in the long-term future, is an application for smartphones. This encourages volunteers to gather litter and record data on their activities, using “social rewards” sourced via the local economy – for example, discounts on pizzas or at the gym.

However, the app will also eventually help to track the amount of plastic waste collected, recycled and used in products, allowing us to get more of a handle on how effectively the circular economy is working.

So many people say they include recycled material in products. If we’re really to make a difference and turn this whole thing round, that’s got to be countable

Although that function is currently under development, Comerford explained that it will be supported using photos and GPS data on collected litter, as well as blockchain technology – which can enable better tracing of the contents of goods by storing data on the movement of materials through a supply chain.

‘So many people say they include recycled material in products,’ said Comerford. ‘If we’re really to make a difference and turn this whole thing round, that’s got to be countable.’

But apart from the pure tech side, public buy-in for solutions to the plastic problem is crucial. Partners in In-No-Plastic, such as non-profit organisation Venice Lagoon Plastic Free (VLPF), are also conducting clean-up initiatives supported by the mobile application and gauging the attitudes of the public on plastic pollution.

Davide Poletto, an executive director at the organisation, says Venice is an ideal place to run plastic pollution initiatives, as a location with an enclosed area of water, and intense marine traffic, aquaculture, fishery activity and tourism. ‘The lagoon of Venice is the largest wetland in the Mediterranean basin and a World Heritage Site of UNESCO, and this is an extraordinary laboratory to work in because you have a lot of different problems,’ he said.

He also points out that the pandemic has provided a ‘unique opportunity’ to analyse just how much overtourism contributes to pollution, including that caused by plastics, and the capacity of the local ecosystem to recover. Poletto cites a study showing that 17 of 40 chemical contaminants previously found in the Venice lagoon were undetectable after early-2020 lockdowns, while the presence of many others was significantly lower.

Boosting awareness

Recent In-No-Plastic events appear to have shown promise for growing public awareness and interest in getting involved. In one clean-up event organised in Venice in 2021, 130 people collected three tonnes of waste, including more than 1,500 kilograms of plastics.

Poletto also cites figures from an ongoing awareness study carried out by his team on more than 1,500 people in Italy, the UK and Croatia, the vast majority from outside related work sectors. Over 85% of respondents per country said joining clean-up events had helped them better understand the seriousness of marine plastic pollution, while almost 95% identified microplastics as a bigger issue than macroplastics – suggesting understanding is now widespread on the perils of invisible fragments.

Poletto pointed to growing coverage in the news and social media, as well as first-hand experience. ‘It’s interesting to see how people are realising all those things,’ he said. ‘And it’s not that they are specialists.’

But apart from stimulating public interest, he said more knowledge is needed on sources of plastic pollution to better advise decision-makers on how to deal with it. Using another app that aids with beach litter identification as part of the Maelstrom project, VLPF found that on some beaches, up to 40% of plastics on nearby islands such as Pellestrina came from fishing gear – mostly mussel nets.

This is important to show, for instance, that a big proportion of plastics in these areas goes straight into the sea rather than originating in rivers, said Poletto. ‘Then there’s evidence brought to the public administration that we should do more in certain locations.’

Aquaculture challenge

Gear is a big issue in the aquaculture industry too, where there is also an urgent need to tackle plastic pollution given that it is the world’s fastest-growing food sector. Aquaculture is estimated to account for more than half of global fish consumption, and could reach over 60% in the next decade.

But Mariana Mata Lara, project manager at environmental technology organisation Geonardo, says that much more knowledge is needed on how to tackle plastic pollution from the sector, caused by items including cages, ropes, nets and buoys.

She also said we need to separate data on pollution caused by aquaculture, or farming of aquatic produce, from that caused by traditional fisheries that catch wild fish. ‘In reality, we don’t know exactly the amount of plastics that comes from this sector,’ added Lara.

With this in mind, a project she led called AQUA-LIT sought to create a knowledge base on both plastics and other marine waste before the problem gets too big as the sector surges. ‘In many things in life, we come up with solutions once the problem exists. The idea with AQUA-LIT was to go in parallel and start solving this as it grows, so we don’t later have to come up with solutions to cover what we did in the past,’ said Lara.

AQUA-LIT did this by developing a toolbox of measures to monitor and prevent marine littering in the sector, as well as to remove and recycle waste.

The idea with AQUA-LIT was to go in parallel and start solving this as it grows, so we don’t later have to come up with solutions to cover what we did in the past

The team gathered the information by working with research institutes, organisations and people involved in aquaculture in the Mediterranean, North Sea and Baltic Sea. Activities included interactive ‘Learning Lab’ workshops to discuss marine litter issues, exchange knowledge and brainstorm ideas.

More than 400 ideas and solutions

The resulting toolbox contains a variety of measures, arranged by topics including different sea basins, aquaculture types, and stage of removal and recycling, as well as policy recommendations. ‘In the toolbox, we have provided more than 400 ideas and solutions,’ said Lara.

As part of its work, AQUA-LIT has created an inventory detailing 65 sources of waste generated by aquaculture, a database on how European ports deal with litter and regional maps on percentages of aquaculture-related litter across its focus sea basins.

Lara added that many of these ideas can be applied or expanded on elsewhere. ‘We wanted this information to be useful not only for these three sea basins we worked in, so we created action plans to transfer the knowledge to other regions,’ she said.

As an example, Lara described how the resources had been used by the Global Ghost Gear Initiative, an alliance involving the fishing industry, private sector, corporates, NGOs, academia and governments that focuses on solving the problem of lost and abandoned fishing gear.

‘The Global Ghost Gear Initiative developed a best-practice framework for the management of aquaculture gear, and they used four of our reports, our marine inventory and our toolbox to help build it,’ said Lara.

With a section in the toolbox for people to contribute ideas, she hopes it will grow further and that the knowledge base will ultimately lead to more practical solutions. ‘The idea is that it’s for everyone and fed by everyone,’ she said.

Lara said that promise was shown by AQUA-LIT being invited to present at events in locations such as the Black Sea, and for a Latin American audience, reflecting the significant need for this type of information and its importance as a widespread issue. ‘I think the value of AQUA-LIT is having done that first step,’ she said.

With In-No-Plastic likewise hoping to provide foundations to drive forward solutions to marine waste, the problem of plastics and other litter is set to be tackled from multiple angles.

That will also require wide societal strategies to deal with waste, said Comerford. ‘It’s a holistic approach we need,’ he said. ‘You need to look at everything in the environment currently, but also we can be a bit cleverer about our products in terms of sustainability and end-of-life options.’

This article was originally published in Horizon, the EU Research and Innovation magazine.

In this episode, Audrow Nash speaks to Steve Macenski, who is the Open Source Robotics Engineering Lead at Samsung Research America. Steve leads the Nav2 project, which is an open source robot navigation framework. In this conversation, they talk about the problem and challenges of robot navigation, how Nav2 works at a high level, hybrid planners, and about Steve’s experience working with the Nav2 community.

Episode Links

• Download the episode
• Nav2’s Website
• Steve’s Website
• Steve’s LinkedIn
• Samsung Research’s Website

Podcast info

• Podcast website
• Apple Podcasts
• Spotify
• Podcast RSS feed
• Full episodes
• Clips

Front view of the flying robot. Image credit: Dr Tim Helps

This new advance, published in the journal Science Robotics, could pave the way for smaller, lighter and more effective micro flying robots for environmental monitoring, search and rescue, and deployment in hazardous environments.

Until now, typical micro flying robots have used motors, gears and other complex transmission systems to achieve the up-and-down motion of the wings. This has added complexity, weight and undesired dynamic effects.

Taking inspiration from bees and other flying insects, researchers from Bristol’s Faculty of Engineering, led by Professor of Robotics Jonathan Rossiter, have successfully demonstrated a direct-drive artificial muscle system, called the Liquid-amplified Zipping Actuator (LAZA), that achieves wing motion using no rotating parts or gears.

The LAZA system greatly simplifies the flapping mechanism, enabling future miniaturization of flapping robots down to the size of insects.

In the paper, the team show how a pair of LAZA-powered flapping wings can provide more power compared with insect muscle of the same weight, enough to fly a robot across a room at 18 body lengths per second.

They also demonstrated how the LAZA can deliver consistent flapping over more than one million cycles, important for making flapping robots that can undertake long-haul flights.

The team expect the LAZA to be adopted as a fundamental building block for a range of autonomous insect-like flying robots.

Dr Tim Helps, lead author and developer of the LAZA system said: “With the LAZA, we apply electrostatic forces directly on the wing, rather than through a complex, inefficient transmission system. This leads to better performance, simpler design, and will unlock a new class of low-cost, lightweight flapping micro-air vehicles for future applications, like autonomous inspection of off-shore wind turbines.”

Professor Rossiter added: “Making smaller and better performing flapping wing micro robots is a huge challenge. LAZA is an important step toward autonomous flying robots that could be as small as insects and perform environmentally critical tasks such as plant pollination and exciting emerging roles such as finding people in collapsed buildings.”

• PAPER – Liquid-amplified zipping actuators for micro-air vehicles with transmission-free flapping. T. Helps, C. Romero, M. Taghavi, A. Conn and J. Rossiter. Science Robotics, vol. 7 (63)

The 35th conference on Neural Information Processing Systems (NeurIPS2021) featured eight invited talks. In this post, we give a flavour of the final presentation.

The collective intelligence of army ants, and the robots they inspire

Radhika Nagpal

Radhika’s research focusses on collective intelligence, with the overarching goal being to understand how large groups of individuals, with local interaction rules, can cooperate to achieve globally complex behaviour. These are fascinating systems. Each individual is miniscule compared to the massive phenomena that they create, and, with a limited view of the actions of the rest of the swarm, they achieve striking coordination.

Looking at collective intelligence from an algorithmic point-of-view, the phenomenon emerges from many individuals interacting using simple rules. When run by these large, decentralised groups, these simple rules result in highly intelligent behaviour.

The subject of Radhika’s talk was army ants, a species which spectacularly demonstrate collective intelligence. Without any leader, millions of ants work together to self-assemble nests and build bridge structures using their own bodies.

One particular aspect of study concerned self-assembly of such bridges. Radhika’s research team, which comprised three roboticists and two biologists, found that the ants created bridges adapt to traffic flow and terrain. The ants also disassembled the bridge when the flow of ants had stopped and it wasn’t needed any more.

The team proposed the following simple hypothesis to explain this behaviour using local rules: if an ant is walking along, and experiences congestion (i.e. another ant steps on it), then it becomes stationary and turns into a bridge, allowing other ants to walk over it. Then, if no ants are walking on it any more, it can get up and leave.

These observations, and this hypothesis, led the team to consider two research questions:

• Could they build a robot swarm with soft robots that can self-assemble amorphous structures, just like the ant bridges?
• Could they formulate rules which allowed these robots to self-assemble temporary and adaptive bridge structures?

There were two motivations for these questions. Firstly, the goal of moving closer to realising robot swarms that can solve problems in a particular environment. Secondly, the use of a synthetic system to better understand the collective intelligence of army ants.

Screenshot from Radhika’s talk

Radhika showed a demonstration of the soft robot designed by her group. It has two feet and a soft body, and moves by flipping – one foot remains attached, while the other detaches from the surface and flips to attach in a different place. This allows movement in any orientation. Upon detaching, a foot searches through space to find somewhere to attach. By using grippers on the feet that can hook onto textured surfaces, and having a stretchable Velcro skin, the robots can climb over each other, like the ants. The robot pulses, and uses a vibration sensor, to detect whether it is in contact with another robot. A video demonstration of two robots interacting showed that they have successfully created a system that can recreate the simple hypothesis outlined above.

In order to investigate the high-level properties of army ant bridges, which would require a vast number of robots, the team created a simulation. Modelling the ants to have the same characteristics as their physical robots, they were able to replicate the high level properties of army ant bridges with their hypothesized rules.

You can read the round-ups of the other NeurIPS invited talks at these links:
#NeurIPS2021 invited talks round-up: part one – Duolingo, the banality of scale and estimating the mean
#NeurIPS2021 invited talks round-up: part two – benign overfitting, optimal transport, and human and machine intelligence

Jakob Foerster an accredited Machine Learning Research Scientist who has been at the forefront of research on Multi-Agent Learning speaks with interviewer Kegan Strawn.

Dr. Foerster explains why incorporating uncertainty into multi-agent interactions is essential to creating robust algorithms that can operate not only in games but in real-world applications.

Jakob Foerster
Jakob Foerster is an Associate Professor at the University of Oxford. His papers have gained prestigious awards at top machine learning conferences (ICML, AAAI) and have helped push deep multi-agent reinforcement learning to the forefront of AI research.

Jakob previously worked at Facebook AI Research and received his Ph.D. from the University of Oxford under the supervision of Shimon Whiteson. During his Ph.D., Jakob also interned at Google Brain, OpenAI, and DeepMind.

Jakob’s research interests span Deep Multi-Agent Reinforcement Learning, Human-AI Coordination, Emergent Communication, Search, Planning, and Game Theory.

Links

• Download mp3 (36.4 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

Through manipulation, a robotic system can physically change the state of the world. This is intertwined with intelligence, which is the ability whereby such system can detect and adapt to change. In his talk, Tamim Asfour gives an overview of the developments in manipulation for robotic systems his lab has done by learning manipulation task models from human observations, and the challenges and open questions associated with this.

Bio: Tamim Asfour is full Professor at the Institute for Anthropomatics and Robotics, where he holds the chair of Humanoid Robotics Systems and is head of the High Performance Humanoid Technologies Lab (H2T). His current research interest is high performance 24/7 humanoid robotics. Specifically, his research focuses on engineering humanoid robot systems integrating the mechano-informatics of such systems with the capabilities of predicting, acting and learning from human demonstration and sensorimotor experience. He is developer and the leader of the development team of the ARMAR humanoid robot family.

The legged robot ANYmal on the rocky path to the summit of Mount Etzel, which stands 1,098 metres above sea level. (Photo: Takahiro Miki)

By Christoph Elhardt

Steep sections on slippery ground, high steps, scree and forest trails full of roots: the path up the 1,098-​metre-high Mount Etzel at the southern end of Lake Zurich is peppered with numerous obstacles. But ANYmal, the quadrupedal robot from the Robotic Systems Lab at ETH Zurich, overcomes the 120 vertical metres effortlessly in a 31-​minute hike. That’s 4 minutes faster than the estimated duration for human hikers – and with no falls or missteps.

This is made possible by a new control technology, which researchers at ETH Zurich led by robotics professor Marco Hutter recently presented in the journal Science Robotics. “The robot has learned to combine visual perception of its environment with proprioception – its sense of touch – based on direct leg contact. This allows it to tackle rough terrain faster, more efficiently and, above all, more robustly,” Hutter says. In the future, ANYmal can be used anywhere that is too dangerous for humans or too impassable for other robots.

Video: Nicole Davidson / ETH Zurich

Perceiving the environment accurately

To navigate difficult terrain, humans and animals quite automatically combine the visual perception of their environment with the proprioception of their legs and hands. This allows them to easily handle slippery or soft ground and move around with confidence, even when visibility is low. Until now, legged robots have been able to do this only to a limited extent.

“The reason is that the information about the immediate environment recorded by laser sensors and cameras is often incomplete and ambiguous,” explains Takahiro Miki, a doctoral student in Hutter’s group and lead author of the study. For example, tall grass, shallow puddles or snow appear as insurmountable obstacles or are partially invisible, even though the robot could actually traverse them. In addition, the robot’s view can be obscured in the field by difficult lighting conditions, dust or fog.

“That’s why robots like ANYmal have to be able to decide for themselves when to trust the visual perception of their environment and move forward briskly, and when it is better to proceed cautiously and with small steps,” Miki says. “And that’s the big challenge.”

A virtual training camp

Thanks to a new controller based on a neural network, the legged robot ANYmal, which was developed by ETH Zurich researchers and commercialized by the ETH spin-​off ANYbotics, is now able to combine external and proprioceptive perception for the first time. Before the robot could put its capabilities to the test in the real world, the scientists exposed the system to numerous obstacles and sources of error in a virtual training camp. This let the network learn the ideal way for the robot to overcome obstacles, as well as when it can rely on environmental data – and when it would do better to ignore that data.

“With this training, the robot is able to master the most difficult natural terrain without having seen it before,” says ETH Zurich Professor Hutter. This works even if the sensor data on the immediate environment is ambiguous or vague. ANYmal then plays it safe and relies on its proprioception. According to Hutter, this allows the robot to combine the best of both worlds: the speed and efficiency of external sensing and the safety of proprioceptive sensing.

Use under extreme conditions

Whether after an earthquake, after a nuclear disaster, or during a forest fire, robots like ANYmal can be used primarily wherever it is too dangerous for humans and where other robots cannot cope with the difficult terrain.

In September of last year, ANYmal was able to demonstrate just how well the new control technology works at the DARPA Subterranean Challenge, the world’s best-​known robotics competition. The ETH Zurich robot automatically and quickly overcame numerous obstacles and difficult terrain while autonomously exploring an underground system of narrow tunnels, caves, and urban infrastructure. This was a major part of why the ETH Zurich researchers, as part of the CERBERUS team, took first place with a prize of 2 million dollars.

• PAPER – Learning robust perceptive locomotion for quadrupedal robots in the wild. Miki T, Lee J, Hwangbo J, Wellhausen L. Science Robotics 2022, doi: 10.1126/scirobotics.abk2822

Did you manage to watch all the holiday robot videos of 2021? If you did but are still hungry for more, I have prepared this compilation of Science Magazine videos featuring robotics research that were released during last year. Enjoy!

Safety and learning in robots

Safety is a fundamental feature in real-world robotics applications: robots should not cause damage to the environment, to themselves, and they must ensure the safety of people operating around them. To ensure safety when we deploy a new application, we want to avoid constraint violation at any time. These stringent safety constraints are difficult to enforce in a reinforcement learning setting. This is the reason why it is hard to deploy learning agents in the real world. Classical reinforcement learning agents use random exploration, such as Gaussian policies, to act in the environment and extract useful knowledge to improve task performance. However, random exploration may cause constraint violations. These constraint violations must be avoided at all costs in robotic platforms, as they often result in a major system failure.

While the robotic framework is challenging, it is also a very well-known and well-studied problem: thus, we can exploit some key results and knowledge from the field. Indeed, often a robot’s kinematics and dynamics are known and can be exploited by the learning systems. Also, physical constraints e.g., avoiding collisions and enforcing joint limits, can be written in analytical form. All this information can be exploited by the learning robot.

Our approach

Many reinforcement learning approaches try to solve the safety problem by incorporating the constraint information in the learning process. This approach often results in slower learning performances, while not being able to ensure safety during the whole learning process. Instead, we present a novel point of view to the problem, introducing ATACOM (Acting on the TAngent space of the COnstraint Manifold). Different from other state-of-the-art approaches, ATACOM tries to create a safe action space in which every action is inherently safe. To do so, we need to construct the constraint manifold and exploit the basic domain knowledge of the agent. Once we have the constraint manifold, we define our action space as the tangent space to the constraint manifold.

We can construct the constraint manifold using arbitrary differentiable constraints. The only requirement is that the constraint function must depend only on controllable variables i.e. the variables that we can directly control with our control action. An example could be the robot joint positions and velocities.

We can support both equality and inequality constraints. Inequality constraints are particularly important as they can be used to avoid specific areas of the state space or to enforce the joint limits. However, they don’t define a manifold. To obtain a manifold, we transform the inequality constraints into equality constraints by introducing slack variables.

With ATACOM, we can ensure safety by taking action on the tangent space of the constraint manifold. An intuitive way to see why this is true is to consider the motion on the surface of a sphere: any point with a velocity tangent to the sphere itself will keep moving on the surface of the sphere. The same idea can be extended to more complex robotic systems, considering the acceleration of system variables (or the generalized coordinates, when considering a mechanical system) instead of velocities.

The above-mentioned framework only works if we consider continuous-time systems, when the control action is the instantaneous velocity or acceleration. Unfortunately, the vast majority of robotic controllers and reinforcement learning approaches are discrete-time digital controllers. Thus, even taking the tangent direction of the constraint manifold will result in a constraint violation. It is always possible to reduce the violations by increasing the control frequency. However, error accumulates over time, causing a drift from the constraint manifold. To solve this issue, we introduce an error correction term that ensures that the system stays on the reward manifold. In our work, we implement this term as a simple proportional controller.

Finally, many robotics systems cannot be controlled directly by velocity or accelerations. However, if an inverse dynamics model or a tracking controller is available, we can use it and compute the correct control action.

Results

We tried ATACOM on a simulated air hockey task. We use two different types of robots. The first one is a planar robot. In this task, we enforce joint velocities and we avoid the collision of the end-effector with table boundaries.

The second robot is a Kuka Iiwa 14 arm. In this scenario, we constrained the end-effector to move on the planar surface and we ensure no collision will occur between the robot arm and the table.

In both experiments, we can learn a safe policy using the Soft Actor-Critic algorithm as a learning algorithm in combination with the ATACOM framework. With our approach, we are able to learn good policies fast and we can ensure low constraint violations at any timestep. Unfortunately, the constraint violation cannot be zero due to discretization, but it can be reduced to be arbitrarily small. This is not a major issue in real-world systems, as they are affected by noisy measurements and non-ideal actuation.

Is the safety problem solved now?

The key question to ask is if we can ensure any safety guarantees with ATACOM. Unfortunately, this is not true in general. What we can enforce are state constraints at each timestep. This includes a wide class of constraints, such as fixed obstacle avoidance, joint limits, surface constraints. We can extend our method to constraints considering not (directly) controllable variables. While we can ensure safety to a certain extent also in this scenario, we cannot ensure that the constraint violation will not be violated during the whole trajectory. Indeed, if the not controllable variables act in an adversarial way, they might find a long-term strategy to cause constraint violation in the long term. An easy example is a prey-predator scenario: even if we ensure that the prey avoids each predator, a group of predators can perform a high-level strategy and trap the agent in the long term.

Thus, with ATACOM we can ensure safety at a step level, but we are not able to ensure long-term safety, which requires reasoning at trajectory level. To ensure this kind of safety, more advanced techniques will be needed.

Find out more

The authors were best paper award finalists at CoRL this year, for their work: Robot reinforcement learning on the constraint manifold.

• Read the paper.
• The GitHub page for the work is here.
• Read more about the winning and shortlisted papers for the CoRL awards here.

MIT researchers have pioneered a new fabrication technique that enables them to produce low-voltage, power-dense, high endurance soft actuators for an aerial microrobot. Credits: Courtesy of the researchers

By Adam Zewe | MIT News Office

When it comes to robots, bigger isn’t always better. Someday, a swarm of insect-sized robots might pollinate a field of crops or search for survivors amid the rubble of a collapsed building.

MIT researchers have demonstrated diminutive drones that can zip around with bug-like agility and resilience, which could eventually perform these tasks. The soft actuators that propel these microrobots are very durable, but they require much higher voltages than similarly-sized rigid actuators. The featherweight robots can’t carry the necessary power electronics that would allow them fly on their own.

Now, these researchers have pioneered a fabrication technique that enables them to build soft actuators that operate with 75 percent lower voltage than current versions while carrying 80 percent more payload. These soft actuators are like artificial muscles that rapidly flap the robot’s wings.

The artificial muscles vastly improve the robot’s payload and allow it to achieve best-in-class hovering performance. Image: Kevin Chen

This new fabrication technique produces artificial muscles with fewer defects, which dramatically extends the lifespan of the components and increases the robot’s performance and payload.   

“This opens up a lot of opportunity in the future for us to transition to putting power electronics on the microrobot. People tend to think that soft robots are not as capable as rigid robots. We demonstrate that this robot, weighing less than a gram, flies for the longest time with the smallest error during a hovering flight. The take-home message is that soft robots can exceed the performance of rigid robots,” says Kevin Chen, who is the D. Reid Weedon, Jr. ’41 assistant professor in the Department of Electrical Engineering and Computer Science, the head of the Soft and Micro Robotics Laboratory in the Research Laboratory of Electronics (RLE), and the senior author of the paper.

Chen’s coauthors include Zhijian Ren and Suhan Kim, co-lead authors and EECS graduate students; Xiang Ji, a research scientist in EECS; Weikun Zhu, a chemical engineering graduate student; Farnaz Niroui, an assistant professor in EECS; and Jing Kong, a professor in EECS and principal investigator in RLE. The research has been accepted for publication in Advanced Materials and is included in the jounal’s Rising Stars series, which recognizes outstanding works from early-career researchers.

Making muscles

The rectangular microrobot, which weighs less than one-fourth of a penny, has four sets of wings that are each driven by a soft actuator. These muscle-like actuators are made from layers of elastomer that are sandwiched between two very thin electrodes and then rolled into a squishy cylinder. When voltage is applied to the actuator, the electrodes squeeze the elastomer, and that mechanical strain is used to flap the wing.

The rectangular microrobot, which weighs less than one-fourth of a penny, has four sets of wings that are each driven by a soft actuator. Credits: Courtesy of the researchers

The more surface area the actuator has, the less voltage is required. So, Chen and his team build these artificial muscles by alternating between as many ultrathin layers of elastomer and electrode as they can. As elastomer layers get thinner, they become more unstable.

For the first time, the researchers were able to create an actuator with 20 layers, each of which is 10 micrometers in thickness (about the diameter of a red blood cell). But they had to reinvent parts of the fabrication process to get there.

One major roadblock came from the spin coating process. During spin coating, an elastomer is poured onto a flat surface and rapidly rotated, and the centrifugal force pulls the film outward to make it thinner.

“In this process, air comes back into the elastomer and creates a lot of microscopic air bubbles. The diameter of these air bubbles is barely 1 micrometer, so previously we just sort of ignored them. But when you get thinner and thinner layers, the effect of the air bubbles becomes stronger and stronger. That is traditionally why people haven’t been able to make these very thin layers,” Chen explains.

He and his collaborators found that if they perform a vacuuming process immediately after spin coating, while the elastomer was still wet, it removes the air bubbles. Then, they bake the elastomer to dry it.

Removing these defects increases the power output of the actuator by more than 300 percent and significantly improves its lifespan, Chen says.

The researchers also optimized the thin electrodes, which are composed of carbon nanotubes, super-strong rolls of carbon that are about 1/50,000 the diameter of human hair. Higher concentrations of carbon nanotubes increase the actuator’s power output and reduce voltage, but dense layers also contain more defects.

For instance, the carbon nanotubes have sharp ends and can pierce the elastomer, which causes the device to short out, Chen explains. After much trial and error, the researchers found the optimal concentration.

Another problem comes from the curing stage — as more layers are added, the actuator takes longer and longer to dry.

“The first time I asked my student to make a multilayer actuator, once he got to 12 layers, he had to wait two days for it to cure. That is totally not sustainable, especially if you want to scale up to more layers,” Chen says.

They found that baking each layer for a few minutes immediately after the carbon nanotubes are transferred to the elastomer cuts down the curing time as more layers are added.

Best-in-class performance

After using this technique to create a 20-layer artificial muscle, they tested it against their previous six-layer version and state-of-the-art, rigid actuators.

During liftoff experiments, the 20-layer actuator, which requires less than 500 volts to operate, exerted enough power to give the robot a lift-to-weight ratio of 3.7 to 1, so it could carry items that are nearly three times its weight.

“We demonstrate that this robot, weighing less than a gram, flies for the longest time with the smallest error during a hovering flight,” says Kevin Chen. Credits: Courtesy of the researchers

They also demonstrated a 20-second hovering flight, which Chen says is the longest ever recorded by a sub-gram robot. Their hovering robot held its position more stably than any of the others. The 20-layer actuator was still working smoothly after being driven for more than 2 million cycles, far outpacing the lifespan of other actuators.

“Two years ago, we created the most power-dense actuator and it could barely fly. We started to wonder, can soft robots ever compete with rigid robots? We observed one defect after another, so we kept working and we solved one fabrication problem after another, and now the soft actuator’s performance is catching up. They are even a little bit better than the state-of-the-art rigid ones. And there are still a number of fabrication processes in material science that we don’t understand. So, I am very excited to continue to reduce actuation voltage,” he says.

Chen looks forward to collaborating with Niroui to build actuators in a clean room at MIT.nano and leverage nanofabrication techniques. Now, his team is limited to how thin they can make the layers due to dust in the air and a maximum spin coating speed. Working in a clean room eliminates this problem and would allow them to use methods, such as doctor blading, that are more precise than spin coating.

While Chen is thrilled about producing 10-micrometer actuator layers, his hope is to reduce the thickness to only 1 micrometer, which would open the door to many applications for these insect-sized robots.

This work is supported, in part, by the MIT Research Laboratory of Electronics and a Mathworks Graduate Fellowship.

• PAPER – High Lift Micro-Aerial-Robot Powered by Low Voltage and Long Endurance Dielectric Elastomer Actuators. Zhijian Ren, Suhan Kim, Xiang Ji, Weikun Zhu, Farnaz Niroui, Jing Kong and Yufeng Chen, Advanced Materials. Accepted Author Manuscript 2106757. https://doi.org/10.1002/adma.202106757

Lilly interviews Stephanie Schneider, a PhD candidate at Stanford working on unconventional space robotics. Schneider explains her work on Reachbot, a long-reach crawling and anchoring robot, which repurposes extendable booms for mobile manipulation. They discuss the challenges and exciting elements of robotic prototyping for low-gravity or otherwise unique environments.

Stephanie Schneider

Stephanie Schneider is a Ph.D. candidate in Aeronautics and Astronautics in the Autonomous Systems Laboratory at Stanford, working with Professor Marco Pavone. She received her BS in Mechanical Engineering from Cornell University in 2014. Stephanie’s research interests include real-time spacecraft motion-planning, grasping and manipulation in space, and adaptive control for autonomous robotics.

Links

• Download mp3 (26.6 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

Congratulations to Huy Ha and Shuran Song who have won the CoRL 2021 best system paper award!

Their work, FlingBot: the unreasonable effectiveness of dynamic manipulations for cloth unfolding, was highly praised by the judging committee. “To me, this paper constitutes the most impressive account of both simulated and real-world cloth manipulation to date.”, commented one of the reviewers.

Below, the authors tell us more about their work, the methodology, and what they are planning next.

What is the topic of the research in your paper?

In my most recent publication with my advisor, Professor Shuran Song, we studied the task of cloth unfolding. The goal of the task is to manipulate a cloth from a crumpled initial state to an unfolded state, which is equivalent to maximizing the coverage of the cloth on the workspace.

Could you tell us about the implications of your research and why it is an interesting area for study?

Historically, most robotic manipulation research topics, such as grasp planning, are concerned with rigid objects, which have only 6 degrees of freedom since their geometry does not change. This allows one to apply the typical state estimation – task & motion planning pipeline in robotics. In contrast, deformable objects could bend and stretch in arbitrary directions, leading to infinite degrees of freedom. It’s unclear what the state of the cloth should even be. In addition, deformable objects such as clothes could experience severe self occlusion – given a crumpled piece of cloth, it’s difficult to identify whether it’s a shirt, jacket, or pair of pants. Therefore, cloth unfolding is a typical first step of cloth manipulation pipelines, since it reveals key features of the cloth for downstream perception and manipulation.

Despite the abundance of sophisticated methods for cloth unfolding over the years, they typically only address the easy case (where the cloth already starts off mostly unfolded) or take upwards of a hundred steps for challenging cases. These prior works all use single arm quasi-static actions, such as pick and place, which is slow and limited by the physical reach range of the system.

Could you explain your methodology?

In our daily lives, humans typically use both hands to manipulate cloths, and with as little as a single high velocity fling or two, we can unfold an initially crumpled cloth. Based on this observation, our key idea is simple: Use dual arm dynamic actions for cloth unfolding.

FlingBot is a self-supervised framework for cloth unfolding which uses a pick, stretch, and fling primitive for a dual-arm setup from visual observations. There are three key components to our approach. First is the decision to use a high velocity dynamic action. By relying on cloths’ mass combined with a high-velocity throw to do most of its work, a dynamic flinging policy can unfold cloths much more efficiently than a quasi-static policy. Second is a dual-arm grasp parameterization which makes satisfying collision safety constraints easy. By treating a dual-arm grasp not as two points but as a line with a rotation and length, we can directly constrain the rotation and length of the line to ensure arms do not cross over each other and do not try to grasp too close to each other. Third is our choice of using Spatial Action Maps, which learns translational, rotational, and scale equivariant value maps, and allows for sample efficient learning.

What were your main findings?

We found that dynamic actions have three desirable properties over quasi-static actions for the task of cloth unfolding. First, they are efficient – FlingBot achieves over 80% coverage within 3 actions on novel cloths. Second, they are generalizable – trained on only square cloths, FlingBot also generalizes to T-shirts. Third, they expand the system’s effective reach range – even when FlingBot can’t fully lift or stretch a cloth larger than the system’s physical reach range, it’s able to use high velocity flings to unfold the cloth.

After training and evaluating our model in simulation, we deployed and finetuned our model on a real world dual-arm system, which achieves above 80% coverage for all cloth categories. Meanwhile, the quasi-static pick & place baseline was only able to achieve around 40% coverage.

What further work are you planning in this area?

Although we motivated cloth unfolding as a precursor for downstream modules such as cloth state estimation, unfolding could also benefit from state estimation. For instance, if the system is confident it has identified the shoulders of the shirt in its state estimation, the unfolding policy could directly grasp the shoulders and unfold the shirt in one step. Based on this observation, we are currently working on a cloth unfolding and state estimation approach which can learn in a self-supervised manner in the real world.

About the authors

Huy Ha is a Ph.D. student in Computer Science at Columbia University. He is advised by Professor Shuran Song and is a member of the Columbia Artificial Intelligence and Robotics (CAIR) lab.

Shuran Song is an assistant professor in computer science department at Columbia University, where she directs the Columbia Artificial Intelligence and Robotics (CAIR) Lab. Her research focuses on computer vision and robotics. She’s interested in developing algorithms that enable intelligent systems to learn from their interactions with the physical world, and autonomously acquire the perception and manipulation skills necessary to execute complex tasks and assist people.

Find out more

• Read the paper on arXiv.
• The videos of the real-world experiments and code are available here, as is a video of the authors’ presentation at CoRL.
• Read more about the winning and shortlisted papers for the CoRL awards here.

AI-designed (C-shaped) organisms push loose stem cells (white) into piles as they move through their environment. Credit: Douglas Blackiston and Sam Kriegman

By Joshua Brown, University of Vermont Communications

To persist, life must reproduce. Over billions of years, organisms have evolved many ways of replicating, from budding plants to sexual animals to invading viruses.

Now scientists at the University of Vermont, Tufts University, and the Wyss Institute for Biologically Inspired Engineering at Harvard University have discovered an entirely new form of biological reproduction—and applied their discovery to create the first-ever, self-replicating living robots.

The same team that built the first living robots (“Xenobots,” assembled from frog cells—reported in 2020) has discovered that these computer-designed and hand-assembled organisms can swim out into their tiny dish, find single cells, gather hundreds of them together, and assemble “baby” Xenobots inside their Pac-Man-shaped “mouth”—that, a few days later, become new Xenobots that look and move just like themselves.

And then these new Xenobots can go out, find cells, and build copies of themselves. Again and again.

“With the right design—they will spontaneously self-replicate,” says Joshua Bongard, Ph.D., a computer scientist and robotics expert at the University of Vermont who co-led the new research.

The results of the new research were published in the Proceedings of the National Academy of Sciences.

Into the Unknown

In a Xenopus laevis frog, these embryonic cells would develop into skin. “They would be sitting on the outside of a tadpole, keeping out pathogens and redistributing mucus,” says Michael Levin, Ph.D., a professor of biology and director of the Allen Discovery Center at Tufts University and co-leader of the new research. “But we’re putting them into a novel context. We’re giving them a chance to reimagine their multicellularity.” Levin is also an Associate Faculty member at the Wyss Institute.

As Pac-man-shaped Xenobot “parents” move around their environment, they collect loose stem cells in their “mouths” that, over time, aggregate to create “offspring” Xenobots that develop to look just like their creators. Credit: Doug Blackiston and Sam Kriegman

And what they imagine is something far different than skin. “People have thought for quite a long time that we’ve worked out all the ways that life can reproduce or replicate. But this is something that’s never been observed before,” says co-author Douglas Blackiston, Ph.D., the senior scientist at Tufts University and the Wyss Institute who assembled the Xenobot “parents” and developed the biological portion of the new study.

“This is profound,” says Levin. “These cells have the genome of a frog, but, freed from becoming tadpoles, they use their collective intelligence, a plasticity, to do something astounding.” In earlier experiments, the scientists were amazed that Xenobots could be designed to achieve simple tasks. Now they are stunned that these biological objects—a computer-designed collection of cells—will spontaneously replicate. “We have the full, unaltered frog genome,” says Levin, “but it gave no hint that these cells can work together on this new task,” of gathering and then compressing separated cells into working self-copies.

“These are frog cells replicating in a way that is very different from how frogs do it. No animal or plant known to science replicates in this way,” says Sam Kriegman, Ph.D., the lead author on the new study, who completed his Ph.D. in Bongard’s lab at UVM and is now a post-doctoral researcher at Tuft’s Allen Center and Harvard University’s Wyss Institute for Biologically Inspired Engineering.

On its own, the Xenobot parent, made of some 3,000 cells, forms a sphere. “These can make children but then the system normally dies out after that. It’s very hard, actually, to get the system to keep reproducing,” says Kriegman. But with an artificial intelligence program working on the Deep Green supercomputer cluster at UVM’s Vermont Advanced Computing Core, an evolutionary algorithm was able to test billions of body shapes in simulation—triangles, squares, pyramids, starfish—to find ones that allowed the cells to be more effective at the motion-based “kinematic” replication reported in the new research.

“We asked the supercomputer at UVM to figure out how to adjust the shape of the initial parents, and the AI came up with some strange designs after months of chugging away, including one that resembled Pac-Man,” says Kriegman. “It’s very non-intuitive. It looks very simple, but it’s not something a human engineer would come up with. Why one tiny mouth? Why not five? We sent the results to Doug and he built these Pac-Man-shaped parent Xenobots. Then those parents built children, who built grandchildren, who built great-grandchildren, who built great-great-grandchildren.” In other words, the right design greatly extended the number of generations.

Kinematic replication is well-known at the level of molecules—but it has never been observed before at the scale of whole cells or organisms.

An AI-designed “parent” organism (C shape; red) beside stem cells that have been compressed into a ball (“offspring”; green). Credit: Douglas Blackiston and Sam Kriegman

“We’ve discovered that there is this previously unknown space within organisms, or living systems, and it’s a vast space,” says Bongard. “How do we then go about exploring that space? We found Xenobots that walk. We found Xenobots that swim. And now, in this study, we’ve found Xenobots that kinematically replicate. What else is out there?”

Or, as the scientists write in the Proceedings of the National Academy of Sciences study: “life harbors surprising behaviors just below the surface, waiting to be uncovered.”

Responding to Risk

Some people may find this exhilarating. Others may react with concern, or even terror, to the notion of a self-replicating biotechnology. For the team of scientists, the goal is deeper understanding.

“We are working to understand this property: replication. The world and technologies are rapidly changing. It’s important, for society as a whole, that we study and understand how this works,” says Bongard. These millimeter-sized living machines, entirely contained in a laboratory, easily extinguished, and vetted by federal, state and institutional ethics experts, “are not what keep me awake at night. What presents risk is the next pandemic; accelerating ecosystem damage from pollution; intensifying threats from climate change,” says UVM’s Bongard. “This is an ideal system in which to study self-replicating systems. We have a moral imperative to understand the conditions under which we can control it, direct it, douse it, exaggerate it.”

Bongard points to the COVID epidemic and the hunt for a vaccine. “The speed at which we can produce solutions matters deeply. If we can develop technologies, learning from Xenobots, where we can quickly tell the AI: ‘We need a biological tool that does X and Y and suppresses Z,’ —that could be very beneficial. Today, that takes an exceedingly long time.” The team aims to accelerate how quickly people can go from identifying a problem to generating solutions—”like deploying living machines to pull microplastics out of waterways or build new medicines,” Bongard says.

“We need to create technological solutions that grow at the same rate as the challenges we face,” Bongard says.

And the team sees promise in the research for advancements toward regenerative medicine. “If we knew how to tell collections of cells to do what we wanted them to do, ultimately, that’s regenerative medicine—that’s the solution to traumatic injury, birth defects, cancer, and aging,” says Levin. “All of these different problems are here because we don’t know how to predict and control what groups of cells are going to build. Xenobots are a new platform for teaching us.”

The scientists behind the Xenobots participated in a live panel discussion on December 1, 2021 to discuss the latest developments in their research. Credit: Wyss Institute at Harvard University

• PUBLICATION – Kinematic self-replication in reconfigurable organisms. Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard, Proceedings of the National Academy of Sciences Dec 2021, 118 (49) e2112672118; DOI: 10.1073/pnas.2112672118

Congratulations to Tao Chen, Jie Xu and Pulkit Agrawal who have won the CoRL 2021 best paper award!

Their work, A system for general in-hand object re-orientation, was highly praised by the judging committee who commented that “the sheer scope and variation across objects tested with this method, and the range of different policy architectures and approaches tested makes this paper extremely thorough in its analysis of this reorientation task”.

Below, the authors tell us more about their work, the methodology, and what they are planning next.

What is the topic of the research in your paper?

We present a system for reorienting novel objects using an anthropomorphic robotic hand with any configuration, with the hand facing both upwards and downwards. We demonstrate the capability of reorienting over 2000 geometrically different objects in both cases. The learned controller can also reorient novel unseen objects.

Could you tell us about the implications of your research and why it is an interesting area for study?

Our learned skill (in-hand object reorientation) can enable fast pick-and-place of objects in desired orientations and locations. For example, in logistics and manufacturing, it is a common demand to pack objects into slots for kitting. Currently, this is usually achieved via a two-stage process involving re-grasping. Our system will be able to achieve it in one step, which can substantially improve the packing speed and boost the manufacturing efficiency.

Another application is enabling robots to operate a wider variety of tools. The most common end-effector in industrial robots is a parallel-jaw gripper, partially due to its simplicity in control. However, such an end-effector is physically unable to handle many tools we see in our daily life. For example, even using pliers is difficult for such a gripper as it cannot dexterously move one handle back and forth. Our system will allow a multi-fingered hand to dexterously manipulate such tools, which opens up a new area for robotics applications.

Could you explain your methodology?

We use a model-free reinforcement learning algorithm to train the controller for reorienting objects. In-hand object reorientation is a challenging contact-rich task. It requires a tremendous amount of training. To speed up the learning process, we first train the policy with privileged state information such as object velocities. Using the privileged state information drastically improves the learning speed. Other than this, we also found that providing a good initialization on the hand and object pose is critical for training the controller to reorient objects when the hand faces downward. In addition, we develop a technique to facilitate the training by building a curriculum on gravitational acceleration. We call this technique “gravity curriculum”.

With these techniques, we are able to train a controller that can reorient many objects even with a downward-facing hand. However, a practical concern of the learned controller is that it makes use of privileged state information, which can be nontrivial to get in the real world. For example, it is hard to measure the object’s velocity in the real world. To ensure that we can deploy a controller reliably in the real world, we use teacher-student training. We use the controller trained with the privileged state information as the teacher. Then we train a second controller (student) that does not rely on any privileged state information and hence has the potential to be deployed reliably in the real world. This student controller is trained to imitate the teacher controller using imitation learning. The training of the student controller becomes a supervised learning problem and is therefore sample-efficient. In the deployment time, we only need the student controller.

What were your main findings?

We developed a general system that can be used to train controllers that can reorient objects with either the robotic hand facing upward or downward. The same system can also be used to train controllers that use external support such as a supporting surface for object re-orientation. Such controllers learned in our system are robust and can also reorient unseen novel objects. We also identified several techniques that are important for training a controller to reorient objects with a downward-facing hand.

A priori one might believe that it is important for the robot to know about object shape in order to manipulate new shapes. Surprisingly, we find that the robot can manipulate new objects without knowing their shape. It suggests that robust control strategies mitigate the need for complex perceptual processing. In other words, we might need much simpler perceptual processing strategies than previously thought for complex manipulation tasks.

What further work are you planning in this area?

Our immediate next step is to achieve such manipulation skills on a real robotic hand. To achieve this, we will need to tackle many challenges. We will investigate overcoming the sim-to-real gap such that the simulation results can be transferred to the real world. We also plan to design new robotic hand hardware through collaboration such that the entire robotic system can be dexterous and low-cost.

About the authors

Tao Chen is a Ph.D. student in the Improbable AI Lab at MIT CSAIL, advised by Professor Pulkit Agrawal. His research interests revolve around the intersection of robot learning, manipulation, locomotion, and navigation. More recently, he has been focusing on dexterous manipulation. His research papers have been published in top AI and robotics conferences. He received his master’s degree, advised by Professor Abhinav Gupta, from the Robotics Institute at CMU, and his bachelor’s degree from Shanghai Jiao Tong University.

Jie Xu is a Ph.D. student at MIT CSAIL, advised by Professor Wojciech Matusik in the Computational Design and Fabrication Group (CDFG). He obtained a bachelor’s degree from Department of Computer Science and Technology at Tsinghua University with honours in 2016. During his undergraduate period, he worked with Professor Shi-Min Hu in the Tsinghua Graphics & Geometric Computing Group. His research mainly focuses on the intersection of Robotics, Simulation, and Machine Learning. Specifically, he is interested in the following topics: robotics control, reinforcement learning, differentiable physics-based simulation, robotics control and design co-optimization, and sim-to-real.

Dr Pulkit Agrawal is the Steven and Renee Finn Chair Professor in the Department of Electrical Engineering and Computer Science at MIT. He earned his Ph.D. from UC Berkeley and co-founded SafelyYou Inc. His research interests span robotics, deep learning, computer vision and reinforcement learning. Pulkit completed his bachelor’s at IIT Kanpur and was awarded the Director’s Gold Medal. He is a recipient of the Sony Faculty Research Award, Salesforce Research Award, Amazon Machine Learning Research Award, Signatures Fellow Award, Fulbright Science and Technology Award, Goldman Sachs Global Leadership Award, OPJEMS, and Sridhar Memorial Prize, among others.

Find out more

• Read the paper on arXiv.
• The videos of the learned policies are available here, as is a video of the authors’ presentation at CoRL.
• Read more about the winning and shortlisted papers for the CoRL awards here.

The Conference on Robot Learning (CoRL) is an annual international conference specialised in the intersection of robotics and machine learning. The fifth edition took place last week in London and virtually around the globe. Apart from the novelty of being a hybrid conference, this year the focus was put on openness. OpenReview was used for the peer review process, meaning that the reviewers’ comments and replies from the authors are public, for anyone to see. The research community suggests that open review could encourage mutual trust, respect, and openness to criticism, enable constructive and efficient quality assurance, increase transparency and accountability, facilitate wider, and more inclusive discussion, give reviewers recognition and make reviews citable [1]. You can access all CoRL 2021 papers and their corresponding reviews here. In addition, you may want to listen to all presentations, available in the conference YouTube channel.

In this post we bring you a glimpse of the conference through the most popular tweets written last week. Cool robot demos, short and sweet explanation of papers and award finalists to look forward to next year’s edition in New Zealand. Enjoy!

Robots, robots, robots!

Papers and presentations

Awards

References

• PAPER – Ten considerations for open peer review. Schmidt B, Ross-Hellauer T, van Edig X, and Moylan EC. F1000Res. 2018;7:969. Published 2018 Jun 29. doi:10.12688/f1000research.15334.1

The new microbot inspired by starfish larva stirs up plastic beads. (Image: Cornel Dillinger/ETH Zurich)

By Rahel Künzler

Among scientists, there is great interest in tiny machines that are set to revolutionise medicine. These microrobots, often only a fraction of the diameter of a hair, are made to swim through the body to deliver medication to specific areas and perform the smallest surgical procedures.

The designs of these robots are often inspired by natural microorganisms such as bacteria or algae. Now, for the first time, a research group at ETH Zurich has developed a microrobot design inspired by starfish larva, which use ciliary bands on their surface to swim and feed. The ultrasound-​activated synthetic system mimics the natural arrangements of starfish ciliary bands and leverages nonlinear acoustics to replicate the larva’s motion and manipulation techniques.

Hairs to push liquid away or suck it in

Depending on whether it is swimming or feeding, the starfish larva generates different patterns of vortices. (Image: Prakash Lab, Stanford University)

At first glance, the microrobots bear only scant similarity to starfish larva. In its larval stage, a starfish has a lobed body that measures just a few millimetres across. Meanwhile, the microrobot is a rectangle and ten times smaller, only a quarter of a millimetre across. But the two do share one important feature: a series of fine, movable hairs on the surface, called cilia.

A starfish larva is blanketed with hundreds of thousands of these hairs. Arranged in rows, they beat back and forth in a coordinated fashion, creating eddies in the surrounding water. The relative orientation of two rows determines the end result: Inclining two bands of beating cilia toward each other creates a vortex with a thrust effect, propelling the larva. On the other hand, inclining two bands away from each other creates a vortex that draws liquid in, trapping particles on which the larva feeds.

Artificial swimmers beat faster

These cilia were the key design element for the new microrobot developed by ETH researchers led by Daniel Ahmed, who is a Professor of Acoustic Robotics for life sciences and healthcare. “In the beginning,” Ahmed said, “we simply wanted to test whether we could create vortices similar to those of the starfish larva with rows of cilia inclined toward or away from each other.

To this end, the researchers used photolithography to construct a microrobot with appropriately inclined ciliary bands. They then applied ultrasound waves from an external source to make the cilia oscillate. The synthetic versions beat back and forth more than ten thousand times per second – about a thousand times faster than those of a starfish larva. And as with the larva, these beating cilia can be used to generate a vortex with a suction effect at the front and a vortex with a thrust effect at the rear, the combined effect “rocketing” the robot forward.

Besides swimming, the new microrobot can collect particles and steer them in a predetermined direction. (Video: Cornel Dillinger/ETH Zurich)

In their lab, the researchers showed that the microrobots can swim in a straight line through liquid such as water. Adding tiny plastic beads to the water made it possible to visualize the vortices created by the microrobot. The result is astonishing: both starfish larva and microrobots generate virtually identical flow patterns.

Next, the researchers arranged the ciliary bands so that a suction vortex was positioned next to a thrust vortex, imitating the feeding technique used by starfish larva. This arrangement enabled the robots to collect particles and send them out in a predetermined direction.

Ultrasound offers many advantages

Ahmed is convinced that this new type of microrobot will be ready for use in medicine in the foreseeable future. This is because a system that relies only on ultrasound offers decisive advantages: ultrasound waves are already widely used in imaging, penetrate deep inside the body, and pose no health risks.

“Our vision is to use ultrasound for propulsion, imaging and drug delivery.”

– Daniel Ahmed

The fact that this therapy requires only an ultrasound device makes it cheap, he adds, and hence suitable for use in both developed and developing countries.

Ahmed believes one initial field of application could be the treatment of gastric tumours. Uptake of conventional drugs by diffusion is inefficient, but having microrobots transport a drug specifically to the site of a stomach tumour and then deliver it there might make the drug’s uptake into tumour cells more efficient and reduce side effects.

Sharper images thanks to contrast agents

But before this vision can be realized, a major challenge remains to be overcome: imaging. Steering the tiny machines to the right place requires that a sharp image be generated in real time. The researchers have plans to make the microrobots more visible by incorporating contrast agents such as those already used in medical imaging with ultrasound.

In addition to medical applications, Ahmed anticipates this starfish-​inspired design to have important implications for the manipulation of smallest liquid volumes in research and in industry. Bands of beating cilia could execute tasks such as mixing, pumping and particle trapping.

• PAPER – Ultrasound-​activated ciliary bands for microrobotic systems inspired by starfish. Dillinger C, Nama N and Ahmed D. Nat Commun. 2021 Nov 9. doi: 10.1038/s41467-​021-26607-ycall_made

This articled was originally published at the official website of ETH Zürich. Read the original article here.

Imagine you’re in an unfamiliar neighborhood with no house numbers and I give you a photo that I took a few days ago of my house, which is not too far away. If you tried to find my house, you might follow the streets and go around the block looking for it. You might take a few wrong turns at first, but eventually you would locate my house. In the process, you would end up with a mental map of my neighborhood. The next time you’re visiting, you will likely be able to navigate to my house right away, without taking any wrong turns.

Such exploration and navigation behavior is easy for humans. What would it take for a robotic learning algorithm to enable this kind of intuitive navigation capability? To build a robot capable of exploring and navigating like this, we need to learn from diverse prior datasets in the real world. While it’s possible to collect a large amount of data from demonstrations, or even with randomized exploration, learning meaningful exploration and navigation behavior from this data can be challenging – the robot needs to generalize to unseen neighborhoods, recognize visual and dynamical similarities across scenes, and learn a representation of visual observations that is robust to distractors like weather conditions and obstacles. Since such factors can be hard to model and transfer from simulated environments, we tackle these problems by teaching the robot to explore using only real-world data.

Formally, we studied the problem of goal-directed exploration for visual navigation in novel environments. A robot is tasked with navigating to a goal location , specified by an image taken at . Our method uses an offline dataset of trajectories, over 40 hours of interactions in the real-world, to learn navigational affordances and builds a compressed representation of perceptual inputs. We deploy our method on a mobile robotic system in industrial and recreational outdoor areas around the city of Berkeley. RECON can discover a new goal in a previously unexplored environment in under 10 minutes, and in the process build a “mental map” of that environment that allows it to then reach goals again in just 20 seconds. Additionally, we make this real-world offline dataset publicly available for use in future research.

Rapid Exploration Controllers for Outcome-driven Navigation

RECON, or Rapid Exploration Controllers for Outcome-driven Navigation, explores new environments by “imagining” potential goal images and attempting to reach them. This exploration allows RECON to incrementally gather information about the new environment.

Our method consists of two components that enable it to explore new environments. The first component is a learned representation of goals. This representation ignores task-irrelevant distractors, allowing the agent to quickly adapt to novel settings. The second component is a topological graph. Our method learns both components using datasets or real-world robot interactions gathered in prior work. Leveraging such large datasets allows our method to generalize to new environments and scale beyond the original dataset.

Learning to Represent Goals

A useful strategy to learn complex goal-reaching behavior in an unsupervised manner is for an agent to set its own goals, based on its capabilities, and attempt to reach them. In fact, humans are very proficient at setting abstract goals for themselves in an effort to learn diverse skills. Recent progress in reinforcement learning and robotics has also shown that teaching agents to set its own goals by “imagining” them can result in learning of impressive unsupervised goal-reaching skills. To be able to “imagine”, or sample, such goals, we need to build a prior distribution over the goals seen during training.

For our case, where goals are represented by high-dimensional images, how should we sample goals for exploration? Instead of explicitly sampling goal images, we instead have the agent learn a compact representation of latent goals, allowing us to perform exploration by sampling new latent goal representations, rather than by sampling images. This representation of goals is learned from context-goal pairs previously seen by the robot. We use a variational information bottleneck to learn these representations because it provides two important properties. First, it learns representations that throw away irrelevant information, such as lighting and pixel noise. Second, the variational information bottleneck packs the representations together so that they look like a chosen prior distribution. This is useful because we can then sample imaginary representations by sampling from this prior distribution.

The architecture for learning a prior distribution for these representations is shown below. As the encoder and decoder are conditioned on the context, the representation only encodes information about relative location of the goal from the context – this allows the model to represent feasible goals. If, instead, we had a typical VAE (in which the input images are autoencoded), the samples from the prior over these representations would not necessarily represent goals that are reachable from the current state. This distinction is crucial when exploring new environments, where most states from the training environments are not valid goals.

The architecture for learning a prior over goals in RECON. The context-conditioned embedding learns to represent feasible goals.

To understand the importance of learning this representation, we run a simple experiment where the robot is asked to explore in an undirected manner starting from the yellow circle in the figure below. We find that sampling representations from the learned prior greatly accelerates the diversity of exploration trajectories and allows a wider area to be explored. In the absence of a prior over previously seen goals, using random actions to explore the environment can be quite inefficient. Sampling from the prior distribution and attempting to reach these “imagined” goals allows RECON to explore the environment efficiently.

Sampling from a learned prior allows the robot to explore 5 times faster than using random actions.

Goal-Directed Exploration with a Topological Memory

We combine this goal sampling scheme with a topological memory to incrementally build a “mental map” of the new environment. This map provides an estimate of the exploration frontier as well as guidance for subsequent exploration. In a new environment, RECON encourages the robot to explore at the frontier of the map – while the robot is not at the frontier, RECON directs it to navigate to a previously seen subgoal at the frontier of the map.

At the frontier, RECON uses the learned goal representation to learn a prior over goals it can reliably navigate to and are thus, feasible to reach. RECON uses this goal representation to sample, or “imagine”, a feasible goal that helps it explore the environment. This effectively means that, when placed in a new environment, if RECON does not know where the target is, it “imagines” a suitable subgoal that it can drive towards to explore and collects information, until it believes it can reach the target goal image. This allows RECON to “search” for the goal in an unknown environment, all the while building up its mental map. Note that the objective of the topological graph is to build a compact map of the environment and encourage the robot to reach the frontier; it does not inform goal sampling once the robot is at the frontier.

Illustration of the exploration algorithm of RECON.

Learning from Diverse Real-world Data

We train these models in RECON entirely using offline data collected in a diverse range of outdoor environments. Interestingly, we were able to train this model using data collected for two independent projects in the fall of 2019 and spring of 2020, and were successful in deploying the model to explore novel environments and navigate to goals during late 2020 and the spring of 2021. This offline dataset of trajectories consists of over 40 hours of data, including off-road navigation, driving through parks in Berkeley and Oakland, parking lots, sidewalks and more, and is an excellent example of noisy real-world data with visual distractors like lighting, seasons (rain, twilight etc.), dynamic obstacles etc. The dataset consists of a mixture of teleoperated trajectories (2-3 hours) and open-loop safety controllers programmed to collect random data in a self-supervised manner. This dataset presents an exciting benchmark for robotic learning in real-world environments due to the challenges posed by offline learning of control, representation learning from high-dimensional visual observations, generalization to out-of-distribution environments and test-time adaptation.

We are releasing this dataset publicly to support future research in machine learning from real-world interaction datasets, check out the dataset page for more information.

We train from diverse offline data (top) and test in new environments (bottom).

RECON in Action

Putting these components together, let’s see how RECON performs when deployed in a park near Berkeley. Note that the robot has never seen images from this park before. We placed the robot in a corner of the park and provided a target image of a white cabin door. In the animation below, we see RECON exploring and successfully finding the desired goal. “Run 1” corresponds to the exploration process in a novel environment, guided by a user-specified target image on the left. After it finds the goal, RECON uses the mental map to distill its experience in the environment to find the shortest path for subsequent traversals. In “Run 2”, RECON follows this path to navigate directly to the goal without looking around.

In “Run 1”, RECON explores a new environment and builds a topological mental map. In “Run 2”, it uses this mental map to quickly navigate to a user-specified goal in the environment.

An illustration of this two-step process from an overhead view is show below, showing the paths taken by the robot in subsequent traversals of the environment:

(Left) The goal specified by the user. (Right) The path taken by the robot when exploring for the first time (shown in cyan) to build a mental map with nodes (shown in white), and the path it takes when revisiting the same goal using the mental map (shown in red).

Deploying in Novel Environments

To evaluate the performance of RECON in novel environments, study its behavior under a range of perturbations and understand the contributions of its components, we run extensive real-world experiments in the hills of Berkeley and Richmond, which have a diverse terrain and a wide variety of testing environments.

We compare RECON to five baselines – RND, InfoBot, Active Neural SLAM, ViNG and Episodic Curiosity – each trained on the same offline trajectory dataset as our method, and fine-tuned in the target environment with online interaction. Note that this data is collected from past environments and contains no data from the target environment. The figure below shows the trajectories taken by the different methods for one such environment.

We find that only RECON (and a variant) is able to successfully discover the goal in over 30 minutes of exploration, while all other baselines result in collision (see figure for an overhead visualization). We visualize successful trajectories discovered by RECON in four other environments below.

(Top) When comparing to other baselines, only RECON is able to successfully find the goal. (Bottom) Trajectories to goals in four other environments discovered by RECON.

Quantitatively, we observe that our method finds goals over 50% faster than the best prior method; after discovering the goal and building a topological map of the environment, it can navigate to goals in that environment over 25% faster than the best alternative method.

Quantitative results in novel environments. RECON outperforms all baselines by over 50%.

Exploring Non-Stationary Environments

One of the important challenges in designing real-world robotic navigation systems is handling differences between training scenarios and testing scenarios. Typically, systems are developed in well-controlled environments, but are deployed in less structured environments. Further, the environments where robots are deployed often change over time, so tuning a system to perform well on a cloudy day might degrade performance on a sunny day. RECON uses explicit representation learning in attempts to handle this sort of non-stationary dynamics.

Our final experiment tested how changes in the environment affected the performance of RECON. We first had RECON explore a new “junkyard” to learn to reach a blue dumpster. Then, without any more supervision or exploration, we evaluated the learned policy when presented with previously unseen obstacles (trash cans, traffic cones, a car) and weather conditions (sunny, overcast, twilight). As shown below, RECON is able to successfully navigate to the goal in these scenarios, showing that the learned representations are invariant to visual distractors that do not affect the robot’s decisions to reach the goal.

First-person videos of RECON successfully navigating to a “blue dumpster” in the presence of novel obstacles (above) and varying weather conditions (below).

What’s Next?

The problem setup studied in this paper – using past experience to accelerate learning in a new environment – is reflective of several real-world robotics scenarios. RECON provides a robust way to solve this problem by using a combination of goal sampling and topological memory.

A mobile robot capable of reliably exploring and visually observing real-world environments can be a great tool for a wide variety of useful applications such as search and rescue, inspecting large offices or warehouses, finding leaks in oil pipelines or making rounds at a hospital, delivering mail in suburban communities. We demonstrated simplified versions of such applications in an earlier project, where the robot has prior experience in the deployment environment; RECON enables these results to scale beyond the training set of environments and results in a truly open-world learning system that can adapt to novel environments on deployment.

We are also releasing the aforementioned offline trajectory dataset, with hours of real-world interaction of a mobile ground robot in a variety of outdoor environments. We hope that this dataset can support future research in machine learning using real-world data for visual navigation applications. The dataset is also a rich source of sequential data from a multitude of sensors and can be used to test sequence prediction models including, but not limited to, video prediction, LiDAR, GPS etc. More information about the dataset can be found in the full-text article.

This blog post is based on our paper Rapid Exploration for Open-World Navigation with Latent Goal Models, which will be presented as an Oral Talk at the 5th Annual Conference on Robot Learning in London, UK on November 8-11, 2021. You can find more information about our results and the dataset release on the project page.

Big thanks to Sergey Levine and Benjamin Eysenbach for helpful comments on an earlier draft of this article.

Proteus and sunset with the Swiss mountains in the background at Lake Thun (photo credit: Gallus Kaufmann)

In 2018, Christian Engler felt he’d studied enough theory at the ETH Zurich and longed to put it all into practice. It was evident to Christian that the best way to get hands-on experience was to start something himself. Others were not so sure. Especially when they heard about his ambition to revive a project from high school.

The project involved underwater robots, also known as Remotely Operated Vehicles (ROVs). But now Christian wanted to step it up a gear. He not only wanted to build an ROV, but he also wanted to take part at the international MATE ROV Competition 2019 – the biggest ROV competition in the world. His passion motivated a further seven students and Tethys Robotics was born.

Start of the journey

To be able to compete at a high level, Tethys Robotics needed support. The main supporter was and still is Professor Siegwart and his Autonomous Systems Lab (ASL). Due to a previous project, SCUBO, the team had the chance to benefit from their experience and were able to use their carbon fibre shell to build their version of the underwater robot, the SCUBO 2.0.

SCUBO taking a sunbath at Lake Zurich (photo credit: Gallus Kaufmann)

While the original SCUBO was developed to film coral reefs, the new version of SCUBO had to be adapted to fulfil the tasks at the MATE ROV Competition. To prevent damaging the coral reefs, the original SCUBO had a different position and orientation of the eight actuators (also known as thrusters) at the cost of stability and control. Furthermore, no batteries were allowed at the competition inside of the robot. For that reason, the entire electronics, software interface and controlling had to be redesigned to be able to take part at the competition.

Essentially, the sensors of both SCUBO versions are the same consisting of Attitude and Heading Reference System (AHRS), pressure sensor, stereo camera and temperature sensor.

The only downside was that the ASL did not have enough space to accommodate us. Therefore, we ended up working in the lab’s basement. But this did not affect our motivation – now complete with ‘garage start-up’ vibe – or the successful outcome. Tethys Robotics was the first Swiss team ever to compete. Out of 75 teams, Tethys came in at 9th place.

Real world applications

After the competition, SCUBO 2.0 was showcased at various exhibitions. This is how we came across the divers from the Swiss Explosive Ordnance Disposal (EOD), whose mission is to retrieve lost ammunition from Swiss lakes. Since this is a dangerous and very challenging task for divers, Tethys started collaborating on an underwater robot up to the task. The robot had to be modular, lightweight, and easily deployable. With the support of the ASL, the algorithms developed for drones have been implemented to the underwater drones of Tethys Robotics. And by having multiple tests at the operation site with the EOD divers and blank ammunition, the new underwater robot Proteus was developed with a real world application in mind. With their help, Proteus has been developed to be modular and more focused to be used as a diver buddy instead of working individually. In particular, the underwater drone can be used as a lift for material and as a communication device to the top side.

https://www.youtube.com/watch?v=quiUgTc_NQM

Challenges and future of Proteus

There have been many changes to the design and the application field between the competition robot and the new Proteus. Proteus is able to be deployed in every Swiss lake (+300 m depth rating) and is powered again, as the original SCUBO, by batteries. The communication and camera streams are transmitted via a fiber optics cable to a control station on the shore. Moreover, in addition to the sensors used in SCUBO, Proteus has been upgraded with sensors mainly used for localization and orientation underwater which consist of a sonar, Doppler Velocity Logger (DVL), acoustic Short Baseline system (SBL) and event cameras.

Proteus mission setup (photo credit: Gallus Kaufmann)

The current research focuses on the improvement of the underwater localization. The different sensors are not accurate, fast or robust enough to be used in Swiss lakes and in particular rivers. Therefore, a sensor fusion algorithm is being implemented to combine the advantages of the different sensors. Furthermore, Tethys Robotics is trying to find further applications and an appropriate market segment to position the developed underwater drones. By finding further partners and interest, the project could make the transition to a company and develop the prototypes to final products.

Although reinforcement learning has shown success in domains such as robotics, chip placement and playing video games, it is usually intractable in its most general form. In particular, deciding when and how to visit new states in the hopes of learning more about the environment can be challenging, especially when the reward signal is uninformative. These questions of reward specification and exploration are closely connected — the more directed and “well shaped” a reward function is, the easier the problem of exploration becomes. The answer to the question of how to explore most effectively is likely to be closely informed by the particular choice of how we specify rewards.

For unstructured problem settings such as robotic manipulation and navigation — areas where RL holds substantial promise for enabling better real-world intelligent agents — reward specification is often the key factor preventing us from tackling more difficult tasks. The challenge of effective reward specification is two-fold: we require reward functions that can be specified in the real world without significantly instrumenting the environment, but also effectively guide the agent to solve difficult exploration problems. In our recent work, we address this challenge by designing a reward specification technique that naturally incentivizes exploration and enables agents to explore environments in a directed way.

Outcome Driven RL and Classifier Based Rewards

While RL in its most general form can be quite difficult to tackle, we can consider a more controlled set of subproblems which are more tractable while still encompassing a significant set of interesting problems. In particular, we consider a subclass of problems which has been referred to as outcome driven RL. In outcome driven RL problems, the agent is not simply tasked with exploring the environment until it chances upon reward, but instead is provided with examples of successful outcomes in the environment. These successful outcomes can then be used to infer a suitable reward function that can be optimized to solve the desired problems in new scenarios.

More concretely, in outcome driven RL problems, a human supervisor first provides a set of successful outcome examples , representing states in which the desired task has been accomplished. Given these outcome examples, a suitable reward function can be inferred that encourages an agent to achieve the desired outcome examples. In many ways, this problem is analogous to that of inverse reinforcement learning, but only requires examples of successful states rather than full expert demonstrations.

When thinking about how to actually infer the desired reward function from successful outcome examples , the simplest technique that comes to mind is to simply treat the reward inference problem as a classification problem – “Is the current state a successful outcome or not?” Prior work has implemented this intuition, inferring rewards by training a simple binary classifier to distinguish whether a particular state is a successful outcome or not, using the set of provided goal states as positives, and all on-policy samples as negatives. The algorithm then assigns rewards to a particular state using the success probabilities from the classifier. This has been shown to have a close connection to the framework of inverse reinforcement learning.

Classifier-based methods provide a much more intuitive way to specify desired outcomes, removing the need for hand-designed reward functions or demonstrations:

These classifier-based methods have achieved promising results on robotics tasks such as fabric placement, mug pushing, bead and screw manipulation, and more. However, these successes tend to be limited to simple shorter-horizon tasks, where relatively little exploration is required to find the goal.

What’s Missing?

Standard success classifiers in RL suffer from the key issue of overconfidence, which prevents them from providing useful shaping for hard exploration tasks. To understand why, let’s consider a toy 2D maze environment where the agent must navigate in a zigzag path from the top left to the bottom right corner. During training, classifier-based methods would label all on-policy states as negatives and user-provided outcome examples as positives. A typical neural network classifier would easily assign success probabilities of 0 to all visited states, resulting in uninformative rewards in the intermediate stages when the goal has not been reached.

Since such rewards would not be useful for guiding the agent in any particular direction, prior works tend to regularize their classifiers using methods like weight decay or mixup, which allow for more smoothly increasing rewards as we approach the successful outcome states. However, while this works on many shorter-horizon tasks, such methods can actually produce very misleading rewards. For example, on the 2D maze, a regularized classifier would assign relatively high rewards to states on the opposite side of the wall from the true goal, since they are close to the goal in x-y space. This causes the agent to get stuck in a local optima, never bothering to explore beyond the final wall!

In fact, this is exactly what happens in practice:

Uncertainty-Aware Rewards through CNML

As discussed above, the key issue with unregularized success classifiers for RL is overconfidence — by immediately assigning rewards of 0 to all visited states, we close off many paths that might eventually lead to the goal. Ideally, we would like our classifier to have an appropriate notion of uncertainty when outputting success probabilities, so that we can avoid excessively low rewards without suffering from the misleading local optima that result from regularization.

Conditional Normalized Maximum Likelihood (CNML)

One method particularly well-suited for this task is Conditional Normalized Maximum Likelihood (CNML). The concept of normalized maximum likelihood (NML) has typically been used in the Bayesian inference literature for model selection, to implement the minimum description length principle. In more recent work, NML has been adapted to the conditional setting to produce models that are much better calibrated and maintain a notion of uncertainty, while achieving optimal worst case classification regret. Given the challenges of overconfidence described above, this is an ideal choice for the problem of reward inference.

Rather than simply training models via maximum likelihood, CNML performs a more complex inference procedure to produce likelihoods for any point that is being queried for its label. Intuitively, CNML constructs a set of different maximum likelihood problems by labeling a particular query point with every possible label value that it might take, then outputs a final prediction based on how easily it was able to adapt to each of those proposed labels given the entire dataset observed thus far. Given a particular query point , and a prior dataset , CNML solves k different maximum likelihood problems and normalizes them to produce the desired label likelihood , where represents the number of possible values that the label may take. Formally, given a model , loss function , training dataset with classes , and a new query point , CNML solves the following maximum likelihood problems:

It then generates predictions for each of the classes using their corresponding models, and normalizes the results for its final output:

Comparison of outputs from a standard classifier and a CNML classifier. CNML outputs more conservative predictions on points that are far from the training distribution, indicating uncertainty about those points’ true outputs. (Credit: Aurick Zhou, BAIR Blog)

Intuitively, if the query point is farther from the original training distribution represented by D, CNML will be able to more easily adapt to any arbitrary label in , making the resulting predictions closer to uniform. In this way, CNML is able to produce better calibrated predictions, and maintain a clear notion of uncertainty based on which data point is being queried.

Leveraging CNML-based classifiers for Reward Inference

Given the above background on CNML as a means to produce better calibrated classifiers, it becomes clear that this provides us a straightforward way to address the overconfidence problem with classifier based rewards in outcome driven RL. By replacing a standard maximum likelihood classifier with one trained using CNML, we are able to capture a notion of uncertainty and obtain directed exploration for outcome driven RL. In fact, in the discrete case, CNML corresponds to imposing a uniform prior on the output space — in an RL setting, this is equivalent to using a count-based exploration bonus as the reward function. This turns out to give us a very appropriate notion of uncertainty in the rewards, and solves many of the exploration challenges present in classifier based RL.

However, we don’t usually operate in the discrete case. In most cases, we use expressive function approximators and the resulting representations of different states in the world share similarities. When a CNML based classifier is learned in this scenario, with expressive function approximation, we see that it can provide more than just task agnostic exploration. In fact, it can provide a directed notion of reward shaping, which guides an agent towards the goal rather than simply encouraging it to expand the visited region naively. As visualized below, CNML encourages exploration by giving optimistic success probabilities in less-visited regions, while also providing better shaping towards the goal.

As we will show in our experimental results, this intuition scales to higher dimensional problems and more complex state and action spaces, enabling CNML based rewards to solve significantly more challenging tasks than is possible with typical classifier based rewards.

However, on closer inspection of the CNML procedure, a major challenge becomes apparent. Each time a query is made to the CNML classifier, different maximum likelihood problems need to be solved to convergence, then normalized to produce the desired likelihood. As the size of the dataset increases, as it naturally does in reinforcement learning, this becomes a prohibitively slow process. In fact, as seen in Table 1, RL with standard CNML based rewards takes around 4 hours to train a single epoch (1000 timesteps). Following this procedure blindly would take over a month to train a single RL agent, necessitating a more time efficient solution. This is where we find meta-learning to be a crucial tool.

Meta-Learning CNML Classifiers

Meta-learning is a tool that has seen a lot of use cases in few-shot learning for image classification, learning quicker optimizers and even learning more efficient RL algorithms. In essence, the idea behind meta-learning is to leverage a set of “meta-training” tasks to learn a model (and often an adaptation procedure) that can very quickly adapt to a new task drawn from the same distribution of problems.

Meta-learning techniques are particularly well suited to our class of computational problems since it involves quickly solving multiple different maximum likelihood problems to evaluate the CNML likelihood. Each the maximum likelihood problems share significant similarities with each other, enabling a meta-learning algorithm to very quickly adapt to produce solutions for each individual problem. In doing so, meta-learning provides us an effective tool for producing estimates of normalized maximum likelihood significantly more quickly than possible before.

The intuition behind how to apply meta-learning to the CNML (meta-NML) can be understood by the graphic above. For a data-set of points, meta-NML would first construct tasks, corresponding to the positive and negative maximum likelihood problems for each datapoint in the dataset. Given these constructed tasks as a (meta) training set, a meta–learning algorithm can be applied to learn a model that can very quickly be adapted to produce solutions to any of these maximum likelihood problems. Equipped with this scheme to very quickly solve maximum likelihood problems, producing CNML predictions around x faster than possible before. Prior work studied this problem from a Bayesian approach, but we found that it often scales poorly for the problems we considered.

Equipped with a tool for efficiently producing predictions from the CNML distribution, we can now return to the goal of solving outcome-driven RL with uncertainty aware classifiers, resulting in an algorithm we call MURAL.

MURAL: Meta-Learning Uncertainty-Aware Rewards for Automated Reinforcement Learning

To more effectively solve outcome driven RL problems, we incorporate meta-NML into the standard classifier based procedure as follows: After each epoch of RL, we sample a batch of points from the replay buffer and use them to construct meta-tasks. We then run iteration of meta-training on our model.
We assign rewards using NML, where the NML outputs are approximated using only one gradient step for each input point.

The resulting algorithm, which we call MURAL, replaces the classifier portion of standard classifier-based RL algorithms with a meta-NML model instead. Although meta-NML can only evaluate input points one at a time instead of in batches, it is substantially faster than naive CNML, and MURAL is still comparable in runtime to standard classifier-based RL, as shown in Table 1 below.

Table 1. Runtimes for a single epoch of RL on the 2D maze task.

We evaluate MURAL on a variety of navigation and robotic manipulation tasks, which present several challenges including local optima and difficult exploration. MURAL solves all of these tasks successfully, outperforming prior classifier-based methods as well as standard RL with exploration bonuses.

Visualization of behaviors learned by MURAL. MURAL is able to perform a variety of behaviors in navigation and manipulation tasks, inferring rewards from outcome examples.

Quantitative comparison of MURAL to baselines. MURAL is able to outperform baselines which perform task-agnostic exploration, standard maximum likelihood classifiers.

This suggests that using meta-NML based classifiers for outcome driven RL provides us an effective way to provide rewards for RL problems, providing benefits both in terms of exploration and directed reward shaping.

Takeaways

In conclusion, we showed how outcome driven RL can define a class of more tractable RL problems. Standard methods using classifiers can often fall short in these settings as they are unable to provide any benefits of exploration or guidance towards the goal. Leveraging a scheme for training uncertainty aware classifiers via conditional normalized maximum likelihood allows us to more effectively solve this problem, providing benefits in terms of exploration and reward shaping towards successful outcomes. The general principles defined in this work suggest that considering tractable approximations to the general RL problem may allow us to simplify the challenge of reward specification and exploration in RL while still encompassing a rich class of control problems.

This post is based on the paper “MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learning”, which was presented at ICML 2021. You can see results on our website, and we provide code to reproduce our experiments.

The IEEE International Conference on Automation Science and Engineering (CASE) is the flagship automation conference of the IEEE Robotics and Automation Society and constitutes the primary forum for cross-industry and multidisciplinary research in automation. Its goal is to provide a broad coverage and dissemination of foundational research in automation among researchers, academics, and practitioners. Here we bring you the online presentations by the finalists of the four awards given at the conference. Congratulations to all the finalists and winners!

Best student paper award

Winner

• Designing a User-Centred and Data-Driven Controller for Pushrim-Activated Power-Assisted Wheels: A Case Study
  Mahsa Khalili, H.F. Machiel Van der Loos and Jaimie Borisoff

Finalists

• Including Sparse Production Knowledge into Variational Autoencoders to Increase Anomaly Detection Reliability
  Tom Hammerbacher, Markus Lange-Hegermann, Gorden Platz

• Synthesis and Implementation of Distributed Supervisory Controllers with Communication Delays
  Lars Moormann, Reinier Hendrik Jacob Schouten, Joanna Maria Van de Mortel-Fronczak, Wan Fokkink, Jacobus E. Rooda

• Optimal Planning of Internet Data Centers Decarbonized by Hydrogen-Water-Based Energy Systems
  Jinhui Liu, Zhanbo Xu, Jiang Wu, kun liu, Xunhang Sun, Xiaohong Guan

• Deep Reinforcement Learning for Prefab Assembly Planning in Robot-Based Prefabricated Construction
  Zhu Aiyu, Gangyan Xu, Pieter Pauwels, Bauke de Vries, Meng Fang

• Singularity-Aware Motion Planning for Multi-Axis Additive Manufacturing
  Charlie C.L. Wang, Tianyu Zhang, Xiangjia Chen, Guoxin Fang, Yingjun Tian

Best conference paper award

Winner

• Extended Fabrication-Aware Convolution Learning Framework for Predicting 3D Shape Deformation in Additive Manufacturing
  Yuanxiang Wang, Cesar Ruiz, Qiang Huang

Finalists

• Probabilistic Movement Primitive Control Via Control Barrier Functions
  Mohammadreza Davoodi, Asif Iqbal, Joe Cloud, William Beksi, Nicholas Gans

• Efficient Optimization-Based Falsification of Cyber-Physical Systems with Multiple Conjunctive Requirements
  Logan Mathesen, Giulia Pedrielli, Georgios Fainekos

Best application paper award

Winner

• A Seamless Workflow for Design and Fabrication of Multimaterial Pneumatic Soft Actuators
  Lawrence Smith, Travis Hainsworth, Zachary Jordan, Xavier Bell, Robert MacCurdy

Finalists

• Dynamic Multi-Goal Motion Planning with Range Constraints for Autonomous Underwater Vehicles Following Surface Vehicles
  James McMahon, Erion Plaku

• OpenUAV Cloud Testbed: a Collaborative Design Studio for Field Robotics
  Harish Anand, Stephen A. Rees, Zhiang Chen, Ashwin Jose Poruthukaran, Sarah Bearman, Lakshmi Gana Prasad Antervedi, Jnaneshwar Das

Best healthcare automation paper award

Winner

• Hospital Beds Planning and Admission Control Policies for COVID-19 Pandemic: A Hybrid Computer Simulation Approach
  Yiruo Lu, Yongpei Guan, Xiang Zhong, Jennifer Fishe, Thanh Hogan

Finalists

• Rollout-Based Gantry Call-Back Control for Proton Therapy Systems
  Feifan Wang, Yu-Li Huang, Feng Ju

• Progress in Development of an Automated Mosquito Salivary Gland Extractor: A Step Forward to Malaria Vaccine Mass Production
  Wanze Li, Zhuoqun Zhang, Zhuohong He, Parth Vora, Alan Lai, Balazs Vagvolgyi, Simon Leonard, Anna Goodridge, Ioan Iulian Iordachita, Stephen L. Hoffman, Sumana Chakravarty, B Kim Lee Sim, Russell H. Taylor

Amit Goel, Director of Product Management for Autonomous Machines at NVIDIA, discusses the new collaboration between Open Robotics and NVIDIA. The collaboration will improve the way ROS and NVIDIA’s line of products such as Isaac SIM and the Jetson line of embedded boards operate together.

NVIDIA’s Isaac SIM lets developers build robust and scalable simulations. Dramatically reducing the costs of capturing real-world data and speeding up development time.

Their Jetson line of embedded boards is core to many robotics architectures, leveraging hardware-optimized chips for machine learning, computer vision, video processing, and more.

The improvements to ROS will allow robotics companies to better utilize the available computational power, while still developing on the robotics-centric platform familiar to many.

For the developers out there, check out the ISAAC ROS Github repository to implement this yourself.

Amit Goel

Amit Goel is Director of Product Management for Autonomous Machines at NVIDIA, where he leads the product development of NVIDIA Jetson, the most advanced platform for AI computing at the edge.

Amit has more than 15 years of experience in the technology industry working in both software and hardware design roles. Prior to joining NVIDIA in 2011, he worked as a senior software engineer at Synopsys, where he developed algorithms for statistical performance modeling of digital designs.

Amit holds a Bachelor of Engineering in electronics and communication from Delhi College of Engineering, a Master of Science in electrical engineering from Arizona State University, and an MBA from the University of California at Berkeley.

Links

• Download mp3 (29.8 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

MIT researchers have developed a system that improves the speed and agility of legged robots as they jump across gaps in the terrain. Credits: Photo courtesy of the researchers

By Adam Zewe | MIT News Office

A loping cheetah dashes across a rolling field, bounding over sudden gaps in the rugged terrain. The movement may look effortless, but getting a robot to move this way is an altogether different prospect.

In recent years, four-legged robots inspired by the movement of cheetahs and other animals have made great leaps forward, yet they still lag behind their mammalian counterparts when it comes to traveling across a landscape with rapid elevation changes.

“In those settings, you need to use vision in order to avoid failure. For example, stepping in a gap is difficult to avoid if you can’t see it. Although there are some existing methods for incorporating vision into legged locomotion, most of them aren’t really suitable for use with emerging agile robotic systems,” says Gabriel Margolis, a PhD student in the lab of Pulkit Agrawal, professor in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT.

Now, Margolis and his collaborators have developed a system that improves the speed and agility of legged robots as they jump across gaps in the terrain. The novel control system is split into two parts — one that processes real-time input from a video camera mounted on the front of the robot and another that translates that information into instructions for how the robot should move its body. The researchers tested their system on the MIT mini cheetah, a powerful, agile robot built in the lab of Sangbae Kim, professor of mechanical engineering.

Unlike other methods for controlling a four-legged robot, this two-part system does not require the terrain to be mapped in advance, so the robot can go anywhere. In the future, this could enable robots to charge off into the woods on an emergency response mission or climb a flight of stairs to deliver medication to an elderly shut-in.

Margolis wrote the paper with senior author Pulkit Agrawal, who heads the Improbable AI lab at MIT and is the Steven G. and Renee Finn Career Development Assistant Professor in the Department of Electrical Engineering and Computer Science; Professor Sangbae Kim in the Department of Mechanical Engineering at MIT; and fellow graduate students Tao Chen and Xiang Fu at MIT. Other co-authors include Kartik Paigwar, a graduate student at Arizona State University; and Donghyun Kim, an assistant professor at the University of Massachusetts at Amherst. The work will be presented next month at the Conference on Robot Learning.

It’s all under control

The use of two separate controllers working together makes this system especially innovative.

A controller is an algorithm that will convert the robot’s state into a set of actions for it to follow. Many blind controllers — those that do not incorporate vision — are robust and effective but only enable robots to walk over continuous terrain.

Vision is such a complex sensory input to process that these algorithms are unable to handle it efficiently. Systems that do incorporate vision usually rely on a “heightmap” of the terrain, which must be either preconstructed or generated on the fly, a process that is typically slow and prone to failure if the heightmap is incorrect.

To develop their system, the researchers took the best elements from these robust, blind controllers and combined them with a separate module that handles vision in real-time.

The robot’s camera captures depth images of the upcoming terrain, which are fed to a high-level controller along with information about the state of the robot’s body (joint angles, body orientation, etc.). The high-level controller is a neural network that “learns” from experience.

That neural network outputs a target trajectory, which the second controller uses to come up with torques for each of the robot’s 12 joints. This low-level controller is not a neural network and instead relies on a set of concise, physical equations that describe the robot’s motion.

“The hierarchy, including the use of this low-level controller, enables us to constrain the robot’s behavior so it is more well-behaved. With this low-level controller, we are using well-specified models that we can impose constraints on, which isn’t usually possible in a learning-based network,” Margolis says.

Teaching the network

The researchers used the trial-and-error method known as reinforcement learning to train the high-level controller. They conducted simulations of the robot running across hundreds of different discontinuous terrains and rewarded it for successful crossings.

Over time, the algorithm learned which actions maximized the reward.

Then they built a physical, gapped terrain with a set of wooden planks and put their control scheme to the test using the mini cheetah.

“It was definitely fun to work with a robot that was designed in-house at MIT by some of our collaborators. The mini cheetah is a great platform because it is modular and made mostly from parts that you can order online, so if we wanted a new battery or camera, it was just a simple matter of ordering it from a regular supplier and, with a little bit of help from Sangbae’s lab, installing it,” Margolis says.

From left to right: PhD students Tao Chen and Gabriel Margolis; Pulkit Agrawal, the Steven G. and Renee Finn Career Development Assistant Professor in the Department of Electrical Engineering and Computer Science; and PhD student Xiang Fu. Credits: Photo courtesy of the researchers

Estimating the robot’s state proved to be a challenge in some cases. Unlike in simulation, real-world sensors encounter noise that can accumulate and affect the outcome. So, for some experiments that involved high-precision foot placement, the researchers used a motion capture system to measure the robot’s true position.

Their system outperformed others that only use one controller, and the mini cheetah successfully crossed 90 percent of the terrains.

“One novelty of our system is that it does adjust the robot’s gait. If a human were trying to leap across a really wide gap, they might start by running really fast to build up speed and then they might put both feet together to have a really powerful leap across the gap. In the same way, our robot can adjust the timings and duration of its foot contacts to better traverse the terrain,” Margolis says.

Leaping out of the lab

While the researchers were able to demonstrate that their control scheme works in a laboratory, they still have a long way to go before they can deploy the system in the real world, Margolis says.

In the future, they hope to mount a more powerful computer to the robot so it can do all its computation on board. They also want to improve the robot’s state estimator to eliminate the need for the motion capture system. In addition, they’d like to improve the low-level controller so it can exploit the robot’s full range of motion, and enhance the high-level controller so it works well in different lighting conditions.

“It is remarkable to witness the flexibility of machine learning techniques capable of bypassing carefully designed intermediate processes (e.g. state estimation and trajectory planning) that centuries-old model-based techniques have relied on,” Kim says. “I am excited about the future of mobile robots with more robust vision processing trained specifically for locomotion.”

The research is supported, in part, by the MIT’s Improbable AI Lab, Biomimetic Robotics Laboratory, NAVER LABS, and the DARPA Machine Common Sense Program.

Robotics Today held three more online talks since we published the one from Amanda Prorok (Learning to Communicate in Multi-Agent Systems). In this post we bring you the last talks that Robotics Today (currently on hiatus) uploaded to their YouTube channel: Raia Hadsell from DeepMind talking about ‘Scalable Robot Learning in Rich Environments’, Koushil Sreenath from UC Berkeley talking about ‘Safety-Critical Control for Dynamic Robots’, and Antonio Bicchi from the Istituto Italiano di Tecnologia talking about ‘Planning and Learning Interaction with Variable Impedance’.

Raia Hadsell (DeepMind) – Scalable Robot Learning in Rich Environments Abstract: As modern machine learning methods push towards breakthroughs in controlling physical systems, games and simple physical simulations are often used as the main benchmark domains. As the field matures, it is important to develop more sophisticated learning systems with the aim of solving more complex real-world tasks, but problems like catastrophic forgetting and data efficiency remain critical, particularly for robotic domains. This talk will cover some of the challenges that exist for learning from interactions in more complex, constrained, and real-world settings, and some promising new approaches that have emerged. Bio: Raia Hadsell is the Director of Robotics at DeepMind. Dr. Hadsell joined DeepMind in 2014 to pursue new solutions for artificial general intelligence. Her research focuses on the challenge of continual learning for AI agents and robots, and she has proposed neural approaches such as policy distillation, progressive nets, and elastic weight consolidation to solve the problem of catastrophic forgetting. Dr. Hadsell is on the executive boards of ICLR (International Conference on Learning Representations), WiML (Women in Machine Learning), and CoRL (Conference on Robot Learning). She is a fellow of the European Lab on Learning Systems (ELLIS), a founding organizer of NAISys (Neuroscience for AI Systems), and serves as a CIFAR advisor.

Koushil Sreenath (UC Berkeley) – Safety-Critical Control for Dynamic Robots: A Model-based and Data-driven Approach Abstract: Model-based controllers can be designed to provide guarantees on stability and safety for dynamical systems. In this talk, I will show how we can address the challenges of stability through control Lyapunov functions (CLFs), input and state constraints through CLF-based quadratic programs, and safety-critical constraints through control barrier functions (CBFs). However, the performance of model-based controllers is dependent on having a precise model of the system. Model uncertainty could lead not only to poor performance but could also destabilize the system as well as violate safety constraints. I will present recent results on using model-based control along with data-driven methods to address stability and safety for systems with uncertain dynamics. In particular, I will show how reinforcement learning as well as Gaussian process regression can be used along with CLF and CBF-based control to address the adverse effects of model uncertainty. Bio: Koushil Sreenath is an Assistant Professor of Mechanical Engineering, at UC Berkeley. He received a Ph.D. degree in Electrical Engineering and Computer Science and a M.S. degree in Applied Mathematics from the University of Michigan at Ann Arbor, MI, in 2011. He was a Postdoctoral Scholar at the GRASP Lab at University of Pennsylvania from 2011 to 2013 and an Assistant Professor at Carnegie Mellon University from 2013 to 2017. His research interest lies at the intersection of highly dynamic robotics and applied nonlinear control. His work on dynamic legged locomotion was featured on The Discovery Channel, CNN, ESPN, FOX, and CBS. His work on dynamic aerial manipulation was featured on the IEEE Spectrum, New Scientist, and Huffington Post. His work on adaptive sampling with mobile sensor networks was published as a book. He received the NSF CAREER, Hellman Fellow, Best Paper Award at the Robotics: Science and Systems (RSS), and the Google Faculty Research Award in Robotics.

Antonio Bicchi (Istituto Italiano di Tecnologia) – Planning and Learning Interaction with Variable Impedance Abstract: In animals and in humans, the mechanical impedance of their limbs changes not only in dependence of the task, but also during different phases of the execution of a task. Part of this variability is intentionally controlled, by either co-activating muscles or by changing the arm posture, or both. In robots, impedance can be varied by varying controller gains, stiffness of hardware parts, and arm postures. The choice of impedance profiles to be applied can be planned off-line, or varied in real time based on feedback from the environmental interaction. Planning and control of variable impedance can use insight from human observations, from mathematical optimization methods, or from learning. In this talk I will review the basics of human and robot variable impedance, and discuss how this impact applications ranging from industrial and service robotics to prosthetics and rehabilitation. Bio: Antonio Bicchi is a scientist interested in robotics and intelligent machines. After graduating in Pisa and receiving a Ph.D. from the University of Bologna, he spent a few years at the MIT AI Lab of Cambridge before becoming Professor in Robotics at the University of Pisa. In 2009 he founded the Soft Robotics Laboratory at the Italian Institute of Technology in Genoa. Since 2013 he is Adjunct Professor at Arizona State University, Tempe, AZ. He has coordinated many international projects, including four grants from the European Research Council (ERC). He served the research community in several ways, including by launching the WorldHaptics conference and the IEEE Robotics and Automation Letters. He is currently the President of the Italian Institute of Robotics and Intelligent Machines. He has authored over 500 scientific papers cited more than 25,000 times. He supervised over 60 doctoral students and more than 20 postdocs, most of whom are now professors in universities and international research centers, or have launched their own spin-off companies. His students have received prestigious awards, including three first prizes and two nominations for the best theses in Europe on robotics and haptics. He is a Fellow of IEEE since 2005. In 2018 he received the prestigious IEEE Saridis Leadership Award.

Researchers at MIT have developed a fully-integrated robotic arm that fuses visual data from a camera and radio frequency (RF) information from an antenna to find and retrieve objects, even when they are buried under a pile and fully out of view. Credits: Courtesy of the researchers

By Adam Zewe | MIT News Office

A busy commuter is ready to walk out the door, only to realize they’ve misplaced their keys and must search through piles of stuff to find them. Rapidly sifting through clutter, they wish they could figure out which pile was hiding the keys.

Researchers at MIT have created a robotic system that can do just that. The system, RFusion, is a robotic arm with a camera and radio frequency (RF) antenna attached to its gripper. It fuses signals from the antenna with visual input from the camera to locate and retrieve an item, even if the item is buried under a pile and completely out of view.

The RFusion prototype the researchers developed relies on RFID tags, which are cheap, battery-less tags that can be stuck to an item and reflect signals sent by an antenna. Because RF signals can travel through most surfaces (like the mound of dirty laundry that may be obscuring the keys), RFusion is able to locate a tagged item within a pile.

Using machine learning, the robotic arm automatically zeroes-in on the object’s exact location, moves the items on top of it, grasps the object, and verifies that it picked up the right thing. The camera, antenna, robotic arm, and AI are fully integrated, so RFusion can work in any environment without requiring a special set up.

In this video still, the robotic arm is looking for keys hidden underneath items. Credits: Courtesy of the researchers

While finding lost keys is helpful, RFusion could have many broader applications in the future, like sorting through piles to fulfill orders in a warehouse, identifying and installing components in an auto manufacturing plant, or helping an elderly individual perform daily tasks in the home, though the current prototype isn’t quite fast enough yet for these uses.

“This idea of being able to find items in a chaotic world is an open problem that we’ve been working on for a few years. Having robots that are able to search for things under a pile is a growing need in industry today. Right now, you can think of this as a Roomba on steroids, but in the near term, this could have a lot of applications in manufacturing and warehouse environments,” said senior author Fadel Adib, associate professor in the Department of Electrical Engineering and Computer Science and director of the Signal Kinetics group in the MIT Media Lab.

Co-authors include research assistant Tara Boroushaki, the lead author; electrical engineering and computer science graduate student Isaac Perper; research associate Mergen Nachin; and Alberto Rodriguez, the Class of 1957 Associate Professor in the Department of Mechanical Engineering. The research will be presented at the Association for Computing Machinery Conference on Embedded Networked Senor Systems next month.

Sending signals

RFusion begins searching for an object using its antenna, which bounces signals off the RFID tag (like sunlight being reflected off a mirror) to identify a spherical area in which the tag is located. It combines that sphere with the camera input, which narrows down the object’s location. For instance, the item can’t be located on an area of a table that is empty.

But once the robot has a general idea of where the item is, it would need to swing its arm widely around the room taking additional measurements to come up with the exact location, which is slow and inefficient.

The researchers used reinforcement learning to train a neural network that can optimize the robot’s trajectory to the object. In reinforcement learning, the algorithm is trained through trial and error with a reward system.

“This is also how our brain learns. We get rewarded from our teachers, from our parents, from a computer game, etc. The same thing happens in reinforcement learning. We let the agent make mistakes or do something right and then we punish or reward the network. This is how the network learns something that is really hard for it to model,” Boroushaki explains.

In the case of RFusion, the optimization algorithm was rewarded when it limited the number of moves it had to make to localize the item and the distance it had to travel to pick it up.

Once the system identifies the exact right spot, the neural network uses combined RF and visual information to predict how the robotic arm should grasp the object, including the angle of the hand and the width of the gripper, and whether it must remove other items first. It also scans the item’s tag one last time to make sure it picked up the right object.

Cutting through clutter

The researchers tested RFusion in several different environments. They buried a keychain in a box full of clutter and hid a remote control under a pile of items on a couch.

But if they fed all the camera data and RF measurements to the reinforcement learning algorithm, it would have overwhelmed the system. So, drawing on the method a GPS uses to consolidate data from satellites, they summarized the RF measurements and limited the visual data to the area right in front of the robot.

Their approach worked well — RFusion had a 96 percent success rate when retrieving objects that were fully hidden under a pile.

“We let the agent make mistakes or do something right and then we punish or reward the network. This is how the network learns something that is really hard for it to model,” co-author Tara Boroushaki, pictured here, explains. Credits: Courtesy of the researchers

“Sometimes, if you only rely on RF measurements, there is going to be an outlier, and if you rely only on vision, there is sometimes going to be a mistake from the camera. But if you combine them, they are going to correct each other. That is what made the system so robust,” Boroushaki says.

In the future, the researchers hope to increase the speed of the system so it can move smoothly, rather than stopping periodically to take measurements. This would enable RFusion to be deployed in a fast-paced manufacturing or warehouse setting.

Beyond its potential industrial uses, a system like this could even be incorporated into future smart homes to assist people with any number of household tasks, Boroushaki says.

“Every year, billions of RFID tags are used to identify objects in today’s complex supply chains, including clothing and lots of other consumer goods. The RFusion approach points the way to autonomous robots that can dig through a pile of mixed items and sort them out using the data stored in the RFID tags, much more efficiently than having to inspect each item individually, especially when the items look similar to a computer vision system,” says Matthew S. Reynolds, CoMotion Presidential Innovation Fellow and associate professor of electrical and computer engineering at the University of Washington, who was not involved in the research. “The RFusion approach is a great step forward for robotics operating in complex supply chains where identifying and ‘picking’ the right item quickly and accurately is the key to getting orders fulfilled on time and keeping demanding customers happy.”

The research is sponsored by the National Science Foundation, a Sloan Research Fellowship, NTT DATA, Toppan, Toppan Forms, and the Abdul Latif Jameel Water and Food Systems Lab.

When it comes to exploring complex and unknown environments such as forests, buildings or caves, drones are hard to beat. They are fast, agile and small, and they can carry sensors and payloads virtually everywhere. However, autonomous drones can hardly find their way through an unknown environment without a map. For the moment, expert human pilots are needed to release the full potential of drones.

“To master autonomous agile flight, you need to understand the environment in a split second to fly the drone along collision-free paths,” says Davide Scaramuzza, who leads the Robotics and Perception Group at the University of Zurich and the NCCR Robotics Rescue Robotics Grand Challenge. “This is very difficult both for humans and for machines. Expert human pilots can reach this level after years of perseverance and training. But machines still struggle.”

The AI algorithm learns to fly in the real world from a simulated expert

In a new study, Scaramuzza and his team have trained an autonomous quadrotor to fly through previously unseen environments such as forests, buildings, ruins and trains, keeping speeds of up to 40 km/h and without crashing into trees, walls or other obstacles. All this was achieved relying only on the quadrotor’s on-board cameras and computation.

The drone’s neural network learned to fly by watching a sort of “simulated expert” – an algorithm that flew a computer-generated drone through a simulated environment full of complex obstacles. At all times, the algorithm had complete information on the state of the quadrotor and readings from its sensors, and could rely on enough time and computational power to always find the best trajectory.

Such a “simulated expert” could not be used outside of simulation, but its data were used to teach the neural network how to predict the best trajectory based only on the data from the sensors. This is a considerable advantage over existing systems, which first use sensor data to create a map of the environment and then plan trajectories within the map – two steps that require time and make it impossible to fly at high-speeds.

No exact replica of the real world needed

After being trained in simulation, the system was tested in the real world, where it was able to fly in a variety of environments without collisions at speeds of up to 40 km/h. “While humans require years to train, the AI, leveraging high-performance simulators, can reach comparable navigation abilities much faster, basically overnight,” says Antonio Loquercio, a PhD student and co-author of the paper. “Interestingly these simulators do not need to be an exact replica of the real world. If using the right approach, even simplistic simulators are sufficient,” adds Elia Kaufmann, another PhD student and co-author.

The applications are not limited to quadrotors. The researchers explain that the same approach could be useful for improving the performance of autonomous cars, or could even open the door to a new way of training AI systems for operations in domains where collecting data is difficult or impossible, for example on other planets.

According to the researchers, the next steps will be to make the drone improve from experience, as well as to develop faster sensors that can provide more information about the environment in a smaller amount of time – thus allowing drones to fly safely even at speeds above 40 km/h.

Literature

• Learning High-speed Flight in the Wild. Antonio Loquercio, Elia Kaufmann, René Ranftl, Matthias Müller, Vladlen Koltun and Davide Scaramuzza. Science Robotics, October 6, 2021. DOI: 10.1126/scirobotics.abg5810

An open-source version of the paper can be found here.

Media contacts

Prof. Dr. Davide Scaramuzza – Robotics and Perception Group
Department of Informatics
University of Zurich
Phone +41 44 635 24 09
E-mail: sdavide@ifi.uzh.ch

Antonio Loquercio – Robotics and Perception Group
Department of Informatics
University of Zurich
Phone +41 44 635 43 73
E-mail: loquercio@ifi.uzh.ch

Elia Kaufmann – Robotics and Perception Group
Institut für Informatik
Universität Zürich
Tel. +41 44 635 43 73
E-Mail: ekaufmann@ifi.uzh.ch

Media Relations University of Zurich
Phone +41 44 634 44 67
E-mail: mediarelations@kommunikation.uzh.ch

By Daniel Quinn

Underwater vehicles haven’t changed much since the submarines of World War II. They’re rigid, fairly boxy and use propellers to move. And whether they are large manned vessels or small robots, most underwater vehicles have one cruising speed where they are most energy efficient.

Fish take a very different approach to moving through water: Their bodies and fins are very flexible, and this flexibility allows them to interact with water more efficiently than rigid machines. Researchers have been designing and building flexible fishlike robots for years, but they still trail far behind real fish in terms of efficiency.

What’s missing?

I am an engineer and study fluid dynamics. My labmates and I wondered if something in particular about the flexibility of fish tails allows fish to be so fast and efficient in the water. So, we created a model and built a robot to study the effect of stiffness on swimming efficiency. We found fish swim so efficiently over a wide range of speeds because they can change how rigid or flexible their tails are in real time.

Why are people still using propellers?

Fluid dynamics applies to both liquids and gasses. Humans have been using rotating rigid objects to move vehicles for hundreds of years – Leonardo Da Vinci incorporated the concept into his helicopter designs, and the first propeller–driven boats were built in the 1830s. Propellers are easy to make, and they work just fine at their designed cruise speed.

It has only been in the past couple of decades that advances in soft robotics have made actively controlled flexible components a reality. Now, marine roboticists are turning to flexible fish and their amazing swimming abilities for inspiration.

When engineers like me talk about flexibility in a swimming robot, we are usually referring to how stiff the tail of the fish is. The tail is the entire rear half of a fish’s body that moves back and forth when it swims.

Consider tuna, which can swim up to 50 mph and are extremely energy efficient over a wide range of speeds.

The tricky part about copying the biomechanics of fish is that biologists don’t know how flexible they are in the real world. If you want to know how flexible a rubber band is, you simply pull on it. If you pull on a fish’s tail, the stiffness depends on how much the fish is tensing its various muscles.

The best that researchers can do to estimate flexibility is film a swimming fish and measure how its body shape changes.

Searching for answers in the math

Researchers have built dozens of robots in an attempt to mimic the flexibility and swimming patterns of tuna and other fish, but none have matched the performance of the real things.

In my lab at the University of Virginia, my colleagues and I ran into the same questions as others: How flexible should our robot be? And if there’s no one best flexibility, how should our robot change its stiffness as it swims?

We looked for the answer in an old NASA paper about vibrating airplane wings. The report explains how when a plane’s wings vibrate, the vibrations change the amount of lift the wings produce. Since fish fins and airplane wings have similar shapes, the same math works well to model how much thrust fish tails produce as they flap back and forth.

Using the old wing theory, postdoctoral researcher Qiang Zhong and I created a mathematical model of a swimming fish and added a spring and pulley to the tail to represent the effects of a tensing muscle. We discovered a surprisingly simple hypothesis hiding in the equations. To maximize efficiency, muscle tension needs to increase as the square of swimming speed. So, if swimming speed doubles, stiffness needs to increase by a factor of four. To swim three times faster while maintaining high efficiency, a fish or fish-like robot needs to pull on its tendon about nine times harder.

To confirm our theory, we simply added an artificial tendon to one of our tunalike robots and then programmed the robot to vary its tail stiffness based on speed. We then put our new robot into our test tank and ran it through various “missions” – like a 200-meter sprint where it had to dodge simulated obstacles. With the ability to vary its tail’s flexibility, the robot used about half as much energy on average across a wide range of speeds compared to robots with a single stiffness.

Why it matters

While it is great to build one excellent robot, the thing my colleagues and I are most excited about is that our model is adaptable. We can tweak it based on body size, swimming style or even fluid type. It can be applied to animals and machines whether they are big or small, swimmers or flyers.

For example, our model suggests that dolphins have a lot to gain from the ability to vary their tails’ stiffness, whereas goldfish don’t get much benefit due to their body size, body shape and swimming style.

The model also has applications for robotic design too. Higher energy efficiency when swimming or flying – which also means quieter robots – would enable radically new missions for vehicles and robots that currently have only one efficient cruising speed. In the short term, this could help biologists study river beds and coral reefs more easily, enable researchers to track wind and ocean currents at unprecedented scales or allow search and rescue teams to operate farther and longer.

In the long term, I hope our research could inspire new designs for submarines and airplanes. Humans have only been working on swimming and flying machines for a couple centuries, while animals have been perfecting their skills for millions of years. There’s no doubt there is still a lot to learn from them.

Daniel Quinn receives funding from The National Science Foundation and The Office of Naval Research.

This article appeared in The Conversation.

Robotic grasping and manipulation has historically been dominated by rigid grippers, force/form closure constraints, and extensive grasp trajectory planning. The advent of soft robotics offers new avenues to diverge from this paradigm by using strategic compliance to passively conform to grasped objects in the absence of active control, and with minimal chance of damage to the object or surrounding environment. However, while the reduced emphasis on sensing, planning, and control complexity simplifies grasping and manipulation tasks, precision and dexterity are often lost.

This talk will discuss efforts to increase the robustness of soft grasping and the dexterity of soft robotic manipulators, with particular emphasis on grasping tasks that are challenging for more traditional robot hands. This includes compliant objects, thin flexible sheets, and delicate organisms. Examples will be drawn from manipulation of everyday objects and field studies of deep sea sampling using soft end effectors

Bio: Robert Wood is the Charles River Professor of Engineering and Applied Sciences in the Harvard John A. Paulson School of Engineering and Applied Sciences and a National Geographic Explorer. Prof. Wood completed his M.S. and Ph.D. degrees in the Dept. of Electrical Engineering and Computer Sciences at the University of California, Berkeley. His current research interests include new micro- and meso-scale manufacturing techniques, bioinspired microrobots, biomedical microrobots, control of sensor-limited and computation-limited systems, active soft materials, wearable robots, and soft grasping and manipulation. He is the winner of multiple awards for his
work including the DARPA Young Faculty Award, NSF Career Award, ONR Young Investigator Award, Air Force Young Investigator Award, Technology Review’s TR35, and multiple best paper awards. In 2010 Wood received the Presidential Early Career Award for Scientists and Engineers from President Obama for his work in microrobotics. In 2012 he was selected for the Alan T. Waterman award, the National Science Foundation’s most prestigious early career award. In 2014 he was named one of National Geographic’s “Emerging Explorers”, and in 2018 he was an inaugural recipient of the Max Planck-Humboldt Medal. Wood’s group is also dedicated to STEM education by using novel robots to motivate young students to pursue careers in science and engineering.

In this episode, Audrow Nash interviews Dave Coleman, Chief Executive Officer at PickNik Robotics. Dave speaks at a high level about what MoveIt is and what problems it helps roboticists solve, they talk about supervised autonomy, including a collaboration with NASA and MoveIt Studio, and Dave talks about MoveIt 3.0.

Episode links

• Download the episode
• Dave Coleman’s LinkedIn profile
• PickNik’s website
• MoveIt’s website
• MoveIt Studio’s website

Subscribe

• YouTube
• Apple Podcasts
• Spotify
• Google Podcasts
• Overcast
• Pocket Casts

In this final video of our focus series on IEEE/RSJ IROS 2020 (International Conference on Intelligent Robots and Systems) original series Real Roboticist, you’ll meet Dennis Hong speaking about the robots he and his team have created (locomotion and new ways of moving; an autonomous car for the visually impaired; disaster relief robots), Star Wars and cooking. All in all, ingredients from different worlds that Dennis is using to benefit society.

Dennis Hong is a Professor and the Founding Director of RoMeLa (Robotics & Mechanisms Laboratory) of the Mechanical & Aerospace Engineering Department at UCLA. If you’d like to find out more about how Star Wars influenced his professional career in robotics, how his experience taking a cooking assistant robot to MasterChef USA inspired a multi-million research project, and all the robots he is creating, check out his video below!

How do humans become so skillful? Well, initially we are not, but from infancy, we discover and practice increasingly complex skills through self-supervised play. But this play is not random – the child development literature suggests that infants use their prior experience to conduct directed exploration of affordances like movability, suckability, graspability, and digestibility through interaction and sensory feedback. This type of affordance directed exploration allows infants to learn both what can be done in a given environment and how to do it. Can we instantiate an analogous strategy in a robotic learning system?

On the left we see videos from a prior dataset collected with a robot accomplishing various tasks such as drawer opening and closing, as well as grasping and relocating objects. On the right we have a lid that the robot has never seen before. The robot has been granted a short period of time to practice with the new object, after which it will be given a goal image and tasked with making the scene match this image. How can the robot rapidly learn to manipulate the environment and grasp this lid without any external supervision?

To do so, we face several challenges. When a robot is dropped in a new environment, it must be able to use its prior knowledge to think of potentially useful behaviors that the environment affords. Then, the robot has to be able to actually practice these behaviors informatively. To now improve itself in the new environment, the robot must then be able to evaluate its own success somehow without an externally provided reward.

If we can overcome these challenges reliably, we open the door for a powerful cycle in which our agents use prior experience to collect high quality interaction data, which then grows their prior experience even further, continuously enhancing their potential utility!

VAL: Visuomotor Affordance Learning

Our method, Visuomotor Affordance Learning, or VAL, addresses these challenges. In VAL, we begin by assuming access to a prior dataset of robots demonstrating affordances in various environments. From here, VAL enters an offline phase which uses this information to learn 1) a generative model for imagining useful affordances in new environments, 2) a strong offline policy for effective exploration of these affordances, and 3) a self-evaluation metric for improving this policy. Finally, VAL is ready for it’s online phase. The agent is dropped in a new environment and can now use these learned capabilities to conduct self-supervised finetuning. The whole framework is illustrated in the figure below. Next, we will go deeper into the technical details of the offline and online phase.

VAL: Offline Phase

Given a prior dataset demonstrating the affordances of various environments, VAL digests this information in three offline steps: representation learning to handle high dimensional real world data, affordance learning to enable self-supervised practice in unknown environments, and behavior learning to attain a high performance initial policy which accelerates online learning efficiency.

1. First, VAL learns a low representation of this data using a Vector Quantized Variational Auto-encoder or VQVAE. This process reduces our 48x48x3 images into a 144 dimensional latent space.

Distances in this latent space are meaningful, paving the way for our crucial mechanism of self-evaluating success. Given the current image s and goal image g, we encode both into the latent space, and threshold their distance to obtain a reward.

Later on, we will also use this representation as the latent space for our policy and Q function.

2. Next, VAL learn an affordance model by training a PixelCNN in the latent space to the learn the distribution of reachable states conditioned on an image from the environment. This is done by maximizing the likelihood of the data,
$p(s_n | s_0)$. We use this affordance model for directed exploration and for relabeling goals.

The affordance model is illustrated in the figure right. On the bottom left of the figure, we see that the conditioning image contains a pot, and the decoded latent goals on the upper right show the lid in different locations. These coherent goals will allow the robot to perform coherent exploration.

3. Last in the offline phase, VAL must learn behaviors from the offline data, which it can then improve upon later with extra online, interactive data collection.

To accomplish this, we train a goal conditioned policy on the prior dataset using Advantage Weighted Actor Critic, an algorithm specifically designed for training offline and being amenable to online fine-tuning.

VAL: Online Phase

Now, when VAL is placed in an unseen environment, it uses its prior knowledge to imagine visual representations of useful affordances, collects helpful interaction data by trying to achieve these affordances, updates its parameters using its self-evaluation metric, and repeats the process all over again.

In this real example, on the left we see the initial state of the environment, which affords opening the drawer as well as other tasks.

In step 1, the affordance model samples a latent goal. By decoding the goal (using the VQVAE decoder, which is never actually used during RL because we operate entirely in the latent space), we can see the affordance is to open a drawer.

In step 2, we roll out the trained policy with the sampled goal. We see it successfully opens the drawer, in fact going too far and pulling the drawer all the way out. But this provides extremely useful interaction for the RL algorithm to further fine-tune on and perfect its policy.

After online finetuning is complete, we can now evaluate the robot on its ability to achieve the corresponding unseen goal images for each environment.

Real World Evaluation

We evaluate our method in five real-world test environments, and assess VAL on its ability to achieve a specific task the environment affords before and after five minutes of unsupervised fine-tuning.

Each test environment consists of at least one unseen interaction object, and two randomly sampled distractor objects. For instance, while there is opening and closing drawers in the training data, the new drawers have unseen handles.

In every case, we begin with the offline trained policy, which solves the task inconsistently. Then, we collect more experience using our affordance model to sample goals. Finally, we evaluate the fine-tuned policy, which consistently solves the task.

We find that in each of these environments, VAL consistently demonstrates effective zero-shot generalization after offline training, followed by rapid improvement with its affordance-directed fine-tuning scheme. Meanwhile, prior self-supervised methods barely improve upon poor zero-shot performance in these new environments. These exciting results illustrate the potential that approaches like VAL possess for enabling robots to successfully operate far beyond the limited factory setting in which they are used to now.

Our dataset of 2,500 high quality robot interaction trajectories, covering 20 drawer handles, 20 pot handles, 60 toys, and 60 distractor objects, is now publicly available on our website.

Simulated Evaluation and Code

For further analysis, we run VAL in a procedurally generated, multi-task environment with visual and dynamic variation. Which objects are in the scene, their colors, and their positions are randomized per environment. The agent can use handles to open drawers, grasp objects to relocate them, press buttons to unlock compartments, and so on.

The robot is given a prior dataset spanning various environments, and is evaluated on its ability to fine-tune on the following test environments.

Again, given a single off-policy dataset, our method quickly learns advanced manipulation skills including grasping, drawer opening, re-positioning, and tool usage for a diverse set of novel objects.

The environments and algorithm code are available; please see our code repository.

Future Work

Like deep learning in domains such as computer vision and natural language processing which have been driven by large datasets and generalization, robotics will likely require learning from a similar scale of data. Because of this, improvements in offline reinforcement learning will be critical for enabling robots to take advantage of large prior datasets. Furthermore, these offline policies will need either rapid non-autonomous finetuning or entirely autonomous finetuning for real world deployment to be feasible. Lastly, once robots are operating on their own, we will have access to a continuous stream of new data, stressing both the importance and value of lifelong learning algorithms.

This post is based on the paper “What Can I Do Here? Learning New Skills by Imagining Visual Affordances”, which was presented at the International Conference on Robotics and Automation (ICRA), 2021. You can see results on our website, and we provide code to reproduce our experiments.

In this episode, Audrow Nash interviews Ryan Gariepy, Chief Technology Officer of both Clearpath Robotics and OTTO Motors. Ryan speaks about the origin of Clearpath, how Clearpath focuses on making reliable robots, and the future of robotics.

Episode links

• Download the episode
• Ryan Gariepy’s LinkedIn
• Clearpath Robotics
• OTTO Motors

Subscribe

• YouTube
• Apple Podcasts
• Spotify
• Google Podcasts
• Overcast
• Pocket Casts