Most of the world is covered in oceans, which are unfortunately highly polluted. One of the strategies to combat the mounds of waste found in these very sensitive ecosystems – especially around coral reefs – is to employ robots to master the cleanup. However, existing underwater robots are mostly bulky with rigid bodies, unable to explore and sample in complex and unstructured environments, and are noisy due to electrical motors or hydraulic pumps. For a more suitable design, scientists at the Max Planck Institute for Intelligent Systems (MPI-IS) in Stuttgart looked to nature for inspiration. They configured a jellyfish-inspired, versatile, energy-efficient and nearly noise-free robot the size of a hand. Jellyfish-Bot is a collaboration between the Physical Intelligence and Robotic Materials departments at MPI-IS. “A Versatile Jellyfish-like Robotic Platform for Effective Underwater Propulsion and Manipulation” was published in Science Advances.

To build the robot, the team used electrohydraulic actuators through which electricity flows. The actuators serve as artificial muscles which power the robot. Surrounding these muscles are air cushions as well as soft and rigid components which stabilize the robot and make it waterproof. This way, the high voltage running through the actuators cannot contact the surrounding water. A power supply periodically provides electricity through thin wires, causing the muscles to contract and expand. This allows the robot to swim gracefully and to create swirls underneath its body.

“When a jellyfish swims upwards, it can trap objects along its path as it creates currents around its body. In this way, it can also collect nutrients. Our robot, too, circulates the water around it. This function is useful in collecting objects such as waste particles. It can then transport the litter to the surface, where it can later be recycled. It is also able to collect fragile biological samples such as fish eggs. Meanwhile, there is no negative impact on the surrounding environment. The interaction with aquatic species is gentle and nearly noise-free”, Tianlu Wang explains. He is a postdoc in the Physical Intelligence Department at MPI-IS and first author of the publication.

His co-author Hyeong-Joon Joo from the Robotic Materials Department continues: “70% of marine litter is estimated to sink to the seabed. Plastics make up more than 60% of this litter, taking hundreds of years to degrade. Therefore, we saw an urgent need to develop a robot to manipulate objects such as litter and transport it upwards. We hope that underwater robots could one day assist in cleaning up our oceans.”

Jellyfish-Bots are capable of moving and trapping objects without physical contact, operating either alone or with several in combination. Each robot works faster than other comparable inventions, reaching a speed of up to 6.1 cm/s. Moreover, Jellyfish-Bot only requires a low input power of around 100 mW. And it is safe for humans and fish should the polymer material insulating the robot one day be torn apart. Meanwhile, the noise from the robot cannot be distinguished from background levels. In this way Jellyfish-Bot interacts gently with its environment without disturbing it – much like its natural counterpart.

The robot consists of several layers: some stiffen the robot, others serve to keep it afloat or insulate it. A further polymer layer functions as a floating skin. Electrically powered artificial muscles known as HASELs are embedded into the middle of the different layers. HASELs are liquid dielectric-filled plastic pouches that are partially covered by electrodes. Applying a high voltage across an electrode charges it positively, while surrounding water is charged negatively. This generates a force between positively-charged electrode and negatively-charged water that pushes the oil inside the pouches back and forth, causing the pouches to contract and relax – resembling a real muscle. HASELs can sustain the high electrical stresses generated by the charged electrodes and are protected against water by an insulating layer. This is important, as HASEL muscles were never before used to build an underwater robot.

The first step was to develop Jellyfish-Bot with one electrode with six fingers or arms. In the second step, the team divided the single electrode into separated groups to independently actuate them.

“We achieved grasping objects by making four of the arms function as a propeller, and the other two as a gripper. Or we actuated only a subset of the arms, in order to steer the robot in different directions. We also looked into how we can operate a collective of several robots. For instance, we took two robots and let them pick up a mask, which is very difficult for a single robot alone. Two robots can also cooperate in carrying heavy loads. However, at this point, our Jellyfish-Bot needs a wire. This is a drawback if we really want to use it one day in the ocean”, Hyeong-Joon Joo says.

Perhaps wires powering robots will soon be a thing of the past. “We aim to develop wireless robots. Luckily, we have achieved the first step towards this goal. We have incorporated all the functional modules like the battery and wireless communication parts so as to enable future wireless manipulation”, Tianlu Wang continues. The team attached a buoyancy unit at the top of the robot and a battery and microcontroller to the bottom. They then took their invention for a swim in the pond of the Max Planck Stuttgart campus, and could successfully steer it along. So far, however, they could not direct the wireless robot to change course and swim the other way.

Knowing the team, it won’t take long to achieve this goal.

• PAPER – A versatile jellyfish-like robotic platform for effective underwater propulsion and manipulation. Wang, T., Joo, H. J., Song, S., Hu, W., Keplinger, C., and Sitti, M. Science Advances, 9(15), 2023, eadg0292.

RoboSalps in action. Credits: Valentina Lo Gatto

These robotic units called RoboSalps, after their animal namesakes, have been engineered to operate in unknown and extreme environments such as extra-terrestrial oceans.

Although salps resemble jellyfish with their semi-transparent barrel-shaped bodies, they belong to the family of Tunicata and have a complex life cycle, changing between solitary and aggregate generations where they connect to form colonies.

RoboSalps have similarly light, tubular bodies and can link to each other to form ‘colonies’ which gives them new capabilities that can only be achieved because they work together.

Researcher Valentina Lo Gatto of Bristol’s Department of Aerospace Engineering is leading the study. She is also a student at the EPSRC Centre of Doctoral Training in Future Autonomous and Robotic Systems (FARSCOPE CDT).

She said: “RoboSalp is the first modular salp-inspired robot. Each module is made of a very light-weight soft tubular structure and a drone propeller which enables them to swim. These simple modules can be combined into ‘colonies’ that are much more robust and have the potential to carry out complex tasks. Because of their low weight and their robustness, they are ideal for extra-terrestrial underwater exploration missions, for example, in the subsurface ocean on the Jupiter moon Europa.”

RoboSalps are unique as each individual module can swim on its own. This is possible because of a small motor with rotor blades – typically used for drones – inserted into the soft tubular structure.

When swimming on their own, RoboSalps modules are difficult to control, but after joining them together to form colonies, they become more stable and show sophisticated movements.

In addition, by having multiple units joined together, scientists automatically obtain a redundant system, which makes it more robust against failure. If one module breaks, the whole colony can still move.

A colony of soft robots is a relatively novel concept with a wide range of interesting applications. RoboSalps are soft, potentially quite energy efficient, and robust due to inherent redundancy. This makes them ideal for autonomous missions where a direct and immediate human control might not be feasible.

Dr Helmut Hauser of Bristol’s Department of Engineering Maths, explained: “These include the exploration of remote submarine environments, sewage tunnels, and industrial cooling systems. Due to the low weight and softness of the RoboSalp modules, they are also ideal for extra-terrestrial missions. They can easily be stored in a reduced volume, ideal for reducing global space mission payloads.”

A compliant body also provides safer interaction with potentially delicate ecosystems, both on earth and extra-terrestrial, reducing the risk of environmental damage. The possibility to detach units or segments, and rearrange them, gives the system adaptability: once the target environment is reached, the colony could be deployed to start its exploration.

At a certain point, it could split into multiple segments, each exploring in a different direction, and afterwards reassemble in a new configuration to achieve a different objective such as manipulation or sample collection.

Prof Jonathan Rossiter added: “We are also developing control approaches that are able to exploit the compliance of the modules with the goal of achieving energy efficient movements close to those observed in biological salps.”

Many people associate prosthetic limbs with nude-colored imitations of human limbs. Something built to blend into a society where people have all of their limbs while serving functional use cases. On the other end of the spectrum are the highly optimized prosthetics used by Athletes, built for speed, low weight, and appearing nothing like a human limb.

As a child under 12 years old, neither of these categories of prosthetics particularly speaks to you. Open Bionics, founded by Joel Gibbard and Samantha Payne, was started to create a third category of prosthetics. One that targets the fun, imaginative side of children, while still providing the daily functional requirements.

Through partnerships with Disney and Lucasfilms, Open Bionics has built an array of imagination-capturing prosthetic limbs that are straight-up cool.

Joel Gibbard dives into why they founded Open Bionics, and why you should invest in their company as they are getting ready to let the general public invest in them for the first time.

Joel Gibbard

Joel Gibbard lives in Bristol, UK and graduated with a first-class honors degree in Robotics from the University of Plymouth, UK.

He co-founded Open Bionics alongside Samantha Payne with the goal of bringing advanced, accessible bionic arms to the market. Open Bionics offers the Hero Arm, which is available in the UK, USA, France, Australia, and New Zealand. Open Bionics is revolutionizing the prosthetics industry through its line of inspiration-capturing products.

Links

• Open Bionics
• Pre-register to invest
• Download mp3
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

Growing up in Rhode Island (the Ocean State), I lived very close to the water. Over the years, I have seen the effects of sea level rise and rapid erosion. Entire houses and beaches have slowly been consumed by the tide. I have witnessed first hand how climate change is rapidly changing the ocean ecosystem. Sometimes I feel overwhelmed by the inexorability of climate change. What can we do in the face of such a global, almost incomprehensible dilemma? The only way I can overcome this perception is by committing to doing something with my life to help, even if it’s in a small way. I think with such a big issue, the only way forward is by starting small, identifying one niche I can work in, and seeing how I can shape my research around solving that challenge.

One major challenge is rapid global ocean temperature rise. When scientists look to make climate associations using temperature data, they generally use fixed temperature loggers attached to buoys or on the ocean floor. Unfortunately, this approach discounts the area between the ocean’s surface and floor. Variable ocean conditions create microclimates, pockets of the ocean that are unaffected by general climate trends. Scientists have shown that most organisms experience climate change via these microclimates. Fish are greatly affected by this rapid increase in temperature as they can only lay eggs in a minimal range of temperatures. Microclimates are changing temperature with celerity. Hence, many species cannot adapt quickly enough to survive. At this rate, 60% of fish species could go extinct by 2100.

Of course, fish are not the only organisms affected by the rapid increase in temperature. Coral in the Great Barrier Reef can only survive in a minimal temperature threshold, and as temperature increases, reefs are experiencing mass coral bleaching. AIMS, the Australian Institute for Marine Science, the government agency that monitors the Great Barrier Reef, utilizes divers pulled behind boats to record reef observations and collect data. Unfortunately, this has led to some casualties due to shark attacks. They have begun deploying large, almost seven feet in length, ocean gliders that can mitigate this risk. These robots come with a hefty price tag of $125,000 to $500,000. They are also too large to navigate portions of the reef.

Our solution in the Soft Robotics Lab at Worcester Polytechnic Institute is building a free-swimming (tetherless), biologically inspired robotic fish, funded in part by the National Science Foundation Future of Robots in the Workplace Research and Development Program. Our goal is for the robot to navigate the complex environment of the Great Barrier Reef and record dense three-dimensional temperature data throughout the water column. Moreover, we will use non-hazardous and affordable material for the fish’s body. Since our motivation is to create a tool to use in climate research, a robot that is cheap and easy to manufacture will increase its effectiveness. Our approach is in stark contrast to traditional autonomous underwater vehicles that utilize propellers that are noisy and incongruous to underwater life. We chose to mimic the motion of real fish to reduce the environmental impact of our robot and enable close observation of other real fish.

We are, of course, not the first people to build a robot fish. In 1994, MIT produced the RoboTuna, a fully rigid fish robot, and since then, there have been many different iterations of fish robots. Some have been made of fully rigid materials like the RoboTuna and used motors that run the caudal tail (rear fin) actuation that powers the fish. However, this does not replicate the fluid motion achieved by real fish as they swim. A possible solution would be to use soft materials. Designs using soft materials, up to this point, utilize a silicone, pneumatically or hydraulically actuated tail. Unfortunately, these robots cannot operate in rough environments since any cuts or abrasions to the silicone could cause a leak in the system and lead to a total failure in the actuation of the tail. Other robots have combined the more durable rigid materials, actuated with cables, and then attached a soft silicone end that bends with the force of the water. All these previous robots are difficult to manufacture and require institutional knowledge to recreate.

MIT Robotuna and MIT SOFI robots

We have fabricated a 3D printed, cable-actuated wave spring tail made from soft materials that can drive a small robot fish. The wave spring gives the robot its biologically inspired shape, but it can bend fluidly like the silicone-based robots and real fish. The wave springis entirely 3D printed from a flexible material that is affordable and easy to use. This material and method creates a very soft yet durable robot, withstands harsh treatment, and runs for hundreds of thousands of cycles without any degradation to any of the robot’s systems. The robot sets itself apart by being very easy to assemble, with only a handful of parts, most of which can be 3D printed.

The wave spring itself has a biologically inspired design. Reef fish are morphologically diverse but share a similar body shape which we emulate with a tapered oval design. The wave spring itself is composed of a mesh of diamond-shaped cells that can compress and bend. To restrict our robot to only lateral bending, we added supports down the dorsal and ventral edges of the wave spring.

Using this design, we have successfully created a robot fish. The robot is able to swim freely in a fish tank, swimming pool, and in a lake. While testing the fish in these environments, we found that the speed and performance of our robot was comparable to other fish robots operating under similar parameters. In order to waterproof the robot (to protect the electronics required for tetherless swimming), we had to add a latex skin. This does increase the manufacturing complexity of the design, so we will look to improve not only the robot’s performance, but also its design to ensure a simplistic yet high functioning robot.

Most importantly, we will add the sensors required to collect data like temperature, which is imperative to a better understanding of the oceans’ rapidly changing microclimates. It’s crucial that we remain focused on this goal, as it drives not only the robot’s design, but our motivation for why we do this work. Climate change is the foremost crisis facing our world. I encourage everyone to connect their interests and work, no matter the field, in some way to this issue as we are the only ones who can do something about it.

Scientists have developed a theory that can explain how flying insects determine the gravity direction without using accelerometers. It also forms a substantial step in the creation of tiny, autonomous drones.

Scientists have discovered a novel manner for flying drones and insects to estimate the gravity direction. Whereas drones typically use accelerometers to this end, the way in which flying insects do this has until now been shrouded in mystery, since they lack a specific sense for acceleration. In an article published in Nature, scientists from TU Delft and Aix Marseille Université / CNRS in France have shown that drones can estimate the gravity direction by combining visual motion sensing with a model of how they move. The study is a great example of the synergy between technology and biology.

On the one hand, the new approach is an important step for the creation of autonomous tiny, insect-sized drones, since it requires fewer sensors. On the other hand, it forms a hypothesis for how insects control their attitude, as the theory forms a parsimonious explanation of multiple phenomena observed in biology.

The importance of finding the gravity direction

Successful flight requires knowing the direction of gravity. As ground-bound animals, we humans typically have no trouble determining which way is down. However, this becomes more difficult when flying. Indeed, the passengers in an airplane are normally not aware of the plane being slightly tilted sideways in the air to make a wide circle. When humans started to take the skies, pilots relied purely on visually detecting the horizon line for determining the plane’s “attitude”, that is, its body orientation with respect to gravity. However, when flying through clouds the horizon line is no longer visible, which can lead to an increasingly wrong impression of what is up and down – with potentially disastrous consequences.

Also drones and flying insects need to control their attitude. Drones typically use accelerometers for determining the gravity direction. However, in flying insects no sensing organ for measuring accelerations has been found. Hence, for insects it is currently still a mystery how they estimate attitude, and some even question whether they estimate attitude at all.

Optic flow suffices for finding attitude

Although it is unknown how flying insects estimate and control their attitude, it is very well known that they visually observe motion by means of “optic flow”. Optic flow captures the relative motion between an observer and its environment. For example, when sitting in a train, trees close by seem to move very fast (have a large optic flow), while mountains in the distance seem to move very slowly (have a small optic flow).

“Optic flow itself carries no information on attitude. However, we found out that combining optic flow with a motion model allows to retrieve the gravity direction.”, says Guido de Croon, full professor of bio-inspired micro air vehicles at TU Delft, “Having a motion model means that a robot or animal can predict how it will move when taking actions. For example, drones can predict what will happen when they spin their two right propellers faster than their left propellers. Since a drone’s attitude determines in which direction it accelerates, and this direction can be picked up by changes in optic flow, the combination allows a drone to determine its attitude.”

The theoretical analysis in the article shows that finding the gravity direction with optic flow works almost under any condition, except for specific cases such as when the observer is completely still. “Whereas engineers would find such an observability problem unacceptable, we hypothesise that nature has simply accepted it”, says Guido de Croon. “In the article we provide a theoretical proof that despite this problem, an attitude controller will still work around hover at the cost of slight oscillations – reminiscent of the more erratic flight behaviour of flying insects.”

Implications for robotics

The researchers confirmed the theory’s validity with robotic implementations, demonstrating its promise for the field of robotics. De Croon: “Tiny flapping wing drones can be useful for tasks like search-and-rescue or pollination. Designing such drones means dealing with a major challenge that nature also had to face; how to achieve a fully autonomous system subject to extreme payload restrictions. This makes even tiny accelerometers a considerable burden. Our proposed theory will contribute to the design of tiny drones by allowing for a smaller sensor suite.”

Biological insights

The proposed theory has the potential to give insight into various biological phenomena. “It was known that optic flow played a role in attitude control, but until now the precise mechanism for this was unclear.”, explains Franck Ruffier, bio-roboticist and director of research at Aixe Marseille Université / CNRS, “The proposed theory can explain how flying insects succeed in estimating and controlling their attitude even in difficult, cluttered environments where the horizon line is not visible. It also provides insight into other phenomena, for example, why locusts fly less well when their ocelli (eyes on the top of their heads) are occluded.”

”We expect that novel biological experiments, specifically designed for testing our theory will be necessary for verifying the use of the proposed mechanism in insects”, adds Franck Ruffier.

Click here for the original publication in Nature. The scientific article shows how the synergy between robotics and biology can lead to technological advances and novel avenues for biological research.

The post Big step towards tiny autonomous drones appeared first on RoboHouse.

• PAPER – Accommodating unobservability to control flight attitude with optic flow. G.C.H.E. de Croon, J.J.G. Dupeyroux, C. De Wagter, A. Chatterjee, D.A. Olejnik, and F. Ruffier. Nature 610, 485–490 (2022). https://doi.org/10.1038/s41586-022-05182-2

MIT chemical engineers have shown that specialized particles can oscillate together, demonstrating a phenomenon known as emergent behavior. Image: Courtesy of the researchers

By Anne Trafton | MIT News Office

Taking advantage of a phenomenon known as emergent behavior in the microscale, MIT engineers have designed simple microparticles that can collectively generate complex behavior, much the same way that a colony of ants can dig tunnels or collect food.

Working together, the microparticles can generate a beating clock that oscillates at a very low frequency. These oscillations can then be harnessed to power tiny robotic devices, the researchers showed.

“In addition to being interesting from a physics point of view, this behavior can also be translated into an on-board oscillatory electrical signal, which can be very powerful in microrobotic autonomy. There are a lot of electrical components that require such an oscillatory input,” says Jingfan Yang, a recent MIT PhD recipient and one of the lead authors of the new study.

The particles used to create the new oscillator perform a simple chemical reaction that allows the particles to interact with each other through the formation and bursting of tiny gas bubbles. Under the right conditions, these interactions create an oscillator that behaves similar to a ticking clock, beating at intervals of a few seconds.

“We’re trying to look for very simple rules or features that you can encode into relatively simple microrobotic machines, to get them to collectively do very sophisticated tasks,” says Michael Strano, the Carbon P. Dubbs Professor of Chemical Engineering at MIT.

Strano is the senior author of the new paper, which appears in Nature Communications. Along with Yang, Thomas Berrueta, a Northwestern University graduate student advised by Professor Todd Murphey, is a lead author of the study.

Collective behavior

Demonstrations of emergent behavior can be seen throughout the natural world, where colonies of insects such as ants and bees accomplish feats that a single member of the group would never be able to achieve.

“Ants have minuscule brains and they do very simple cognitive tasks, but collectively they can do amazing things. They can forage for food and build these elaborate tunnel structures,” Strano says. “Physicists and engineers like myself want to understand these rules because it means we can make tiny things that collectively do complex tasks.”

In this study, the researchers wanted to design particles that could generate rhythmic movements, or oscillations, with a very low frequency. Until now, building low-frequency micro-oscillators has required sophisticated electronics that are expensive and difficult to design, or specialized materials with complex chemistries.

The simple particles that the researchers designed for this study are discs as small as 100 microns in diameter. The discs, made from a polymer called SU-8, have a platinum patch that can catalyze the breakdown of hydrogen peroxide into water and oxygen.

When the particles are placed at the surface of a droplet of hydrogen peroxide on a flat surface, they tend to travel to the top of the droplet. At this liquid-air interface, they interact with any other particles found there. Each particle produces its own tiny bubble of oxygen, and when two particles come close enough that their bubbles interact, the bubbles pop, propelling the particles away from each other. Then, they begin forming new bubbles, and the cycle repeats over and over.

“One particle by itself stays still and doesn’t do anything interesting, but through teamwork, they can do something pretty amazing and useful, which is actually a difficult thing to achieve at the microscale,” Yang says.

MIT chemical engineers showed that specialized particles can oscillate together, demonstrating a phenomenon known as emergent behavior. At left, two particles oscillate together, and at right, eight particles. Video courtesy of the researchers.

The researchers found that two particles could make a very reliable oscillator, but as more particles were added, the rhythm would get thrown off. However, if they added one particle that was slightly different from the others, that particle could act as a “leader” that reorganized the other particles back into a rhythmic oscillator.

This leader particle is the same size as the other particles but has a slightly larger platinum patch, which enables it to create a larger oxygen bubble. This allows this particle to move to the center of the group, where it coordinates the oscillations of all of the other particles. Using this approach, the researchers found they could create oscillators containing up to at least 11 particles.

Depending on the number of particles, this oscillator beats at a frequency of about 0.1 to 0.3 hertz, which is on the order of the low-frequency oscillators that govern biological functions such as walking and the beating of the heart.

Oscillating current

The researchers also showed that they could use the rhythmic beating of these particles to generate an oscillating electric current. To do that, they swapped out the platinum catalyst for a fuel cell made of platinum and ruthenium or gold. The mechanical oscillation of the particles rhythmically alters the resistance from one end of the fuel cell to the other, which converts the voltage generated by the fuel cell to an oscillating current.

“Like a dripping faucet, catalytic microdiscs floating at a liquid interface use a chemical reaction to drive the periodic growth and release of gas bubbles. The study shows how these oscillatory dynamics can be harnessed for mechanical actuation and electrochemical signaling relevant to microrobotics,” says Kyle Bishop, a professor of chemical engineering at Columbia University, who was not involved in the study.

Generating an oscillating current instead of a constant one could be useful for applications such as powering tiny robots that can walk. The MIT researchers used this approach to show that they could power a microactuator, which was previously used as legs on a tiny walking robot developed by researchers at Cornell University. The original version was powered by a laser that had to be alternately pointed at each set of legs, to manually oscillate the current. The MIT team showed that the on-board oscillating current generated by their particles could drive the cyclic actuation of the microrobotic leg, using a wire to transfer the current from the particles to the actuator.

“It shows that this mechanical oscillation can become an electrical oscillation, and then that electrical oscillation can actually power activities that a robot would do,” Strano says.

One possible application for this kind of system would be to control swarms of tiny autonomous robots that could be used as sensors to monitor water pollution.

The research was funded in part by the U.S. Army Research Office, the U.S. Department of Energy, and the National Science Foundation.

Ant leading other ant to new nest, known as tandem running. Image credit: Norasmah Basari and Nigel R Franks

The team built the robot to mimic the behaviour of rock ants that use one-to-one tuition, in which an ant that has discovered a much better new nest can teach the route there to another individual.

The findings, published in the Journal of Experimental Biology, confirm that most of the important elements of teaching in these ants are now understood because the teaching ant can be replaced by a machine.

Key to this process of teaching is tandem running where one ant literally leads another ant quite slowly along a route to the new nest. The pupil ant learns the route sufficiently well that it can find its own way back home and then lead a tandem-run with another ant to the new nest, and so on.

Prof Nigel Franks of Bristol’s School of Biological Sciences said: “Teaching is so important in our own lives that we spend a great deal of time either instructing others or being taught ourselves. This should cause us to wonder whether teaching actually occurs among non-human animals. And, in fact, the first case in which teaching was demonstrated rigorously in any other animal was in an ant.” The team wanted to determine what was necessary and sufficient in such teaching. If they could build a robot that successfully replaced the teacher, this should show that they largely understood all the essential elements in this process.

Prof Nigel Franks showing Sir David Attenborough the gantry during the opening of the new Life Sciences Building in 2014. Image credit: University of Bristol

The researchers built a large arena so there was an appreciable distance between the ants’ old nest, which was deliberately made to be of low quality, and a new much better one that ants could be led to by a robot. A gantry was placed atop the arena to move back and forth with a small sliding robot attached to it, so that the scientists could direct the robot to move along either straight or wavy routes. Attractive scent glands, from a worker ant, were attached to the robot to give it the pheromones of an ant teacher.

Prof Franks explained: “We waited for an ant to leave the old nest and put the robot pin, adorned with attractive pheromones, directly ahead of it. The pinhead was programmed to move towards the new nest either on a straight path or on a beautifully sinuous one. We had to allow for the robot to be interrupted in its journey, by us, so that we could wait for the following ant to catch up after it had looked around to learn landmarks.”

Diagram of ant pheromone glands. Image credit: Norasmah Basari

“When the follower ant had been led by the robot to the new nest, we allowed it to examine the new nest and then, in its own time, begin its homeward journey. We then used the gantry automatically to track the path of the returning ant.”

The team found that the robot had indeed taught the route successfully to the apprentice ant. The ants knew their way back to the old nest whether they had taken a winding path or a straight one.

Prof Franks explained: “A straight path might be quicker but a winding path would provide more time in which the following ant could better learn landmarks so that it could find its way home as efficiently as if it had been on a straight path.

“Crucially, we could compare the performance of the ants that the robot had taught with ones that we carried to the site of the new nest and that had not had an opportunity to learn the route. The taught ants found their way home much more quickly and successfully.”

The experiments were conducted by undergraduates Jacob Podesta, who is now a PhD student at York, and Edward Jarvis, who was also a Masters student at Professor Nigel Franks’s Lab. The gantry programming was accomplished by Dr. Alan Worley and all the statistical analyses were driven by Dr. Ana Sendova-Franks.

Their approach should make it possible to interrogate further exactly what is involved in successful teaching.

• PAPER – Robotic communication with ants. Nigel R. Franks, Jacob A. Podesta, Edward C. Jarvis, Alan Worley, and Ana B. Sendova-Franks. Journal of Experimental Biology 1 August 2022; 225 (15)

Marine environments present a unique set of challenges that render several technologies that were developed for land applications completely useless. Communicating using sound, or at least as people use sound to communicate, is one of them.

Michael Fulton tackles this challenge with his presentation at ICRA 2022 by using body language to communicate with an AUV underwater. Tune in for more.

His poster can be viewed here.

Michael Fulton

Michael Fulton is a Ph.D. Candidate at the University of Minnesota Twin Cities. His research focuses primarily on underwater robotics with a focus on applications where robots work with humans. Specifically, human-robot interaction and robot perception using computer vision and deep learning, with the intent of creating systems that can work collaboratively with humans in challenging environments.

Image credit: rtbilder / Shutterstock.com

By Conn Hastings, science writer

Honeybees use a sophisticated dance to tell their sisters about the location of nearby flowers. This phenomenon forms the inspiration for a form of robot-robot communication that does not rely on digital networks. A recent study presents a simple technique whereby robots view and interpret each other’s movements or a gesture from a human to communicate a geographical location. This approach could prove invaluable when network coverage is unreliable or absent, such as in disaster zones.

Where are those flowers and how far away are they? This is the crux of the ‘waggle dance’ performed by honeybees to alert others to the location of nectar-rich flowers. A new study in Frontiers in Robotics and AI has taken inspiration from this technique to devise a way for robots to communicate. The first robot traces a shape on the floor, and the shape’s orientation and the time it takes to trace it tell the second robot the required direction and distance of travel. The technique could prove invaluable in situations where robot labor is required but network communications are unreliable, such as in a disaster zone or in space.

Honeybees excel at non-verbal communication

If you have ever found yourself in a noisy environment, such as a factory floor, you may have noticed that humans are adept at communicating using gestures. Well, we aren’t the only ones. In fact, honeybees take non-verbal communication to a whole new level.

By wiggling their backside while parading through the hive, they can let other honeybees know about the location of food. The direction of this ‘waggle dance’ lets other bees know the direction of the food with respect to the hive and the sun, and the duration of the dance lets them know how far away it is. It is a simple but effective way to convey complex geographical coordinates.

Applying the dance to robots

This ingenious method of communication inspired the researchers behind this latest study to apply it to the world of robotics. Robot cooperation allows multiple robots to coordinate and complete complex tasks. Typically, robots communicate using digital networks, but what happens when these are unreliable, such as during an emergency or in remote locations? Moreover, how can humans communicate with robots in such a scenario?

To address this, the researchers designed a visual communication system for robots with on-board cameras, using algorithms that allow the robots to interpret what they see. They tested the system using a simple task, where a package in a warehouse needs to be moved. The system allows a human to communicate with a ‘messenger robot’, which supervises and instructs a ‘handling robot’ that performs the task.

Robot dancing in practice

In this situation, the human can communicate with the messenger robot using gestures, such as a raised hand with a closed fist. The robot can recognize the gesture using its on-board camera and skeletal tracking algorithms. Once the human has shown the messenger robot where the package is, it conveys this information to the handling robot.

This involves positioning itself in front of the handling robot and tracing a specific shape on the ground. The orientation of the shape indicates the required direction of travel, while the length of time it takes to trace it indicates the distance. This robot dance would make a worker bee proud, but did it work?

• Read original article
• Download original article (pdf)

The researchers put it to the test using a computer simulation, and with real robots and human volunteers. The robots interpreted the gestures correctly 90% and 93.3% of the time, respectively, highlighting the potential of the technique.

“This technique could be useful in places where communication network coverage is insufficient and intermittent, such as robot search-and-rescue operations in disaster zones or in robots that undertake space walks,” said Prof Abhra Roy Chowdhury of the Indian Institute of Science, senior author on the study. “This method depends on robot vision through a simple camera, and therefore it is compatible with robots of various sizes and configurations and is scalable,” added Kaustubh Joshi of the University of Maryland, first author on the study.

Video credit: K Joshi and AR Chowdury

This article was originally published on the Frontiers blog.

Inspired by fireflies, MIT researchers have created soft actuators that can emit light in different colors or patterns. Credits: Courtesy of the researchers

By Adam Zewe | MIT News Office

Fireflies that light up dusky backyards on warm summer evenings use their luminescence for communication — to attract a mate, ward off predators, or lure prey.

These glimmering bugs also sparked the inspiration of scientists at MIT. Taking a cue from nature, they built electroluminescent soft artificial muscles for flying, insect-scale robots. The tiny artificial muscles that control the robots’ wings emit colored light during flight.

This electroluminescence could enable the robots to communicate with each other. If sent on a search-and-rescue mission into a collapsed building, for instance, a robot that finds survivors could use lights to signal others and call for help.

The ability to emit light also brings these microscale robots, which weigh barely more than a paper clip, one step closer to flying on their own outside the lab. These robots are so lightweight that they can’t carry sensors, so researchers must track them using bulky infrared cameras that don’t work well outdoors. Now, they’ve shown that they can track the robots precisely using the light they emit and just three smartphone cameras.

“If you think of large-scale robots, they can communicate using a lot of different tools — Bluetooth, wireless, all those sorts of things. But for a tiny, power-constrained robot, we are forced to think about new modes of communication. This is a major step toward flying these robots in outdoor environments where we don’t have a well-tuned, state-of-the-art motion tracking system,” says Kevin Chen, who is the D. Reid Weedon, Jr. Assistant Professor in the Department of Electrical Engineering and Computer Science (EECS), the head of the Soft and Micro Robotics Laboratory in the Research Laboratory of Electronics (RLE), and the senior author of the paper.

He and his collaborators accomplished this by embedding miniscule electroluminescent particles into the artificial muscles. The process adds just 2.5 percent more weight without impacting the flight performance of the robot.

Joining Chen on the paper are EECS graduate students Suhan Kim, the lead author, and Yi-Hsuan Hsiao; Yu Fan Chen SM ’14, PhD ’17; and Jie Mao, an associate professor at Ningxia University. The research was published this month in IEEE Robotics and Automation Letters.

A light-up actuator

These researchers previously demonstrated a new fabrication technique to build soft actuators, or artificial muscles, that flap the wings of the robot. These durable actuators are made by alternating ultrathin layers of elastomer and carbon nanotube electrode in a stack and then rolling it into a squishy cylinder. When a voltage is applied to that cylinder, the electrodes squeeze the elastomer, and the mechanical strain flaps the wing.

To fabricate a glowing actuator, the team incorporated electroluminescent zinc sulphate particles into the elastomer but had to overcome several challenges along the way.

First, the researchers had to create an electrode that would not block light. They built it using highly transparent carbon nanotubes, which are only a few nanometers thick and enable light to pass through.

However, the zinc particles only light up in the presence of a very strong and high-frequency electric field. This electric field excites the electrons in the zinc particles, which then emit subatomic particles of light known as photons. The researchers use high voltage to create a strong electric field in the soft actuator, and then drive the robot at a high frequency, which enables the particles to light up brightly.

“Traditionally, electroluminescent materials are very energetically costly, but in a sense, we get that electroluminescence for free because we just use the electric field at the frequency we need for flying. We don’t need new actuation, new wires, or anything. It only takes about 3 percent more energy to shine out light,” Kevin Chen says.

As they prototyped the actuator, they found that adding zinc particles reduced its quality, causing it to break down more easily. To get around this, Kim mixed zinc particles into the top elastomer layer only. He made that layer a few micrometers thicker to accommodate for any reduction in output power.

While this made the actuator 2.5 percent heavier, it emitted light without impacting flight performance.

“We put a lot of care into maintaining the quality of the elastomer layers between the electrodes. Adding these particles was almost like adding dust to our elastomer layer. It took many different approaches and a lot of testing, but we came up with a way to ensure the quality of the actuator,” Kim says.

Adjusting the chemical combination of the zinc particles changes the light color. The researchers made green, orange, and blue particles for the actuators they built; each actuator shines one solid color.

They also tweaked the fabrication process so the actuators could emit multicolored and patterned light. The researchers placed a tiny mask over the top layer, added zinc particles, then cured the actuator. They repeated this process three times with different masks and colored particles to create a light pattern that spelled M-I-T.

These artificial muscles, which control the wings of featherweight flying robots, light up while the robot is in flight, which provides a low-cost way to track the robots and also could enable them to communicate. Credits: Courtesy of the researchers

Following the fireflies

Once they had finetuned the fabrication process, they tested the mechanical properties of the actuators and used a luminescence meter to measure the intensity of the light.

From there, they ran flight tests using a specially designed motion-tracking system. Each electroluminescent actuator served as an active marker that could be tracked using iPhone cameras. The cameras detect each light color, and a computer program they developed tracks the position and attitude of the robots to within 2 millimeters of state-of-the-art infrared motion capture systems.

“We are very proud of how good the tracking result is, compared to the state-of-the-art. We were using cheap hardware, compared to the tens of thousands of dollars these large motion-tracking systems cost, and the tracking results were very close,” Kevin Chen says.

In the future, they plan to enhance that motion tracking system so it can track robots in real-time. The team is working to incorporate control signals so the robots could turn their light on and off during flight and communicate more like real fireflies. They are also studying how electroluminescence could even improve some properties of these soft artificial muscles, Kevin Chen says.

“This work is really interesting because it minimizes the overhead (weight and power) for light generation without compromising flight performance,” says Kaushik Jayaram, an assistant professor in Department of Mechanical Engineering at the University of Colorado at Boulder, who was not involved with this research. “The wingbeat synchronized flash generation demonstrated in this work will make it easier for motion tracking and flight control of multiple microrobots in low-light environments both indoors and outdoors.”

“While the light production, the reminiscence of biological fireflies, and the potential use of communication presented in this work are extremely interesting, I believe the true momentum is that this latest development could turn out to be a milestone toward the demonstration of these robots outside controlled laboratory conditions,” adds Pakpong Chirarattananon, an associate professor in the Department of Biomedical Engineering at the City University of Hong Kong, who also was not involved with this work. “The illuminated actuators potentially act as active markers for external cameras to provide real-time feedback for flight stabilization to replace the current motion capture system. The electroluminescence would allow less sophisticated equipment to be used and the robots to be tracked from distance, perhaps via another larger mobile robot, for real-world deployment. That would be a remarkable breakthrough. I would be thrilled to see what the authors accomplish next.”

This work was supported by the Research Laboratory of Electronics at MIT.

• PAPER – FireFly: An Insect-Scale Aerial Robot Powered by Electroluminescent Soft Artificial Muscles. Suhan Kim, Yi-Hsuan Hsiao, YuFan Chen, Jie Mao, and YuFeng Chen. IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 6950-6957, July 2022, doi: 10.1109/LRA.2022.3179486.

By Alexander Badri-Sprowitz, Alborz Aghamaleki Sarvestani, Metin Sitti and Linda Behringer

If a Tyrannosaurus Rex living 66 million years ago featured a similar leg structure as an ostrich running in the savanna today, then we can assume bird legs stood the test of time – a good example of evolutionary selection.

Graceful, elegant, powerful – flightless birds like the ostrich are a mechanical wonder. Ostriches, some of which weigh over 100kg, run through the savanna at up to 55km/h. The ostrich’s outstanding locomotor performance is thought to be enabled by the animal’s leg structure. Unlike humans, birds fold their feet back when pulling their legs up towards their bodies. Why do the animals do this? Why is this foot movement pattern energy-efficient for walking and running? And can the bird’s leg structure with all its bones, muscles, and tendons be transferred to walking robots?

Alexander Badri-Spröwitz has spent more than five years on these questions. At the Max Planck Institute for Intelligent Systems (MPI-IS), he leads the Dynamic Locomotion Group. His team works at the interface between biology and robotics in the field of biomechanics and neurocontrol. The dynamic locomotion of animals and robots is the group’s main focus.

Together with his doctoral student Alborz Aghamaleki Sarvestani, Badri-Spröwitz has constructed a robot leg that, like its natural model, is energy-efficient: BirdBot needs fewer motors than other machines and could, theoretically, scale to large size. On March 16th, Badri-Spröwitz, Aghamaleki Sarvestani, the roboticist Metin Sitti, a director at MPI-IS, and biology professor Monica A. Daley of the University of California, Irvine, published their research in the renowned journal Science Robotics.

Compliant spring-tendon network made of muscles and tendons

When walking, humans pull their feet up and bend their knees, but feet and toes point forward almost unchanged. It is known that Birds are different — in the swing phase, they fold their feet backward. But what is the function of this motion? Badri-Spröwitz and his team attribute this movement to a mechanical coupling. “It’s not the nervous system, it’s not electrical impulses, it’s not muscle activity,” Badri-Spröwitz explains. “We hypothesized a new function of the foot-leg coupling through a network of muscles and tendons that extends across multiple joints”. These multi-joint muscle-tendon coordinate foot folding in the swing phase. In our robot, we have implemented the coupled mechanics in the leg and foot, which enables energy-efficient and robust robot walking. Our results demonstrating this mechanism in a robot lead us to believe that similar efficiency benefits also hold true for birds,” he explains.

The coupling of the leg and foot joints and the forces and movements involved could be the reason why a large animal like an ostrich can not only run fast but also stand without tiring, the researchers speculate. A person weighing over 100kg can also stand well and for a long time, but only with the knees ‘locked’ in an extended position. If the person were to squat slightly, it becomes strenuous after a few minutes. The bird, however, does not seem to mind its bent leg structure; many birds even stand upright while sleeping. A robotic bird’s leg should be able to do the same: no motor power should be needed to keep the structure standing upright.

Robot walks on treadmill

To test their hypothesis, the researchers built a robotic leg modeled after the leg of a flightless bird. They constructed their artificial bird leg so that its foot features no motor, but instead a joint equipped with a spring and cable mechanism. The foot is mechanically coupled to the rest of the leg’s joints through cables and pulleys. Each leg contains only two motors— the hip joints motor, which swings the leg back and forth, and a small motor that flexes the knee joint to pull the leg up. After assembly, the researchers walked BirdBot on a treadmill to observe the robot’s foot folding and unfolding. “The foot and leg joints don’t need actuation in the stance phase,” says Aghamaleki Sarvestani. “Springs power these joints, and the multi-joint spring-tendon mechanism coordinates joint movements. When the leg is pulled into swing phase, the foot disengages the leg’s spring – or the muscle-tendon spring, as we believe it happens in animals,” Badri-Spröwitz adds. A video shows BirdBot walking in the research group’s laboratory.

Zero effort when standing, and when flexing the leg and knee

When standing, the leg expends zero energy. “Previously, our robots had to work against the spring or with a motor either when standing or when pulling the leg up, to prevent the leg from colliding with the ground during leg swing. This energy input is not necessary in BirdBot’s legs,” says Badri-Spröwitz and Aghamaleki Sarvestani adds: “Overall, the new robot requires a mere quarter of the energy of its predecessor.”

The treadmill is now switched back on, the robot starts running, and with each leg swing, the foot disengages the leg’s spring. To disengage, the large foot movement slacks the cable and the remaining leg joints swing loosely. This transition of states, between standing and leg swing, is provided in most robots by a motor at the joint. And a sensor sends a signal to a controller, which turns the robot’s motors on and off. “Previously, motors were switched depending on whether the leg was in the swing or stance phase. Now the foot takes over this function in the walking machine, mechanically switching between stance and swing. We only need one motor at the hip joint and one motor to bend the knee in the swing phase. We leave leg spring engagement and disengagement to the bird-inspired mechanics. This is robust, fast, and energy-efficient,” says Badri-Spröwitz.

Motion sequence of BirdBot’s leg; left is touch-down, then stance (the three first snapshots on the left), then mid-swing with the leg’s characteristic leg flexing posture, and back to touch-down on the right. Springs and spring-tendons are shown on top.

Monica Daley observed in several of her earlier biology studies that the bird’s leg structure not only saves energy during walking and standing but is also adapted by nature so that the animal hardly stumbles and injures itself. In experiments with guineafowls running over hidden potholes, she quantified the birds’ remarkable locomotion robustness. A morphological intelligence is built into the system that allows the animal to act quickly – without having to think about it. Daley had shown that the animals control their legs during locomotion not only with the help of the nervous system. If an obstacle unexpectedly lies in the way, it is not always the animal’s sense of touch or sight that comes into play.

“The structure with its multi-jointed muscle-tendons and its unique foot movement can explain why even heavy, large birds run so quickly, robustly, and energy-efficient. If I assume that everything in the bird is based on sensing and action, and the animal steps onto an unexpected obstacle, the animal might not be able to react quickly enough. Perception and sensing, even the transmission of the stimuli, and the reaction cost time,” Daley says.

Yet Daley’s work on running birds over 20 years demonstrates that birds respond more rapidly than the nervous system allows, indicating mechanical contributions to control. Now that the team developed BirdBot, which is a physical model that directly demonstrates how these mechanisms work, it all makes more sense: the leg switches mechanically if there is a bump in the ground. The switch happens immediately and without time delay. Like birds, the robot features high locomotion robustness.

Whether it’s on the scale of a Tyrannosaurus Rex or a small quail, or a small or large robotic leg. Theoretically, meter-high legs can now be implemented to carry robots with the weight of several tons, that walk around with little power input.

The knowledge gained through BirdBot developed at the Dynamic Locomotion Group and the University of California, Irvine, leads to new insights about animals, which are adapted by evolution. Robots allow testing and sometimes confirming hypotheses from Biology, and advancing both fields.

• PAPER – BirdBot achieves energy-efficient gait with minimal control using avian-inspired leg clutching. Alexander Badri-Spröwitz, Alborz Aghamaleki Sarvestani, Metin Sitti and Monica A. Daley. Science Robotics, Vol 7, Issue 64. DOI: 10.1126/scirobotics.abg4055

Front view of the flying robot. Image credit: Dr Tim Helps

This new advance, published in the journal Science Robotics, could pave the way for smaller, lighter and more effective micro flying robots for environmental monitoring, search and rescue, and deployment in hazardous environments.

Until now, typical micro flying robots have used motors, gears and other complex transmission systems to achieve the up-and-down motion of the wings. This has added complexity, weight and undesired dynamic effects.

Taking inspiration from bees and other flying insects, researchers from Bristol’s Faculty of Engineering, led by Professor of Robotics Jonathan Rossiter, have successfully demonstrated a direct-drive artificial muscle system, called the Liquid-amplified Zipping Actuator (LAZA), that achieves wing motion using no rotating parts or gears.

The LAZA system greatly simplifies the flapping mechanism, enabling future miniaturization of flapping robots down to the size of insects.

In the paper, the team show how a pair of LAZA-powered flapping wings can provide more power compared with insect muscle of the same weight, enough to fly a robot across a room at 18 body lengths per second.

They also demonstrated how the LAZA can deliver consistent flapping over more than one million cycles, important for making flapping robots that can undertake long-haul flights.

The team expect the LAZA to be adopted as a fundamental building block for a range of autonomous insect-like flying robots.

Dr Tim Helps, lead author and developer of the LAZA system said: “With the LAZA, we apply electrostatic forces directly on the wing, rather than through a complex, inefficient transmission system. This leads to better performance, simpler design, and will unlock a new class of low-cost, lightweight flapping micro-air vehicles for future applications, like autonomous inspection of off-shore wind turbines.”

Professor Rossiter added: “Making smaller and better performing flapping wing micro robots is a huge challenge. LAZA is an important step toward autonomous flying robots that could be as small as insects and perform environmentally critical tasks such as plant pollination and exciting emerging roles such as finding people in collapsed buildings.”

• PAPER – Liquid-amplified zipping actuators for micro-air vehicles with transmission-free flapping. T. Helps, C. Romero, M. Taghavi, A. Conn and J. Rossiter. Science Robotics, vol. 7 (63)

The 35th conference on Neural Information Processing Systems (NeurIPS2021) featured eight invited talks. In this post, we give a flavour of the final presentation.

The collective intelligence of army ants, and the robots they inspire

Radhika Nagpal

Radhika’s research focusses on collective intelligence, with the overarching goal being to understand how large groups of individuals, with local interaction rules, can cooperate to achieve globally complex behaviour. These are fascinating systems. Each individual is miniscule compared to the massive phenomena that they create, and, with a limited view of the actions of the rest of the swarm, they achieve striking coordination.

Looking at collective intelligence from an algorithmic point-of-view, the phenomenon emerges from many individuals interacting using simple rules. When run by these large, decentralised groups, these simple rules result in highly intelligent behaviour.

The subject of Radhika’s talk was army ants, a species which spectacularly demonstrate collective intelligence. Without any leader, millions of ants work together to self-assemble nests and build bridge structures using their own bodies.

One particular aspect of study concerned self-assembly of such bridges. Radhika’s research team, which comprised three roboticists and two biologists, found that the ants created bridges adapt to traffic flow and terrain. The ants also disassembled the bridge when the flow of ants had stopped and it wasn’t needed any more.

The team proposed the following simple hypothesis to explain this behaviour using local rules: if an ant is walking along, and experiences congestion (i.e. another ant steps on it), then it becomes stationary and turns into a bridge, allowing other ants to walk over it. Then, if no ants are walking on it any more, it can get up and leave.

These observations, and this hypothesis, led the team to consider two research questions:

• Could they build a robot swarm with soft robots that can self-assemble amorphous structures, just like the ant bridges?
• Could they formulate rules which allowed these robots to self-assemble temporary and adaptive bridge structures?

There were two motivations for these questions. Firstly, the goal of moving closer to realising robot swarms that can solve problems in a particular environment. Secondly, the use of a synthetic system to better understand the collective intelligence of army ants.

Screenshot from Radhika’s talk

Radhika showed a demonstration of the soft robot designed by her group. It has two feet and a soft body, and moves by flipping – one foot remains attached, while the other detaches from the surface and flips to attach in a different place. This allows movement in any orientation. Upon detaching, a foot searches through space to find somewhere to attach. By using grippers on the feet that can hook onto textured surfaces, and having a stretchable Velcro skin, the robots can climb over each other, like the ants. The robot pulses, and uses a vibration sensor, to detect whether it is in contact with another robot. A video demonstration of two robots interacting showed that they have successfully created a system that can recreate the simple hypothesis outlined above.

In order to investigate the high-level properties of army ant bridges, which would require a vast number of robots, the team created a simulation. Modelling the ants to have the same characteristics as their physical robots, they were able to replicate the high level properties of army ant bridges with their hypothesized rules.

You can read the round-ups of the other NeurIPS invited talks at these links:
#NeurIPS2021 invited talks round-up: part one – Duolingo, the banality of scale and estimating the mean
#NeurIPS2021 invited talks round-up: part two – benign overfitting, optimal transport, and human and machine intelligence

AI-designed (C-shaped) organisms push loose stem cells (white) into piles as they move through their environment. Credit: Douglas Blackiston and Sam Kriegman

By Joshua Brown, University of Vermont Communications

To persist, life must reproduce. Over billions of years, organisms have evolved many ways of replicating, from budding plants to sexual animals to invading viruses.

Now scientists at the University of Vermont, Tufts University, and the Wyss Institute for Biologically Inspired Engineering at Harvard University have discovered an entirely new form of biological reproduction—and applied their discovery to create the first-ever, self-replicating living robots.

The same team that built the first living robots (“Xenobots,” assembled from frog cells—reported in 2020) has discovered that these computer-designed and hand-assembled organisms can swim out into their tiny dish, find single cells, gather hundreds of them together, and assemble “baby” Xenobots inside their Pac-Man-shaped “mouth”—that, a few days later, become new Xenobots that look and move just like themselves.

And then these new Xenobots can go out, find cells, and build copies of themselves. Again and again.

“With the right design—they will spontaneously self-replicate,” says Joshua Bongard, Ph.D., a computer scientist and robotics expert at the University of Vermont who co-led the new research.

The results of the new research were published in the Proceedings of the National Academy of Sciences.

Into the Unknown

In a Xenopus laevis frog, these embryonic cells would develop into skin. “They would be sitting on the outside of a tadpole, keeping out pathogens and redistributing mucus,” says Michael Levin, Ph.D., a professor of biology and director of the Allen Discovery Center at Tufts University and co-leader of the new research. “But we’re putting them into a novel context. We’re giving them a chance to reimagine their multicellularity.” Levin is also an Associate Faculty member at the Wyss Institute.

As Pac-man-shaped Xenobot “parents” move around their environment, they collect loose stem cells in their “mouths” that, over time, aggregate to create “offspring” Xenobots that develop to look just like their creators. Credit: Doug Blackiston and Sam Kriegman

And what they imagine is something far different than skin. “People have thought for quite a long time that we’ve worked out all the ways that life can reproduce or replicate. But this is something that’s never been observed before,” says co-author Douglas Blackiston, Ph.D., the senior scientist at Tufts University and the Wyss Institute who assembled the Xenobot “parents” and developed the biological portion of the new study.

“This is profound,” says Levin. “These cells have the genome of a frog, but, freed from becoming tadpoles, they use their collective intelligence, a plasticity, to do something astounding.” In earlier experiments, the scientists were amazed that Xenobots could be designed to achieve simple tasks. Now they are stunned that these biological objects—a computer-designed collection of cells—will spontaneously replicate. “We have the full, unaltered frog genome,” says Levin, “but it gave no hint that these cells can work together on this new task,” of gathering and then compressing separated cells into working self-copies.

“These are frog cells replicating in a way that is very different from how frogs do it. No animal or plant known to science replicates in this way,” says Sam Kriegman, Ph.D., the lead author on the new study, who completed his Ph.D. in Bongard’s lab at UVM and is now a post-doctoral researcher at Tuft’s Allen Center and Harvard University’s Wyss Institute for Biologically Inspired Engineering.

On its own, the Xenobot parent, made of some 3,000 cells, forms a sphere. “These can make children but then the system normally dies out after that. It’s very hard, actually, to get the system to keep reproducing,” says Kriegman. But with an artificial intelligence program working on the Deep Green supercomputer cluster at UVM’s Vermont Advanced Computing Core, an evolutionary algorithm was able to test billions of body shapes in simulation—triangles, squares, pyramids, starfish—to find ones that allowed the cells to be more effective at the motion-based “kinematic” replication reported in the new research.

“We asked the supercomputer at UVM to figure out how to adjust the shape of the initial parents, and the AI came up with some strange designs after months of chugging away, including one that resembled Pac-Man,” says Kriegman. “It’s very non-intuitive. It looks very simple, but it’s not something a human engineer would come up with. Why one tiny mouth? Why not five? We sent the results to Doug and he built these Pac-Man-shaped parent Xenobots. Then those parents built children, who built grandchildren, who built great-grandchildren, who built great-great-grandchildren.” In other words, the right design greatly extended the number of generations.

Kinematic replication is well-known at the level of molecules—but it has never been observed before at the scale of whole cells or organisms.

An AI-designed “parent” organism (C shape; red) beside stem cells that have been compressed into a ball (“offspring”; green). Credit: Douglas Blackiston and Sam Kriegman

“We’ve discovered that there is this previously unknown space within organisms, or living systems, and it’s a vast space,” says Bongard. “How do we then go about exploring that space? We found Xenobots that walk. We found Xenobots that swim. And now, in this study, we’ve found Xenobots that kinematically replicate. What else is out there?”

Or, as the scientists write in the Proceedings of the National Academy of Sciences study: “life harbors surprising behaviors just below the surface, waiting to be uncovered.”

Responding to Risk

Some people may find this exhilarating. Others may react with concern, or even terror, to the notion of a self-replicating biotechnology. For the team of scientists, the goal is deeper understanding.

“We are working to understand this property: replication. The world and technologies are rapidly changing. It’s important, for society as a whole, that we study and understand how this works,” says Bongard. These millimeter-sized living machines, entirely contained in a laboratory, easily extinguished, and vetted by federal, state and institutional ethics experts, “are not what keep me awake at night. What presents risk is the next pandemic; accelerating ecosystem damage from pollution; intensifying threats from climate change,” says UVM’s Bongard. “This is an ideal system in which to study self-replicating systems. We have a moral imperative to understand the conditions under which we can control it, direct it, douse it, exaggerate it.”

Bongard points to the COVID epidemic and the hunt for a vaccine. “The speed at which we can produce solutions matters deeply. If we can develop technologies, learning from Xenobots, where we can quickly tell the AI: ‘We need a biological tool that does X and Y and suppresses Z,’ —that could be very beneficial. Today, that takes an exceedingly long time.” The team aims to accelerate how quickly people can go from identifying a problem to generating solutions—”like deploying living machines to pull microplastics out of waterways or build new medicines,” Bongard says.

“We need to create technological solutions that grow at the same rate as the challenges we face,” Bongard says.

And the team sees promise in the research for advancements toward regenerative medicine. “If we knew how to tell collections of cells to do what we wanted them to do, ultimately, that’s regenerative medicine—that’s the solution to traumatic injury, birth defects, cancer, and aging,” says Levin. “All of these different problems are here because we don’t know how to predict and control what groups of cells are going to build. Xenobots are a new platform for teaching us.”

The scientists behind the Xenobots participated in a live panel discussion on December 1, 2021 to discuss the latest developments in their research. Credit: Wyss Institute at Harvard University

• PUBLICATION – Kinematic self-replication in reconfigurable organisms. Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard, Proceedings of the National Academy of Sciences Dec 2021, 118 (49) e2112672118; DOI: 10.1073/pnas.2112672118

MIT researchers have developed a system that improves the speed and agility of legged robots as they jump across gaps in the terrain. Credits: Photo courtesy of the researchers

By Adam Zewe | MIT News Office

A loping cheetah dashes across a rolling field, bounding over sudden gaps in the rugged terrain. The movement may look effortless, but getting a robot to move this way is an altogether different prospect.

In recent years, four-legged robots inspired by the movement of cheetahs and other animals have made great leaps forward, yet they still lag behind their mammalian counterparts when it comes to traveling across a landscape with rapid elevation changes.

“In those settings, you need to use vision in order to avoid failure. For example, stepping in a gap is difficult to avoid if you can’t see it. Although there are some existing methods for incorporating vision into legged locomotion, most of them aren’t really suitable for use with emerging agile robotic systems,” says Gabriel Margolis, a PhD student in the lab of Pulkit Agrawal, professor in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT.

Now, Margolis and his collaborators have developed a system that improves the speed and agility of legged robots as they jump across gaps in the terrain. The novel control system is split into two parts — one that processes real-time input from a video camera mounted on the front of the robot and another that translates that information into instructions for how the robot should move its body. The researchers tested their system on the MIT mini cheetah, a powerful, agile robot built in the lab of Sangbae Kim, professor of mechanical engineering.

Unlike other methods for controlling a four-legged robot, this two-part system does not require the terrain to be mapped in advance, so the robot can go anywhere. In the future, this could enable robots to charge off into the woods on an emergency response mission or climb a flight of stairs to deliver medication to an elderly shut-in.

Margolis wrote the paper with senior author Pulkit Agrawal, who heads the Improbable AI lab at MIT and is the Steven G. and Renee Finn Career Development Assistant Professor in the Department of Electrical Engineering and Computer Science; Professor Sangbae Kim in the Department of Mechanical Engineering at MIT; and fellow graduate students Tao Chen and Xiang Fu at MIT. Other co-authors include Kartik Paigwar, a graduate student at Arizona State University; and Donghyun Kim, an assistant professor at the University of Massachusetts at Amherst. The work will be presented next month at the Conference on Robot Learning.

It’s all under control

The use of two separate controllers working together makes this system especially innovative.

A controller is an algorithm that will convert the robot’s state into a set of actions for it to follow. Many blind controllers — those that do not incorporate vision — are robust and effective but only enable robots to walk over continuous terrain.

Vision is such a complex sensory input to process that these algorithms are unable to handle it efficiently. Systems that do incorporate vision usually rely on a “heightmap” of the terrain, which must be either preconstructed or generated on the fly, a process that is typically slow and prone to failure if the heightmap is incorrect.

To develop their system, the researchers took the best elements from these robust, blind controllers and combined them with a separate module that handles vision in real-time.

The robot’s camera captures depth images of the upcoming terrain, which are fed to a high-level controller along with information about the state of the robot’s body (joint angles, body orientation, etc.). The high-level controller is a neural network that “learns” from experience.

That neural network outputs a target trajectory, which the second controller uses to come up with torques for each of the robot’s 12 joints. This low-level controller is not a neural network and instead relies on a set of concise, physical equations that describe the robot’s motion.

“The hierarchy, including the use of this low-level controller, enables us to constrain the robot’s behavior so it is more well-behaved. With this low-level controller, we are using well-specified models that we can impose constraints on, which isn’t usually possible in a learning-based network,” Margolis says.

Teaching the network

The researchers used the trial-and-error method known as reinforcement learning to train the high-level controller. They conducted simulations of the robot running across hundreds of different discontinuous terrains and rewarded it for successful crossings.

Over time, the algorithm learned which actions maximized the reward.

Then they built a physical, gapped terrain with a set of wooden planks and put their control scheme to the test using the mini cheetah.

“It was definitely fun to work with a robot that was designed in-house at MIT by some of our collaborators. The mini cheetah is a great platform because it is modular and made mostly from parts that you can order online, so if we wanted a new battery or camera, it was just a simple matter of ordering it from a regular supplier and, with a little bit of help from Sangbae’s lab, installing it,” Margolis says.

From left to right: PhD students Tao Chen and Gabriel Margolis; Pulkit Agrawal, the Steven G. and Renee Finn Career Development Assistant Professor in the Department of Electrical Engineering and Computer Science; and PhD student Xiang Fu. Credits: Photo courtesy of the researchers

Estimating the robot’s state proved to be a challenge in some cases. Unlike in simulation, real-world sensors encounter noise that can accumulate and affect the outcome. So, for some experiments that involved high-precision foot placement, the researchers used a motion capture system to measure the robot’s true position.

Their system outperformed others that only use one controller, and the mini cheetah successfully crossed 90 percent of the terrains.

“One novelty of our system is that it does adjust the robot’s gait. If a human were trying to leap across a really wide gap, they might start by running really fast to build up speed and then they might put both feet together to have a really powerful leap across the gap. In the same way, our robot can adjust the timings and duration of its foot contacts to better traverse the terrain,” Margolis says.

Leaping out of the lab

While the researchers were able to demonstrate that their control scheme works in a laboratory, they still have a long way to go before they can deploy the system in the real world, Margolis says.

In the future, they hope to mount a more powerful computer to the robot so it can do all its computation on board. They also want to improve the robot’s state estimator to eliminate the need for the motion capture system. In addition, they’d like to improve the low-level controller so it can exploit the robot’s full range of motion, and enhance the high-level controller so it works well in different lighting conditions.

“It is remarkable to witness the flexibility of machine learning techniques capable of bypassing carefully designed intermediate processes (e.g. state estimation and trajectory planning) that centuries-old model-based techniques have relied on,” Kim says. “I am excited about the future of mobile robots with more robust vision processing trained specifically for locomotion.”

The research is supported, in part, by the MIT’s Improbable AI Lab, Biomimetic Robotics Laboratory, NAVER LABS, and the DARPA Machine Common Sense Program.

By Daniel Quinn

Underwater vehicles haven’t changed much since the submarines of World War II. They’re rigid, fairly boxy and use propellers to move. And whether they are large manned vessels or small robots, most underwater vehicles have one cruising speed where they are most energy efficient.

Fish take a very different approach to moving through water: Their bodies and fins are very flexible, and this flexibility allows them to interact with water more efficiently than rigid machines. Researchers have been designing and building flexible fishlike robots for years, but they still trail far behind real fish in terms of efficiency.

What’s missing?

I am an engineer and study fluid dynamics. My labmates and I wondered if something in particular about the flexibility of fish tails allows fish to be so fast and efficient in the water. So, we created a model and built a robot to study the effect of stiffness on swimming efficiency. We found fish swim so efficiently over a wide range of speeds because they can change how rigid or flexible their tails are in real time.

Why are people still using propellers?

Fluid dynamics applies to both liquids and gasses. Humans have been using rotating rigid objects to move vehicles for hundreds of years – Leonardo Da Vinci incorporated the concept into his helicopter designs, and the first propeller–driven boats were built in the 1830s. Propellers are easy to make, and they work just fine at their designed cruise speed.

It has only been in the past couple of decades that advances in soft robotics have made actively controlled flexible components a reality. Now, marine roboticists are turning to flexible fish and their amazing swimming abilities for inspiration.

When engineers like me talk about flexibility in a swimming robot, we are usually referring to how stiff the tail of the fish is. The tail is the entire rear half of a fish’s body that moves back and forth when it swims.

Consider tuna, which can swim up to 50 mph and are extremely energy efficient over a wide range of speeds.

The tricky part about copying the biomechanics of fish is that biologists don’t know how flexible they are in the real world. If you want to know how flexible a rubber band is, you simply pull on it. If you pull on a fish’s tail, the stiffness depends on how much the fish is tensing its various muscles.

The best that researchers can do to estimate flexibility is film a swimming fish and measure how its body shape changes.

Searching for answers in the math

Researchers have built dozens of robots in an attempt to mimic the flexibility and swimming patterns of tuna and other fish, but none have matched the performance of the real things.

In my lab at the University of Virginia, my colleagues and I ran into the same questions as others: How flexible should our robot be? And if there’s no one best flexibility, how should our robot change its stiffness as it swims?

We looked for the answer in an old NASA paper about vibrating airplane wings. The report explains how when a plane’s wings vibrate, the vibrations change the amount of lift the wings produce. Since fish fins and airplane wings have similar shapes, the same math works well to model how much thrust fish tails produce as they flap back and forth.

Using the old wing theory, postdoctoral researcher Qiang Zhong and I created a mathematical model of a swimming fish and added a spring and pulley to the tail to represent the effects of a tensing muscle. We discovered a surprisingly simple hypothesis hiding in the equations. To maximize efficiency, muscle tension needs to increase as the square of swimming speed. So, if swimming speed doubles, stiffness needs to increase by a factor of four. To swim three times faster while maintaining high efficiency, a fish or fish-like robot needs to pull on its tendon about nine times harder.

To confirm our theory, we simply added an artificial tendon to one of our tunalike robots and then programmed the robot to vary its tail stiffness based on speed. We then put our new robot into our test tank and ran it through various “missions” – like a 200-meter sprint where it had to dodge simulated obstacles. With the ability to vary its tail’s flexibility, the robot used about half as much energy on average across a wide range of speeds compared to robots with a single stiffness.

Why it matters

While it is great to build one excellent robot, the thing my colleagues and I are most excited about is that our model is adaptable. We can tweak it based on body size, swimming style or even fluid type. It can be applied to animals and machines whether they are big or small, swimmers or flyers.

For example, our model suggests that dolphins have a lot to gain from the ability to vary their tails’ stiffness, whereas goldfish don’t get much benefit due to their body size, body shape and swimming style.

The model also has applications for robotic design too. Higher energy efficiency when swimming or flying – which also means quieter robots – would enable radically new missions for vehicles and robots that currently have only one efficient cruising speed. In the short term, this could help biologists study river beds and coral reefs more easily, enable researchers to track wind and ocean currents at unprecedented scales or allow search and rescue teams to operate farther and longer.

In the long term, I hope our research could inspire new designs for submarines and airplanes. Humans have only been working on swimming and flying machines for a couple centuries, while animals have been perfecting their skills for millions of years. There’s no doubt there is still a lot to learn from them.

Daniel Quinn receives funding from The National Science Foundation and The Office of Naval Research.

This article appeared in The Conversation.

The big idea

Segmented hinges in the long, thin bones of fish fins are critical to the incredible mechanical properties of fins, and this design could inspire improved underwater propulsion systems, new robotic materials and even new aircraft designs.

Fish fins are not simple membranes that fish flap right and left for propulsion. They probably represent one of the most elegant ways to interact with water. Fins are flexible enough to morph into a wide variety of shapes, yet they are stiff enough to push water without collapsing.

The secret is in the structure: Most fish have rays – long, bony spikes that stiffen the thin membranes of collagen that make up their fins. Each of these rays is made of two stiff rows of small bone segments surrounding a softer inner layer. Biologists have long known that fish can change the shape of their fins using muscles and tendons that push or pull on the base of each ray, but very little research has been done looking specifically at the mechanical benefits of the segmented structure.

A pufferfish uses its small but efficient fins to swim against, and maneuver in, a strong current.

To study the mechanical properties of segmented rays, my colleagues and I used theoretical models and 3D-printed fins to compare segmented rays with rays made of a non-segmented flexible material.

We showed that the numerous small, bony segments act as hinge points, making it easy to flex the two bony rows in the ray side to side. This flexibility allows the muscles and tendons at the base of rays to morph a fin using minimal amounts of force. Meanwhile, the hinge design makes it hard to deform the ray along its length. This prevents fins from collapsing when they are subjected to the pressure of water during swimming. In our 3D-printed rays, the segmented designs were four times easier to morph than continuous designs while maintaining the same stiffness.

Why it matters

Morphing materials – materials whose shape can be changed – come in two varieties. Some are very flexible – like hydrogels – but these materials collapse easily when you subject them to external forces. Morphing materials can also be very stiff – like some aerospace composites – but it takes a lot of force to make small changes in their shape.

The segmented structure design of fish fins overcomes this functional trade-off by being highly flexible as well as strong. Materials based on this design could be used in underwater propulsion and improve the agility and speed of fish-inspired submarines. They could also be incredibly valuable in soft robotics and allow tools to change into a wide variety of shapes while still being able to grasp objects with a lot of force. Segmented ray designs could even benefit the aerospace field. Morphing wings that could radically change their geometry, yet carry large aerodynamic forces, could revolutionize the way aircraft take off, maneuver and land.

What still isn’t known

While this research goes a long way in explaining how fish fins work, the mechanics at play when fish fins are bent far from their normal positions are still a bit of a mystery. Collagen tends to get stiffer the more deformed it gets, and my colleagues and I suspect that this stiffening response – together with how collagen fibers are oriented within fish fins – improves the mechanical performance of the fins when they are highly deformed.

What’s next

I am fascinated by the biomechanics of natural fish fins, but my ultimate goal is to develop new materials and devices that are inspired by their mechanical properties. My colleagues and I are currently developing proof-of-concept materials that we hope will convince a broader range of engineers in academia and the private sector that fish fin-inspired designs can provide improved performance for a variety of applications.

Francois Barthelat does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

This article appeared in The Conversation.

Scientists at the Biorobotics Laboratory (BioRob) in EPFL’s School of Engineering are developing innovative robots in order to study locomotion in animals and, ultimately, gain a better understanding of the neuroscience behind the generation of movement. One such robot is AgnathaX, a swimming robot employed in an international study with researchers from EPFL as well as Tohoku University in Japan, Institut Mines-Télécom Atlantique in Nantes, France, and Université de Sherbrooke in Canada. The study has just been published in Science Robotics.

A long, undulating swimming robot

“Our goal with this robot was to examine how the nervous system processes sensory information so as to produce a given kind of movement,” says Prof. Auke Ijspeert, the head of BioRob and a member of the Rescue Robotics Grand Challenge at NCCR Robotics. “This mechanism is hard to study in living organisms because the different components of the central and peripheral nervous systems* are highly interconnected within the spinal cord. That makes it hard to understand their dynamics and the influence they have on each other.”

AgnathaX is a long, undulating swimming robot designed to mimic a lamprey, which is a primitive eel-like fish. It contains a series of motors that actuate the robot’s ten segments, which replicate the muscles along a lamprey’s body. The robot also has force sensors distributed laterally along its segments that work like the pressure-sensitive cells on a lamprey’s skin and detect the force of the water against the animal.

The research team ran mathematical models with their robot to simulate the different components of the nervous system and better understand its intricate dynamics. “We had AgnathaX swim in a pool equipped with a motion tracking system so that we could measure the robot’s movements,” says Laura Paez, a PhD student at BioRob. “As it swam, we selectively activated and deactivated the central and peripheral inputs and outputs of the nervous system at each segment, so that we could test our hypotheses about the neuroscience involved.”

Two systems working in tandem

The scientists found that both the central and peripheral nervous systems contribute to the generation of robust locomotion. The benefit of having the two systems work in tandem is that it provides increased resilience against neural disruptions, such as failures in the communication between body segments or muted sensing mechanisms. “In other words, by drawing on a combination of central and peripheral components, the robot could resist a larger number of neural disruptions and keep swimming at high speeds, as opposed to robots with only one kind of component,” says Kamilo Melo, a co-author of the study. “We also found that the force sensors in the skin of the robot, along with the physical interactions of the robot’s body and the water, provide useful signals for generating and synchronizing the rhythmic muscle activity necessary for locomotion.” As a result, when the scientists cut communication between the different segments of the robot to simulate a spinal cord lesion, the signals from the pressure sensors measuring the pressure of the water pushing against the robot’s body were enough to maintain its undulating motion.

These findings can be used to design more effective swimming robots for search and rescue missions and environmental monitoring. For instance, the controllers and force sensors developed by the scientists can help such robots navigate through flow perturbations and better withstand damage to their technical components. The study also has ramifications in the field of neuroscience. It confirms that peripheral mechanisms provide an important function which is possibly being overshadowed by the well-known central mechanisms. “These peripheral mechanisms could play an important role in the recovery of motor function after spinal cord injury, because, in principle, no connections between different parts of the spinal cord are needed to maintain a traveling wave along the body,” says Robin Thandiackal, a co-author of the study. “That could explain why some vertebrates are able to retain their locomotor capabilities after a spinal cord lesion.”

Literature

• R. Thandiackal, K. Melo, L. Paez, J. Herault, T. Kano, K. Akiyama, F. Boyer, D. Ryczko, A. Ishiguro, and A. J. Ijspeert. “Emergence of Robust Self-Organized Undulatory Swimming Based on Local Hydrodynamic Force Sensing.” Science Robotics. (2021)

Tree squirrels are the Olympic divers of the rodent world, leaping gracefully among branches and structures high above the ground. And as with human divers, a squirrel’s success in this competition requires both physical strength and mental adaptability.

The Jacobs lab studies cognition in free-ranging fox squirrels on the Berkeley campus. Two species – the eastern gray squirrel (Sciurus carolinensis) and the fox squirrel (Sciurus niger) – thrive on campus landscapes and are willing participants in our behavioral experiments. They are also masters in two- and three-dimensional spatial orientation – using sensory cues to move through space.

In a newly published study, we show that squirrels leap and land without falling by making trade-offs between the distance they have to cover and the springiness of their takeoff perch. This research provides new insights into the roles of decision-making, learning and behavior in challenging environments that we are sharing with researchers of human movement and with engineers. At present, there is no robot as agile as a squirrel, and none that can learn or make decisions about dynamic tasks in complex environments – but our research suggests the kinds of abilities that such robots would need.

Thinking on the go

While a squirrel’s life may look simple to human observers – climb, eat, sleep, repeat – it involves finely tuned cognitive skills. Squirrels are specialized seed dispersers: They harvest their winter’s supply of nuts and acorns during a six- to eight-week span in the fall, bury each nut separately and rely on spatial memory to retrieve them, sometimes months later.

We know that squirrels organize their caches hierarchically. When provided with five nut species in a random order, Berkeley fox squirrels buried nuts in clusters according to species. Because larger nuts contain more calories, squirrels invest more heavily in them, carrying them to safer locations and spacing their hiding places farther apart.

We also discovered that a squirrel assesses the value of a nut by flicking its head with the nut in its mouth, just as a human might bob a pencil in her hand to assess its weight. And we know that they create their cache maps based on factors that include the scarcity of food in that season, the quantity of nuts already cached and the risk of being observed caching by other squirrels.

Along with observational studies, we have also assessed how squirrels perform abstract spatial tasks. For example, we have measured how well they are able to inhibit a lunge toward a remembered food location – part of an international study on the evolution of self control. In another experiment, we put squirrels through a vertical maze that mimicked the branching decisions they face when navigating in trees to see how they return to locations that they remember.

We also have found that while squirrels were solving a tabletop memory puzzle, their cognitive flexibility peaked during the intense period of storing their winter food supply. This explains why Berkeley squirrels are able to switch more easily between types of landmarks during the caching season.

Going airborne

Our new study brought together squirrel psychologists and comparative biomechanists to ask whether squirrels’ cognitive decision-making extends to dynamic changes in locomotion – the famous squirrel leap. How do squirrels’ perceived capabilities of their bodies and their guesses about the stability of the environment shape their decisions about movement?

Robert Full from the PolyPEDAL Laboratory is renowned for studies that extract fundamental design principles through experiments on locomotion in species with unique specializations for movement, from crabs to cockroaches to leaping lizards. Graduate students Nathaniel Hunt, who is trained in biomechanics, and Judy Jinn, trained in animal cognition, took on the challenge of assessing how a leaping squirrel could respond to sudden changes in the location and flexibility of experimental branches.

To study this question in wild squirrels, we designed a magnetic climbing wall that could be mounted on wheels and rolled out to the famous Berkeley Eucalyptus grove to meet the squirrels on their own turf. We brought high-speed cameras and peanuts for persuading squirrels to patiently wait for their turn on the wall.

Our goal was to persuade squirrels to take off from a flexible springboard attached to the climbing wall and jump to a fixed perch protruding from the wall that held a shelled walnut reward. And once again, squirrels surprised us with their acrobatics and innovation.

By increasing the springiness of the springboard and the distance between it and the goal, we could simulate the challenge a squirrel faces as it races through tree branches that vary in size, shape and flexibility. Squirrels leaping across a gap must decide where to take off based on a trade-off between branch flexibility and the size of the gap.

We found that squirrels ran farther along a stiff branch, so they had a shorter, easier jump. In contrast, they took off with just a few steps from flexible branches, risking a longer leap.

Using three branches differing in flexibility, we guessed the position of their takeoff by assuming equal risk for leaping from an unstable branch and jump distance. We were wrong: Our model showed that squirrels cared six times more about a stable takeoff position than how far they had to jump.

Next we had squirrels leap from a very stiff platform. Unbeknownst to the squirrels, we then substituted an identical-looking platform that was three times more flexible. From our high-speed video, we calculated how far away the center of the squirrel’s body was from the landing perch. This allowed us to to determine the landing error – how far the center of the squirrel’s body landed from the goal perch. Squirrels quickly learned to jump from the very bendy branch that they expected to be stiff and could stick the landing in just five tries.

When we raised the ante still further by raising the height and increasing the distance to the goal perch, the squirrels surprised us. They instantly adopted a novel solution: parkour, literally bouncing off the climbing wall to adjust their speed and accomplish a graceful landing. Once more, we discovered the remarkable agility that allows squirrels to evade predators in one of nature’s most challenging environments, the tree canopy.

Millions of people have watched squirrels solve and raid “squirrel-proof” bird feeders, either live in their backyard or in documentaries and viral videos. Like Olympic divers, squirrels must be flexible both physically and cognitively to succeed, making rapid error corrections on the fly and innovating new moves.

With the funding this project attracted, we have joined a team of roboticists, neuroscientists, material scientists and mathematicians to extract design principles from squirrel leaps and landings. Our team is even looking for insights into brain function by studying leap planning in lab rats.

Our analysis of squirrels’ remarkable feats can help us understand how to help humans who have walking or grasping impairments. Moreover, with our interdisciplinary team of biologists and engineers, we are attempting to create new materials for the most intelligent, agile robot ever built – one that can assist in search-and-rescue efforts and rapidly detect catastrophic environmental hazards, such as toxic chemical releases.

A future vision for our efforts? First-responder robotic squirrels, equipped with the physical and cognitive toughness and flexibility of a squirrel at a bird feeder.

Judy Jinn, who participated in this study as a graduate student, is a quantitative UX Researcher at Facebook.

Lucia F. Jacobs receives funding from a Multi-University Research Initiative (MURI) from the Army Research Office (ARO).

Nathaniel Hunt receives funding from the National Institutes of Health.

Robert J. Full receives funding from a Multi-University Research Initiative (MURI) from the Army Research Office (ARO).

This article appeared in The Conversation.

Image: AlesiaKan/Shutterstock.com

By Mischa Dijkstra, Frontiers science writer / Toyoaki Tanikawa, Department of Mechanical Engineering, Osaka University

In the young discipline of robotics-inspired biology, robots replace experimental animals, allowing researchers to learn about animals under a wider range of conditions than exist in nature or the laboratory.

What is the secret behind the steady but oh-so-elegant way in which cats move? That’s the subject of a study in Frontiers in Neurorobotics by scientists from Osaka University, who built a novel, 47cm-long and 7.6kg-heavy robotic cat. Based on previous research on the gait of real domestic cats, the authors deduced that key to the cats’ sleek movement must lie in a previously unknown reflex circuit, which they call the “reciprocal excitatory circuit between hip and knee extensors”.

According to their hypothesis, this reflex circuit has two essential features. The first is the mechanical unloading of a cat’s ankle extensor muscles, which prompts the transition from the stance to the swing phase. The second is muscle movement within the hip joint, which initiates the next transition, from swing to stance.

Trial-and-error with their robotic cat showed that their hypothesis was correct: even without a central pattern generator, a key feature of most moving robots, the novel reflex circuit gave a steady gait to the ‘robocat’, mimicking the movement of real cats.

The study’s first author, Toyoaki Tanikawa, is a young roboticist from Japan. After graduating from the advanced course in industrial and systems engineering at the National Institute of Technology-Kagawa College in early 2019, Tanikawa entered the MSc program at Osaka University’s Department of Mechanical Engineering, where he has done research on walking robots for the past two years.

Here he explains what first brought him to the field of robotics, how he and his supervisors assistant professor Masuda Youichi and Prof Ishikawa Masato developed the new ‘robocat’, and how mutual inspiration and exchange of knowledge between robotics and biology can benefit both fields.

Toyoaki Tanikawa

Because open access and sharing research is part of Frontiers’ mission, we want to give researchers the voice to express themselves and their research with more creativity and freedom than they otherwise would have in publishing an academic paper.

If you’ve recently published your paper with Frontiers and believe you have a great story to tell, then send an email to press@frontiersin.org with ‘Frontier Scientists’ and your name in the subject line.

What inspired you to become a researcher? Do you have any specific memories that set off a spark?

What made me decide to do the research was the robot competition, called RoboCon, in my school days. When I was a student at Kagawa Tech, I was a member of a team participating in that competition. I developed the circuitry and software for some competition robots, and experienced that the performance of a robot can vary greatly depending on its control.

Since then, I have been interested in what kind of control can maximize the performance of a robot. For this reason, in my master’s course, I conducted research on control inspired by animals that can move adaptively in various environments.

Can you tell us about the research you’re currently working on?

We focused on understanding the mechanisms of animal locomotion by developing quadruped robots that can reproduce the neuromuscular dynamics of animals.

Reproducing control structures of animals in robots has two aims. First, to contribute to the well-known field of biology-inspired robotics, where the structure of a robot is made similar to that of an animal. We hope that our results will thus result in fundamental new technology for realizing robots that move efficiently and flexibly like animals.

Second, we aim to contribute to biology in return. At present, experiments using real animals are strictly controlled and restricted, making it difficult to study the inner workings of living, moving animals. In the young research field of robotics-inspired biology, researchers are trying to understand animals by using robots instead of experimental animals. In the future, as more and more robots replace experimental animals, researchers may be able to investigate animal mechanisms under a wider variety of experimental conditions.

In the future, as more and more robots replace experimental animals, researchers may be able to investigate animal mechanisms under a wider variety of experimental conditions.

Toyoaki Tanikawa

In your opinion, why is your research important?

First, in our new study we contribute to the development of novel scientific methods to reproduce and understand quadrupeds using robots, as described above. Another important result is that we have provided a new hypothesis about the locomotion mechanism of cats.

By exploring reflex circuits that could result in a walking quadruped robot, we arrived at a simple novel reflex circuit, which we have named ‘reciprocal excitatory circuit between hip and knee extensors’. By simply reproducing the reciprocal circuit in each leg, the robot generated steady walking motions. We also show that the robot’s gait becomes unstable when the reciprocal circuit is cut off, and that the reciprocal excitatory circuit produced a phenomenon known to occur in cats, called ‘prolongation of the stance phase’. This suggests that the reciprocal excitatory circuit is an important component of the cat’s neural circuit.

Are there any common misconceptions about this area of research? How would you address them?

Robotics is often thought of as a technology that is only for building better robots. However, if used in a different way, robotics can be a very useful scientific tool for reproducing and understanding dynamical systems with complex interactions, such as animals. Of course, not all phenomena discovered in the robot will be observed in animals, but the cycle of discovering new phenomena can be accelerated by providing new hypotheses to biology based on findings from the robot.

What are some of the areas of research you’d like to see tackled in the years ahead?

I graduated from the master’s course this spring and got a job at a company. If I have a chance to do my research in the future, I would like to work on the motion control of robots, which is a further development of our results.

How has open science benefited the reach and impact of your research?

Papers submitted to open access journals have fewer copyright issues. Therefore, I can use various means such as images and videos to advertise the results of my research to the public. Also, since I am leaving the university this spring, being able to read academic papers for free will be a great advantage.

Read Tanikawa et al’s study in Frontiers in Neurorobotics: A Reciprocal Excitatory Reflex Between Extensors Reproduces the Prolongation of Stance Phase in Walking Cats: Analysis on a Robotic Platform.

This article was initially published on the Frontiers blog.

From a software perspective, it is really challenging to come up with an algorithm capable of autonomous and collaborative navigation within such tight resource constraints. State-of-the-art solutions like Simultaneous Localization and Mapping (SLAM) require too much memory and processing power. A promising line of work is to use bug algorithms [2], which can be implemented as computationally efficient finite state machines (FSMs), and can navigate around obstacles without requiring a map.

A downside of using FSMs is that the resulting behavior can be very sensitive to their hyperparameters, and therefore may not generalize outside of the tested environments. This is especially true for the problem of gas source localization (GSL), as wind conditions and obstacle configurations drastically change the problem. In this article, we show how we tackled the complex problem of swarm GSL in cluttered environments by using a simple bug algorithm with evolved parameters, and then tested it onboard a fully autonomous swarm of tiny drones. We will focus on the problems that were encountered along the way, and the design choices we made as a result. At the end of this post, we will also add a short discussion about the future of tiny drones.

Why gas source localization?

Overall we are interested in finding novel ways to enable autonomy on constrained devices, like tiny drones. Two years ago, we showed that a swarm of tiny drones was able to explore unknown, cluttered environments and come back to the base station. Since then, we have been working on an even more complex task: using such a swarm for Gas Source Localization (GSL).

There has been a lot of research focussing on autonomous GSL in robotics, since it is an important but very hard problem [3]. The difficulty of the task comes from the complexity of how odor can spread in an environment. In an empty room without wind, a gas will slowly diffuse from the source. This allows a robot to find the source by simply moving in the direction that makes the gas concentration go up, just like small bacteria like E. Coli do to find nutrients. However, if the environment becomes larger with many obstacles and walls, and wind comes into play, the spreading of gas is much less regular. Large parts of the environment may have no gas or wind at all, while at the same time there may be pockets of gas away from the source. Moreover, chemical sensors for robots are much less capable than the smelling organs of animals. Available chemical sensors for robots are typically less sensitive, noisier, and much slower.

Due to these difficulties, most work in the GSL field has focused on a single robot that has to find a gas source in environments that are relatively small and without obstacles. Relatively recently, there have been studies in which groups of robots solve this task in a collaborative fashion, for example with Particle Swarm Optimization (PSO). PSO was first invented as a way to model the social behavior of foraging birds, in which the birds communicate with each other how good the food at their location is. The birds then follow a direction that is determined both by their own observations and the best observed location of the swarm. PSO turned out to be a great optimization algorithm for many different real-world problems. Thanks to the different particles in the swarm, it can escape local optima. In the case of GSL, PSO allows a swarm of robots to collaboratively seek a gas source, while ignoring pockets of gas away from the gas source. Until now this concept has been shown in simulation [4] and on large outdoor drones equipped with LiDAR and GPS [5], but never before on tiny drones in complex, GPS-denied, indoor environments.

Required Infrastructure

In our project, we introduce a new bug algorithm, Sniffy Bug, which uses PSO for gas source localization. In order to tune the FSM of Sniffy Bug, we used an evolutionary algorithm. This type of algorithm mimics the survival of the fittest in natural evolution, but now with “fitness” defined as being able to localize a gas source as efficiently as possible. The evolution starts out with random controllers for the swarm of robots, evaluates them in simulation, and selects the fittest controllers for reproduction. Over the generations, the controllers become increasingly good at the task. After evolution, the best controller is transferred to the real robots in the swarm.

Of course, for this evolution to work well, we need a good simulation of how gas spreads in a complex, indoor environment. However, early in the project, we realized that this would be a challenge, as no end-to-end gas modeling pipeline existed yet. It is important to have an easy-to-use pipeline that does not require any aerodynamics domain knowledge, such that as many researchers as possible can generate environments to test their algorithms. It would also make it easier to compare contributions and to better understand in which conditions certain algorithms work or don’t work. The GADEN ROS package [6] is a great open source tool for modelling gas distribution when you have an environment and flow field, but for our objective, we needed a fully automated tool that could generate a great variety of random environments on-demand with just a few parameters. Below is an overview of our simulation pipeline: AutoGDM.

AutoGDM, a fully automated gas dispersion modeling (GDM) simulation pipeline.

First, we use a procedural environment generator proposed in [7] to generate random walls and obstacles inside of the environment. Then, gas dispersion modeling (GDM) is done by first modeling a 3D flow field, i.e., the direction and magnitude of wind velocity at every point in space. Next, a gas source location is chosen. Then, at the source, gas filaments are released in the flow field and randomly expanded over time. This results in a time-varying gas concentration field. The 3D flow field is heavily impacted by something referred to as “boundary conditions”: the conditions at the walls in the environment that we feed into the computational fluid dynamics (CFD) solver. In our case, this means for some walls we tell the CFD to force a wind velocity of 0 m/s, while for others we may tell it to model an inlet or an outlet of air, like an open window. Determining the boundary conditions is done automatically by AutoGDM.

Since a hard requirement for us was that AutoGDM needed to be free to use, we chose to use the open-source CFD tool OpenFOAM. It’s used for cutting-edge aerodynamics research, and also the tool suggested by the authors of GADEN. Without AutoGDM, using OpenFOAM isn’t trivial, as a large number of parameters that require field expertise need to be selected, resulting in a complicated process. GADEN was used to take the environment definition (CAD files) and the flow field from OpenFOAM to generate the gas concentration field over time.

After we built this pipeline, we still needed a robot simulator. Since we weren’t planning on using a camera, our main requirement was for the simulator to be efficient (preferably in 2D) so that evolutions would take relatively little time. We decided to use Swarmulator [8], a computationally efficient C++ robot simulator designed for swarming and we plugged in our gas data.

Algorithm Design

Roughly speaking, we considered two categories of algorithms for controlling the drones: 1) a neural network, and 2) an FSM that included PSO, with evolved parameters. We first evolved neural networks in simulation. One of the first experiments is shown below.

A single agent in simulation seeking a gas source using a tiny neural network.

While it worked pretty well in simple environments with few obstacles, it seemed challenging to make this work in real life with complex obstacles and multiple agents that need to collaborate. Given the time constraints of the project, we opted for evolving the FSM. This also facilitated crossing the reality gap (i.e., the difference between simulation and real-world behaviour), as the simulated evolution could build on basic behaviors that we developed and validated on the real platform, including obstacle avoidance with four tiny laser rangers, while communicating with and avoiding other drones. An additional advantage of PSO with respect to the reality gap is that it only needs gas concentration and no gradient of the gas concentration or wind direction (which many algorithms in literature use). On a real robot at this scale, estimating the gas concentration gradient or the direction of a light breeze is hard if not impossible.

Hardware

We deploy a 37.5g Bitcraze CrazyFlie nano drone that is capable of avoiding obstacles, executing velocity commands, sensing gas, and estimating the other agent’s position in its own frame. For navigation we added a down-facing optic flow sensor and four laser rangers, whereas for gas sensing we used a TGS8100 gas sensor that was used on a CrazyFlie before in previous work [9]. The sensor is lightweight and inexpensive, but accurately estimating gas concentrations can be difficult because of its size. It tends to drift and needs time to recover after a spike in concentration is observed. Another thing we noticed is that it is possible to break them, a crash can definitely destroy the sensor.

To estimate the relative position between agents, we use a Decawave Ultra-Wideband (UWB) module and communicate states, as proposed in [10]. We also use the UWB module to communicate gas information between agents and collaboratively seek the source. The complete configuration is visible below.

A 37.5 g nano quadcopter, capable of fully autonomous waypoint tracking, obstacle avoidance, relative localization, communication and gas sensing.

Evaluation in Simulation

After we optimized the parameters of our model using Swarmulator and AutoGDM, and of course trying many different versions of our algorithm, we ended up with the final Sniffy Bug algorithm. Below is a video that shows evolved Sniffy Bug evaluated in six different environments.

The red dots are an agent’s personal target waypoint, whereas the yellow dot is the best-known position for the swarm. Simulation shows that Sniffy Bug is effective at locating the gas source in randomly generated environments. The drones successfully collaborate by means of PSO.

Real Flight Testing

After observing Sniffy Bug in simulation we were optimistic, but unsure about performance in real life. First, inspired by previous works, we dispersed alcohol through the air by placing liquid alcohol into a can which was then dispersed using a computer fan.

Dispersion of liquid alcohol in flight tests.

We tested Sniffy Bug in our flight arena of size 10 x 10 meters with large obstacles that were shaped like walls and orange poles. The image below shows four flight tests of Sniffy Bug in cluttered environments, flying fully autonomously, i.e., without the help from any external infrastructure.

Time-lapse images of real-world experiments in our flight arena. Sniffy was evaluated on four distinct environments, 10 x 10 meters in size, seeking a real isopropyl alcohol source. The trajectories of the nano quadcopters are clearly visible due to their blue lights.

In the total of 24 runs we executed, we compared Sniffy Bug with manually selected and evolved parameters. The figure below shows that the evolved parameters are more efficient in locating the source as compared to the manual parameters.

Maximum recorded gas reading by the swarm, for each time step for each run.

This does not only show that our system can successfully locate a gas source in challenging environments, but it also demonstrates the usefulness of the simulation pipeline. The parameters that were learned in simulation yield a high-performance model, validating the environment generation, randomization, and gas modeling parts of our pipeline.

Conclusion and Discussion

With this work, we believe we have made an important step towards swarms of gas-seeking drones. The proposed solution has been shown to work in real flight tests with obstacles, and without any external systems to help in localization or communication. We believe this methodology can be extended to larger environments or even to 3 dimensions, since PSO is a robust, multi-dimensional heuristic search method. Moreover, we hope that AutoGDM will help the community to better compare gas seeking algorithms, and to more easily learn parameters or models in simulation, and deploy them in the real world.

To improve Sniffy Bug’s performance, adding more laser rangers will definitely help. When working with only four laser rangers you realize how little information they actually provide. If one of the rangers senses a low value, it is unclear if a slim pole or a massive wall is detected, adding inefficiency to the algorithm. Adding more laser rangers or using other sensor modalities like vision will help to avoid also more complex obstacles than walls and poles in a reliable manner.

Another interesting discussion can be held on the hardware required for real deployment. When working with 40 grams of maximum take-off weight, the sensors and actuators that can be selected are limited. For example, the low-power and lightweight flow deck works great but fails in low-light scenarios or with smoke. Future work exploring novel sensors for highly constrained nano robots could really help increase the Technological Readiness Level (TRL) of these systems.

Finally, this has been a really fun project to work on for us and we can’t wait to hear your thoughts on Sniffy Bug!

References

[1] Carrillo-Zapata D, Milner E, Hird J, et al. Mutual Shaping in Swarm Robotics: User Studies in Fire and Rescue, Storage Organization, and Bridge Inspection. Front Robot AI. 2020;7:53. Published 2020 Apr 21. doi:10.3389/frobt.2020.00053

[2] K. N. McGuire, C. De Wagter, K. Tuyls, H. J. Kappen, and G. C. H. E.de Croon, “Minimal navigation solution for a swarm of tiny flying robotsto explore an unknown environment,”Science Robotics, vol. 4, no. 35,2019.

[3] Jing T, Meng Q-H, Ishida H. Recent progress and trend of robot odor source localization. IEEJ trans electr electron eng. 2021;16(7):938-953

[4] W. Jatmiko, K. Sekiyama and T. Fukuda, “A pso-based mobile robot for odor source localization in dynamic advection-diffusion with obstacles environment: theory, simulation and measurement,” in IEEE Computational Intelligence Magazine, vol. 2, no. 2, pp. 37-51, May 2007, doi: 10.1109/MCI.2007.353419.

[5] Steiner, JA, Bourne, JR, He, X, Cropek, DM, & Leang, KK. “Chemical-Source Localization Using a Swarm of Decentralized Unmanned Aerial Vehicles for Urban/Suburban Environments.” Proceedings of the ASME 2019 Dynamic Systems and Control Conference. Volume 3, Park City, Utah, USA. October 8–11, 2019. V003T21A006. ASME. https://doi.org/10.1115/DSCC2019-9099

[6] Monroy, V. Hernandez-Bennetts, H. Fan, A. Lilienthal, andJ. Gonzalez-Jimenez, “Gaden: A 3D gas dispersion simulator for mobilerobot olfaction in realistic environments,”MDPI Sensors, vol. 17, no.7: 1479, pp. 1–16, 2017.

[7] K. McGuire, G. de Croon, and K. Tuyls, “A comparative study of bug algorithms for robot navigation,”Robotics and Autonomous Systems, vol.121, p. 103261, 2019.

[8] https://github.com/coppolam/swarmulator

[9] J. Burgues, V. Hern ́andez, A. J. Lilienthal, and S. Marco, “Smelling nano aerial vehicle for gas source localization and mapping, ”Sensors(Switzerland), vol. 19, no. 3, 2019.

[10] S. Li, M. Coppola, C. D. Wagter, and G. C. H. E. de Croon, “An autonomous swarm of micro flying robots with range-based relative localization,” Arxiv, 2020.

Links

ArXiv: https://arxiv.org/abs/2107.05490

Video: https://www.youtube.com/watch?v=hj_SBSpK5qg

Code: https://github.com/tudelft/sniffy-bug

Please reach out if you have any questions or ideas, you can reach us at: b.p.duisterhof@gmail.com or g.c.h.e.decroon@tudelft.nl

Enrica Soria, a PhD student at LIS © Alain Herzog / 2021 EPFL

By Clara Marc

There is strength in numbers. That’s true not only for humans, but for drones too. By flying in a swarm, they can cover larger areas and collect a wider range of data, since each drone can be equipped with different sensors.

Preventing drones from bumping into each other
One reason why drone swarms haven’t been used more widely is the risk of gridlock within the swarm. Studies on the collective movement of animals show that each agent tends to coordinate its movements with the others, adjusting its trajectory so as to keep a safe inter-agent distance or to travel in alignment, for example.

“In a drone swarm, when one drone changes its trajectory to avoid an obstacle, its neighbors automatically synchronize their movements accordingly,” says Dario Floreano, a professor at EPFL’s School of Engineering and head of the Laboratory of Intelligent Systems (LIS). “But that often causes the swarm to slow down, generates gridlock within the swarm or even leads to collisions.”

Not just reacting, but also predicting
Enrica Soria, a PhD student at LIS, has come up with a new method for getting around that problem. She has developed a predictive control model that allows drones to not just react to others in a swarm, but also to anticipate their own movements and predict those of their neighbors. “Our model gives drones the ability to determine when a neighbor is about to slow down, meaning the slowdown has less of an effect on their own flight,” says Soria. The model works by programing in locally controlled, simple rules, such as a minimum inter-agent distance to maintain, a set velocity to keep, or a specific direction to follow. Soria’s work has just been published in Nature Machine Intelligence.

With Soria’s model, drones are much less dependent on commands issued by a central computer. Drones in aerial light shows, for example, get their instructions from a computer that calculates each one’s trajectory to avoid a collision. “But with our model, drones are commanded using local information and can modify their trajectories autonomously,” says Soria.

A model inspired by nature
Tests run at LIS show that Soria’s system improves the speed, order and safety of drone swarms in areas with a lot of obstacles. “We don’t yet know if, or to what extent, animals are able to predict the movements of those around them,” says Floreano. “But biologists have recently suggested that the synchronized direction changes observed in some large groups would require a more sophisticated cognitive ability than what has been believed until now.”

References

• PAPER – Predictive control of aerial swarms in cluttered environments. Soria, E., Schiano, F. and Floreano, D., Nat Mach Intell (2021). DOI: 10.1038/s42256-021-00341-y

The World Bank estimates that 3.5 million tons of solid waste is produced by humans everyday, with America accounting for more than 250 million tons a year or over 4 pounds of trash per citizen. This figure does not include the 34 billion gallons of human organic materials that is processed in water treatment centers across the country each year. To the fictional Dr. Emmett Brown, this garbage is akin to “black gold” – ecologically powering cities, cars, and machines. In reality, the movie, “Back to The Future II” was inspired by the biomass gasification movement of 20th century in powering cars with wood during World War II when petroleum was scarce. The technology has advanced so much that a few years ago the GENeco water treatment plant in the United Kingdom built a biomethane gas bus that relied solely on sewage. In reflecting on the importance of the technology, Collin Field of Bath Bus Company declared, “We will never, ever, ever, while we are on this planet, run out of human waste.”

Less than twenty miles away from the GENco plant, Professor Loannis Leropoulos of the University of Bristol’s Robotics Laboratory is working on the next generation of bio-engineered fuel cells. Last week, Dr. Leropoulos demonstrated his revolutionary Microbial Fuel Cells (MFCs) for me. As witnessed, he is not just inspired by nature, but harnessing its beauty to power the next generation of robots. The MFCs mimic an animal’s stomach with microbes breaking down food to create adenosine triphosphate (ATP). The Bristol lab began building MFCs to power its suite of EcoBots. “I started this journey about twenty years ago with the main purpose of building sustainable autonomous robots for remote area access,” reflects Leropoulos. He was inspired by Dr. Stuart Wilkinson’s Gastrobot in the early 2000s that first promoted the idea of “an intelligent machine that digests real food for energy.” At the time the media hyped Wilkinson’s invention as a flesh-eating robot, when in reality it digested sugar cubes, turning the carbohydrates into electrical energy. Unfortunately, the Gastrobot’s clumsy oversized form factor suffered from long charging times with an 18-hour “carbo-loading” process for every 15 minutes of power.

Springboarding off of Wilkinson’s concept, Leropoulos’ team started with the idea of using MFC to power machines by putting the microbes directly inside the unit to more efficiently produce energy from any sugar-based substance, even waste (e.g., urine, feces, and trash). The Professor described the elegance of his technology that creates a “uniform colonization” of microbes multiplying every 8 minutes with parent lifecycles succeeded by daughter cells in a continuous pattern of ‘feed-growth-energy’. He compares it to the human microflora process of breaking down fresh food in the digestive system that results in healthy bathroom visits, “the same is with Microbial Fuel Cells as long as we continue feeding them, the MFCs will continue to generate electricity.” The Bristol professor boasts that batteries are better performing than anything on the market as biological lifeforms have no denigration since “the progeny keep refreshing the community or electrodes so we have stable levels of power.” By contrast, the most popular non-fossil fuel available, lithium, degrades over time and leads to destructive mining practices in scarring the Earth in search of declining ore resources with the explosion of mobile phones, portable devices, and electric vehicles.

While MFCs are still in their infancy, Leropoulos shared with me his plan for commercializing the invention. His lab recently announced the success of its MFCs prototypes in powering mobile phones, smart watches, and other devices (including the EcoBots). In addition, Leropoulos has pushed his team to miniaturize the size of his batteries from its 6″ prototype to smaller than a AA, while at the same time rivaling the performance of alkaline. Backed by the Gates foundation, he has also reduced the production costs from $18 to $1 a unit, this is before achieving the economies of scale with mass production. Today, his business plan has expanded beyond just autonomous robots to power smart homes, by connecting multiple MFCs to a house’s sanitation and waste systems. “Our research is all about optimizing miniaturization and stacking them with minimum losses so we can end up with a car battery-like shape and size that gives us the amount of power we require,” explains the professor. When I questioned Leropoulos about using MFCs in the future of autonomous fleets, and even to offset the high energy demands of something like bitcoin mining, he remarked, “It would be naive of me to say a straight yes, but this is of course the work we are doing. I strongly believe with the development of new materials that will help with the energy density. We are at a stage where we have done some groundbreaking research using 3D printed electric materials as a low cost scalable technology. There is a lot to say about the functionalization of the electrodes that enables colonization from the microbes in making sure all the progeny cells colonize and the ceramic separators that allows for target ion transfer that makes the whole operation smarter and more efficient.” In thinking more about his work, he declared, “We have yet to see the full potential of Microbial Fuel Cells. I do think one day we will have a ‘Back to the Future’ scenario, feeding your food scraps to your car,”

Pressing Leropoulos on how he envisions robot-charging stations working in a factory or home in the near future, he illustrated it best, “with a Roomba example its actually picking of food scraps in the kitchen that would be a very nutritious source of fuel for the Microbial Fuel Cell, but that’s a few steps down the line.” He continued, “a straightforward application for something like a Roomba is to leave the charging station where it is and connect it to the toilet or kitchen sink. The fuel cycle would be continuous as the robot would not be drawing energy from the house, but the wastewater.” Processing the impact of his vision, it took me back to the early days of the global pandemic shutdown with animals returning to ancient grazing areas and pollution clouds clearing over heavily populated areas with many seeing for the first time distant mountains and blue skies. Innovations like MFCs are part of a new wave of mechatronic environmentally-focused solutions. Before parting, I asked Leropoulos how hopeful is he about the environment, “I do feel optimistic, I have more faith in the the younger generation that they will do things better. The shock we sustained as a society [this past year] is a lesson in seeing the true color of our natural world, if we learn from this then I think the future is a good one.”

The deepest regions of the oceans still remain one of the least explored areas on Earth, despite their considerable scientific interest and the richness of lifeforms inhabiting them.

Two reasons for this are the low temperatures and enormous pressures exerted at such depths, which require the exploration equipment be carefully shielded inside high-strength metal or ceramic chambers to withstand them. This makes deep-sea exploration vessels bulky, expensive and unwieldy, as well as difficult to design, manufacture and transport.

But a new small self-powered underwater robotic fish appears to offer an alternative. According to a recent paper, the robot was able to reach the deepest part of the Pacific Ocean – the Mariana Trench – at a depth of almost 11 km (6.8 miles).

The pressure there is more than a thousand times that on the surface of the sea. Yet various animals, including fish, are able to withstand this staggering pressure and have adapted to life in such adverse conditions. The morphology and skull structure of one of these marine organisms, the hadal snailfish, reportedly inspired the design of this remarkable robot swimmer.

The main breakthrough that enabled this significant achievement was a specially-designed compliant polymer body which deforms, without breaking, under high pressure. The team of researchers from Hangzhou in China were able to embed the delicate electronic components required for power, movement and control in a protective silicone matrix.

The electronic components were separated from each other, instead of being tightly packed together as is the usual practice, to make them more resilient to the pressure, similar to the skull bones of the snailfish.

The robot also looks like the snailfish, with an elongated body and tail, as well as two large side fins made of thin silicone. The fins flap to propel the small robot, which measures only 22cm (8.7 inches) in length with a wingspan of 28cm.

Read more:
Curious Kids: how do creatures living in the deep sea stay alive given the pressure?

The fins are not operated by rigid motors, but by soft artificial muscles. The muscles contain dielectric elastomers – a class of smart materials which contract when electrical voltage is applied to create large movements.

Disk-shaped dielectric elastomer elements create the fin flapping motion that propels the robot, reaching speeds of up to about half a body-length per second (around 0.2 km/h), even at significant depths.

However, this type of actuators – the parts that make a machine move – requires very high voltage. A compact high-voltage amplifier multiplies the lithium-ion battery voltage more than a thousand times to meet this requirement, while an infrared receiver allows remote control of the robot. The soft fins and soft actuators were carefully designed to survive and perform well at the low temperatures and high pressures of the deep-sea environment.

The team performed extensive computational studies and laboratory testing of the propulsion methodology and of how the electronics cope under extreme pressures. Then, they conducted free-swimming field tests, first in a deep lake, then at the South China Sea at depths of more than 3km, before deploying it in the Mariana Trench.

In the Mariana field tests, the robot was mounted on a deep-sea lander, so wasn’t allowed to swim freely. But, it was able to maintain its flapping motion, as recorded by the cameras of the lander, for 45 minutes at a depth of 10,900 metres.

Soft robots

This deep-sea swimmer is an example of a new generation of robots inspired by living organisms, both animals and plants. They are built exploiting the advantages of compliant materials like silicone and other polymers, gels or even textiles.

These robots can bend, yield and adapt in response to forces from their environment, so are inherently safer to work next to humans compared to the typical rigid industrial robots. On the other hand, their design, actuation, sensing and control can pose significant challenges, which lie at the core of their scientific and technological interest.

There is currently intense interdisciplinary research activity in this new area, called soft robotics, leading to exciting innovative advances for a host of related applications, ranging from agriculture to medicine and space. The Harvard Octobot is an example of this class of robots, which appears to have been, among others, a source of inspiration for the design and the technologies employed in this deep-sea robot.

The current version of the deep-sea swimmer appears to be relatively slow, not very easy to manoeuvre, and possibly not able to withstand the strong underwater currents which would disturb its course while attempting to follow a desired path. However, its designers already seem to have plans for further improvements that will make it more manoeuvrable, more efficient and smarter.

Despite any shortcomings, we should not underestimate the robotic design principles and technological advances that led to such a dramatic demonstration.

Dimitris Tsakiris has received funding from HEFCW, CRUK, and EC/EU.

Original post published in The Conversation.

Insects’ remarkable acrobatic traits help them navigate the aerial world, with all of its wind gusts, obstacles, and general uncertainty. Image: courtesy of Kevin Yufeng Chen

By Daniel Ackerman

If you’ve ever swatted a mosquito away from your face, only to have it return again (and again and again), you know that insects can be remarkably acrobatic and resilient in flight. Those traits help them navigate the aerial world, with all of its wind gusts, obstacles, and general uncertainty. Such traits are also hard to build into flying robots, but MIT Assistant Professor Kevin Yufeng Chen has built a system that approaches insects’ agility.

Chen, a member of the Department of Electrical Engineering and Computer Science and the Research Laboratory of Electronics, has developed insect-sized drones with unprecedented dexterity and resilience. The aerial robots are powered by a new class of soft actuator, which allows them to withstand the physical travails of real-world flight. Chen hopes the robots could one day aid humans by pollinating crops or performing machinery inspections in cramped spaces.

Chen’s work appears this month in the journal IEEE Transactions on Robotics. His co-authors include MIT PhD student Zhijian Ren, Harvard University PhD student Siyi Xu, and City University of Hong Kong roboticist Pakpong Chirarattananon.

Typically, drones require wide open spaces because they’re neither nimble enough to navigate confined spaces nor robust enough to withstand collisions in a crowd. “If we look at most drones today, they’re usually quite big,” says Chen. “Most of their applications involve flying outdoors. The question is: Can you create insect-scale robots that can move around in very complex, cluttered spaces?”

According to Chen, “The challenge of building small aerial robots is immense.” Pint-sized drones require a fundamentally different construction from larger ones. Large drones are usually powered by motors, but motors lose efficiency as you shrink them. So, Chen says, for insect-like robots “you need to look for alternatives.”

The principal alternative until now has been employing a small, rigid actuator built from piezoelectric ceramic materials. While piezoelectric ceramics allowed the first generation of tiny robots to take flight, they’re quite fragile. And that’s a problem when you’re building a robot to mimic an insect — foraging bumblebees endure a collision about once every second.

Chen designed a more resilient tiny drone using soft actuators instead of hard, fragile ones. The soft actuators are made of thin rubber cylinders coated in carbon nanotubes. When voltage is applied to the carbon nanotubes, they produce an electrostatic force that squeezes and elongates the rubber cylinder. Repeated elongation and contraction causes the drone’s wings to beat — fast.

Chen’s actuators can flap nearly 500 times per second, giving the drone insect-like resilience. “You can hit it when it’s flying, and it can recover,” says Chen. “It can also do aggressive maneuvers like somersaults in the air.” And it weighs in at just 0.6 grams, approximately the mass of a large bumble bee. The drone looks a bit like a tiny cassette tape with wings, though Chen is working on a new prototype shaped like a dragonfly.

“Achieving flight with a centimeter-scale robot is always an impressive feat,” says Farrell Helbling, an assistant professor of electrical and computer engineering at Cornell University, who was not involved in the research. “Because of the soft actuators’ inherent compliance, the robot can safely run into obstacles without greatly inhibiting flight. This feature is well-suited for flight in cluttered, dynamic environments and could be very useful for any number of real-world applications.”

Helbling adds that a key step toward those applications will be untethering the robots from a wired power source, which is currently required by the actuators’ high operating voltage. “I’m excited to see how the authors will reduce operating voltage so that they may one day be able to achieve untethered flight in real-world environments.”

Building insect-like robots can provide a window into the biology and physics of insect flight, a longstanding avenue of inquiry for researchers. Chen’s work addresses these questions through a kind of reverse engineering. “If you want to learn how insects fly, it is very instructive to build a scale robot model,” he says. “You can perturb a few things and see how it affects the kinematics or how the fluid forces change. That will help you understand how those things fly.” But Chen aims to do more than add to entomology textbooks. His drones can also be useful in industry and agriculture.

Chen says his mini-aerialists could navigate complex machinery to ensure safety and functionality. “Think about the inspection of a turbine engine. You’d want a drone to move around [an enclosed space] with a small camera to check for cracks on the turbine plates.”

Other potential applications include artificial pollination of crops or completing search-and-rescue missions following a disaster. “All those things can be very challenging for existing large-scale robots,” says Chen. Sometimes, bigger isn’t better.

Most of the ocean is unknown. Yet we know that the most challenging environments on the planet reside in it. Understanding the ocean in its totality is a key component for the sustainable development of human activities and for the mitigation of climate change, as proclaimed by the United Nations. We are glad to share our perspective about the role of soft robots in ocean exploration and offshore operations at the outset of the ocean decade (2021-2030).

In this study of the Soft Systems Group (part of The School of Engineering at The University of Edinburgh), we focus on the two ends of the water column: the abyss and the surface. The former is mostly unexplored, containing unknown physical, biological and chemical properties; the latter is where industrial offshore activities take place, and where human operators face dangerous environmental conditions.

The analysis of recent developments in soft robotics brought to light their potential in solving some of the challenges that industry and scientists are facing at sea. The paper offers a discussion about how we can use the latest technological advances in soft robotics to overcome the limitations of existing technology. We synthesise the crucial characteristics that future marine robots should include.

Today we know that industrial growth needs to be supported by deep environmental and technological knowledge in order to develop human activities in a sustainable manner. Therefore, the remotest areas of the ocean constitute a common frontier for oceanography, robotics and offshore industry. The first challenge then is to create an interdisciplinary space where marine scientists, ocean engineers, roboticists and industry can communicate.

The offshore renewable energy sector is growing fast. By definition, wind and wave energy is better sourced from areas of the seas with a high energy content, which translates into wave dominated environments. In such conditions, operating in the vicinity of an offshore platform is dangerous for personnel and potentially destructive for traditional robots.

Recent development in soft robotics unveiled novel, bio-inspired maneuvering techniques, fluidic logic data logging capabilities and unprecedented dexterity. These characteristics would enable data collection, maintenance and repair interventions, unfeasible with rigid robots.

Traditional devices, often tethered, rigid and with limited activity range, cannot tackle the environmental conditions where ocean exploration is needed the most. The devices needed for ocean exploration require an innovative payload and delicate sampling capabilities. Soft sensors and soft gripping techniques are optimum candidates to push ocean exploration into fragile and unknown areas that require delicate sampling and gentle navigation. The inherent compliance of soft robots also protects the environment and the payload from damage. More importantly, recent studies have highlighted the potential for the manufacturing of soft robots to be entirely biocompatible and, hence, to minimise the impact on the natural environment in case of loss or mass deployment.

In a nutshell, soft robots for marine exploration and offshore deployment offer the advantage, over traditional devices, of novel navigation and manipulation techniques, unprecedented sensing and sampling capabilities, biocompatibility and novel memory and data logging methods. With this study we wish to invite a multidisciplinary approach in order to create novel sustainable soft systems.

• PUBLICATION – Soft Robots for Ocean Exploration and Offshore Operations: A Perspective. Simona Aracri, Francesco Giorgio-Serchi, Giuseppe Suaria, Mohammed E. Sayed, Markus P. Nemitz, Stephen Mahon and Adam A. Stokes. Soft Robotics (2021).

Bluebots are fish-shaped robots that can coordinate their movements in three dimensions underwater, rather than the two dimensions previously achieved by Kilobots. Credit: Harvard SEAS

By Leah Burrows / SEAS Communications

Schools of fish exhibit complex, synchronized behaviors that help them find food, migrate, and evade predators. No one fish or sub-group of fish coordinates these movements, nor do fish communicate with each other about what to do next. Rather, these collective behaviors emerge from so-called implicit coordination — individual fish making decisions based on what they see their neighbors doing.

This type of decentralized, autonomous self-organization and coordination has long fascinated scientists, especially in the field of robotics.

Now, a team of researchers at Harvard’s Wyss Institute and John A. Paulson School of Engineering and Applied Sciences (SEAS) have developed fish-inspired robots that can synchronize their movements like a real school of fish, without any external control. It is the first time researchers have demonstrated complex 3D collective behaviors with implicit coordination in underwater robots.

“Robots are often deployed in areas that are inaccessible or dangerous to humans, areas where human intervention might not even be possible,” said Florian Berlinger, a Ph.D. Candidate at the Wyss Institute and SEAS and first author of the paper. “In these situations, it really benefits you to have a highly autonomous robot swarm that is self-sufficient. By using implicit rules and 3D visual perception, we were able to create a system that has a high degree of autonomy and flexibility underwater where things like GPS and WiFi are not accessible.”

The research is published in Science Robotics.

The fish-inspired robotic swarm, dubbed Blueswarm, was created in the lab of Wyss Associate Faculty member Radhika Nagpal, Ph.D., who is also the Fred Kavli Professor of Computer Science at SEAS. Nagpal’s lab is a pioneer in self-organizing systems, from their 1,000 robot Kilobot swarm to their termite-inspired robotic construction crew.

However, most previous robotic swarms operated in two-dimensional space. Three-dimensional spaces, like air and water, pose significant challenges to sensing and locomotion.

To overcome these challenges, the researchers developed a vision-based coordination system in their fish robots based on blue LED lights. Each underwater robot, called a Bluebot, is equipped with two cameras and three LED lights. The on-board, fisheye-lens cameras detect the LEDs of neighboring Bluebots and use a custom algorithm to determine their distance, direction and heading. Based on the simple production and detection of LED light, the researchers demonstrated that the Blueswarm could exhibit complex self-organized behaviors, including aggregation, dispersion, and circle formation.

These fish-inspired robots can synchronize their movements without any outside control. Based on the simple production and detection of LED light, the robotic collective exhibits complex self-organized behaviors, including aggregation, dispersion, and circle formation. Credit: Harvard University’s Self-organizing Systems Research Group

“Each Bluebot implicitly reacts to its neighbors’ positions,” said Berlinger. “So, if we want the robots to aggregate, then each Bluebot will calculate the position of each of its neighbors and move towards the center. If we want the robots to disperse, the Bluebots do the opposite. If we want them to swim as a school in a circle, they are programmed to follow lights directly in front of them in a clockwise direction.”

The researchers also simulated a simple search mission with a red light in the tank. Using the dispersion algorithm, the Bluebots spread out across the tank until one comes close enough to the light source to detect it. Once the robot detects the light, its LEDs begin to flash, which triggers the aggregation algorithm in the rest of the school. From there, all the Bluebots aggregate around the signaling robot.

Blueswarm, a Harvard Wyss- and SEAS-developed underwater robot collective, uses a 3D vision-based coordination system and 3D locomotion to coordinate the movements of its individual Bluebots autonomously, mimicking the behavior of schools of fish. Credit: Harvard SEAS

“Our results with Blueswarm represent a significant milestone in the investigation of underwater self-organized collective behaviors,” said Nagpal. “Insights from this research will help us develop future miniature underwater swarms that can perform environmental monitoring and search in visually-rich but fragile environments like coral reefs. This research also paves a way to better understand fish schools, by synthetically recreating their behavior.”

The research was co-authored by Melvin Gauci, Ph.D., a former Wyss Technology Development Fellow. It was supported in part by the Office of Naval Research, the Wyss Institute for Biologically Inspired Engineering, and an Amazon AWS Research Award.

• PUBLICATION – Implicit coordination for 3D underwater collective behaviors in a fish-inspired robot swarm. Florian Berlinger, Melvin Gauci and Radhika Nagpal. Science Robotics, 6(50).

Interesting discussion with Prof. Ali Khademhosseini, CEO of the Terasaki Institute, and one of the pioneers of the Bioengineering field. Prof. Ali’s journey from Harvard and UCLA to the Terasaki Institute is truly inspiring. What does the institute do to bring a product to the real world? Learn about the design challenges of biomaterials, organs on a chip, and soft robotics in this episode of the IEEE RAS Soft Robotics Podcast.

You can also listen to this podcast on Spotify and Apple Podcasts, and explore support options on Patreon.

How do honeybees land on flowers or avoid obstacles? One would expect such questions to be mostly of interest to biologists. However, the rise of small electronics and robotic systems has also made them relevant to robotics and Artificial Intelligence (AI). For example, small flying robots are extremely restricted in terms of the sensors and processing that they can carry onboard. If these robots are to be as autonomous as the much larger self-driving cars, they will have to use an extremely efficient type of artificial intelligence – similar to the highly developed intelligence possessed by flying insects.

Optical flow

One of the main tricks up the insect’s sleeve is the extensive use of ‘optical flow’: the way in which objects move in their view. They use it to land on flowers and avoid obstacles or predators. Insects use surprisingly simple and elegant optical flow strategies to tackle complex tasks. For example, for landing, honeybees keep the optical flow divergence (how quickly things get bigger in view) constant when going down. By following this simple rule, they automatically make smooth, soft landings.

I started my work on optical flow control from enthusiasm about such elegant, simple strategies. However, developing the control methods to actually implement these strategies in flying robots turned out to be far from trivial. For example, when I first worked on optical flow landing my flying robots would not actually land, but they started to oscillate, continuously going up and down, just above the landing surface.

Honeybees are a fertile source of inspiration for the AI of small drones. They are able to perform an impressive repertoire of complex behaviors with very limited processing (~960,000 neurons). Drones are in their turn very interesting “models” for biology. Testing out hypotheses from biology on drones can bring novel insights into the problems faced and solved by flying insects like honeybees.

Fundamental problems

Optical flow has two fundamental problems that have been widely described in the growing literature on bio-inspired robotics. The first problem is that optical flow only provides mixed information on distances and velocities – and not on distance or velocity separately. To illustrate, if there are two landing drones and one of them flies twice as high and twice as fast as the other drone, then they experience exactly the same optical flow. However, for good control these two drones should actually react differently to deviations in the optical flow divergence. If a drone does not adapt its reactions to the height when landing, it will never arrive and start to oscillate above the landing surface.

The second problem is that optical flow is very small and little informative in the direction in which a robot is moving. This is very unfortunate for obstacle avoidance, because it means that the obstacles straight ahead of the robot are the hardest ones to detect! The problems are illustrated in the figures below.

Left: Problem 1: The white drone is twice as high and goes down twice as fast as the red drone. However, they both see the same optical flow divergence, as in both cases the object in view gets twice as big. This can be seen by the colored triangles – at the highest position it captures the full angle of the landing platform, at the lowest position it only covers half of the landing platform in the field of view.
Right: Problem 2: The drone moves straight forward, in the direction of its forward-looking camera. Hence, the focus of expansion is straight ahead. Objects close to this direction, like the red obstacle, have very little flow. This is illustrated by the red lines in the figure: The angles of these lines with respect to the camera are very similar. Objects further from this direction, like the green obstacle, have considerable flow. Indeed, the green lines show that the angle gets quickly bigger when the drone moves forward.

Learning visual appearance as the solution

In an article published in Nature Machine Intelligence today [1], we propose a solution to both problems. The main idea was that both problems of optical flow would disappear if the robots were able to interpret not only optical flow, but also the visual appearance of objects in their environment. This solution becomes evident from the above figures. The rectangular insets show the images captured by the drones. For the first problem it is evident that the image perfectly captures the difference in height between the white and red drone: The landing platform is simply larger in the red drone’s image. For the second problem the red obstacle is as large as the green one in the drone’s image. Given their identical size, the obstacles are equally close to the drone.

Exploiting visual appearance as captured by an image would allow robots to see distances to objects in the scene similarly to how we humans can estimate distances in a still picture. This would allow drones to immediately pick the right control gain for optical flow control and it would allow them to see obstacles in the flight direction. The only question was: How can a flying robot learn to see distances like that?

The key to this question lay in a theory I devised a few years back [2], which showed that flying robots can actively induce optical flow oscillations to perceive distances to objects in the scene. In the approach proposed in the Nature Machine Intelligence article the robots use such oscillations in order to learn what the objects in their environment look like at different distances. In this way, the robot can for example learn how fine the texture of grass is when looking at it from different heights during landing, or how thick tree barks are at different distances when navigating in a forest.

Relevance to robotics and applications

Implementing this learning process on flying robots led to much faster, smoother optical flow landings than we ever achieved before. Moreover, for obstacle avoidance, the robots were now also able to see obstacles in the flight direction very clearly. This did not only improve obstacle detection performance, but also allowed our robots to speed up. We believe that the proposed methods will be very relevant to resource-constrained flying robots, especially when they operate in a rather confined environment, such as flying in greenhouses to monitor crop or keeping track of the stock in warehouses.

It is interesting to compare our way of distance learning with recent methods in the computer vision domain for single camera (monocular) distance perception. In the field of computer vision, self-supervised learning of monocular distance perception is done with the help of projective geometry and the reconstruction of images. This results in impressively accurate, dense distance maps. However, these maps are still “unscaled” – they can show that one object is twice as far as another one but cannot convey distances in an absolute sense.

In contrast, our proposed method provides “scaled” distance estimates. Interestingly, the scaling is not in terms of meters but in terms of control gains that would lead the drone to oscillate. This makes it very relevant for control. This feels very much like the way in which we humans perceive distances. Also for us it may be more natural to reason in terms of actions (“Is an object within reach?”, “How many steps do I roughly need to get to a place?”) than in terms of meters. It hence reminds very much of the perception of “affordances”, a concept forwarded by Gibson, who introduced the concept of optical flow [3].

Picture of our drone during obstacle avoidance experiments. At first, the drone flies forward while trying to keep the height constant by keeping the vertical optical flow equal to zero. If it gets close to an obstacle, it will oscillate. The increasing oscillation when approaching an obstacle is used to learn how to see distance by means of visual appearance. After learning, the drone can fly faster and safer. For the experiments we used a Parrot Bebop 2 drone, replacing its onboard software with the Paparazzi open source autopilot – performing all computing onboard of the drone’s native processor.

Relevance to biology

The findings are not only relevant to robotics, but also provide a new hypothesis for insect intelligence. Typical honeybee experiments start with a learning phase, in which honeybees exhibit various oscillatory behaviors when they get acquainted with a new environment and related novel cues like artificial flowers. The final measurements presented in articles typically take place after this learning phase has finished and focus predominantly on the role of optical flow. The presented learning process forms a novel hypothesis on how flying insects improve their navigational skills over their lifetime. This suggests that we should set up more studies to investigate and report on this learning phase.

Guido de Croon, Christophe De Wagter, and Tobias Seid.

References

• Enhancing optical-flow-based control by learning visual appearance cues for flying robots. G.C.H.E. de Croon, C. De Wagter, and T. Seidl, Nature Machine Intelligence 3(1), 2021.
• Monocular distance estimation with optical flow maneuvers and efference copies: a stability-based strategy. G.C.H.E. de Croon, Bioinspiration & biomimetics, 11(1), 016004.
• The Ecological Approach to Visual Perception. J.J. Gibson (1979).

Interesting discussion with Hod Lipson, head of Creative Machines Lab, Columbia University in New York. Can robots be self-aware? Can they design other robots and self-repair? Why should we evolve robots to do tasks that animals do so well? Why don’t we have useful autonomous robots in the real world yet? Find out Hod’s answers to these questions and updates on VoxCAD development for designing and simulation of soft robots in this episode of the IEEE RAS Soft Robotics Podcast.

What’s more, Hod gave his personal advice to roboticists being interviewed for an assistant professorship and to 1st-year robotics PhD students looking for a thesis topic, and he also commented on his approach to the ethical dilemma of military funding scientific research. I hope you enjoy listening to the episode! You can check Hod Lipson’s portfolio here: https://www.hodlipson.com/

You can also listen to this podcast on Spotify and Apple Podcasts, and explore support options on Patreon.

Abate interviews Benjamin “Pietro” Filardo, CEO and founder of Pliant Energy Systems. At PES, they developed a novel form of actuation using two undulating fins on a robot. These fins present multiple benefits over traditional propeller systems including excellent energy efficiency, low water turbulence, and an ability to maneuver in water, land, and ice. Aside from its benefits on a robot, Pietro also talks about its advantages for harnessing energy from moving water.

Watch this episode on YouTube

Benjamin “Pietro” Filardo
After several years in the architectural profession, Pietro founded Pliant Energy Systems to explore renewable energy concepts he first pondered while earning his first degree in marine biology and oceanography. With funding from four federal agencies he has broadened the application of these concepts into marine propulsion and a highly novel robotics platform.

Links

• Download mp3 (16.1 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

NCCR Robotics researchers at EPFL have developed a drone with a feathered wing and tail that give it unprecedented flight agility.

The northern goshawk is a fast, powerful raptor that flies effortlessly through forests. This bird was the design inspiration for the next-generation drone developed by scientists of the Laboratory of Intelligent Systems of EPFL led by Dario Floreano. They carefully studied the shape of the bird’s wings and tail and its flight behavior, and used that information to develop a drone with similar characteristics.

“Goshawks move their wings and tails in tandem to carry out the desired motion, whether it is rapid changes of direction when hunting in forests, fast flight when chasing prey in the open terrain, or when efficiently gliding to save energy,” says Enrico Ajanic, the first author and PhD student in Floreano’s lab. Floreano adds: “our design extracts principles of avian agile flight to create a drone that can approximate the flight performance of raptors, but also tests the biological hypothesis that a morphing tail plays an important role in achieving faster turns, decelerations, and even slow flight.”

A drone that moves its wings and tail

The engineers already designed a bird-inspired drone with morphing wing back in 2016. In a step forward, their new model can adjust the shape of its wing and tail thanks to its artificial feathers. “It was fairly complicated to design and build these mechanisms, but we were able to improve the wing so that it behaves more like that of a goshawk,” says Ajanic. “Now that the drone includes a feathered tail that morphs in synergy with the wing, it delivers unparalleled agility.” The drone changes the shape of its wing and tail to change direction faster, fly slower without falling to the ground, and reduce air resistance when flying fast. It uses a propeller for forward thrust instead of flapping wings because it is more efficient and makes the new wing and tail system applicable to other winged drones and airplanes.

The advantage of winged drones over quadrotor designs is that they have a longer flight time for the same weight. However, quadrotors tend to have greater dexterity, as they can hover in place and make sharp turns. “The drone we just developed is somewhere in the middle. It can fly for a long time yet is almost as agile as quadrotors,” says Floreano. This combination of features is especially useful for flying in forests or in cities between buildings, as it can be necessary during rescue operation. The project is part of the Rescue Robotics Grand Challenge of NCCR Robotics.

Opportunities for artificial intelligence

Flying this new type of drone isn’t easy, due to the large number of wing and tail configurations possible. To take full advantage of the drone’s flight capabilities, Floreano’s team plans to incorporate artificial intelligence into the drone’s flight system so that it can fly semi-automatically. The team’s research has been published in Science Robotics.

• PUBLICATION – Bioinspired wing and tail morphing extends drone flight capabilities. E. Ajanic, M. Feroskhan, S. Mintchev, F. Noca, D. Floreano. Science Robotics, 5, eabc2897 (2020)

Lily flies through gusts: Scientists from Bristol and the RVC have discovered how birds fly in gusty conditions – with implications for small-scale aircraft design. Image credit: Cheney et al 2020

“Birds routinely fly in high winds close to buildings and terrain – often in gusts as fast as their flight speed. So the ability to cope with strong and sudden changes in wind is essential for their survival and to be able to do things like land safely and capture prey,” said Dr Shane Windsor from the Department of Aerospace Engineering at the University of Bristol.

“We know birds cope amazingly well in conditions which challenge engineered air vehicles of a similar size but, until now, we didn’t understand the mechanics behind it,” said Dr Windsor.

The study, published in Proceedings of the Royal Society B, reveals how bird wings act as a suspension system to cope with changing wind conditions. The team, which included Bristol PhD student Nicholas Durston and researchers Jialei Song and James Usherwood from Dongguan University of Technology in China and the RVC respectively, used an innovative combination of high-speed, video-based 3D surface reconstruction, computed tomography (CT) scans, and computational fluid dynamics (CFD) to understand how birds ‘reject’ gusts through wing morphing, i.e. by changing the shape and posture of their wings.

In the experiment, conducted in the Structure and Motion Laboratory at the Royal Veterinary College, the team filmed Lily, a barn owl, gliding through a range of fan-generated vertical gusts, the strongest of which was as fast as her flight speed. Lily is a trained falconry bird who is a veteran of many nature documentaries, so wasn’t fazed in the least by all the lights and cameras. “We began with very gentle gusts in case Lily had any difficulties, but soon found that – even at the highest gust speeds we could make – Lily was unperturbed; she flew straight through to get the food reward being held by her trainer, Lloyd Buck,” commented Professor Richard Bomphrey of the Royal Veterinary College.

“Lily flew through the bumpy gusts and consistently kept her head and torso amazingly stable over the trajectory, as if she was flying with a suspension system. When we analysed it, what surprised us was that the suspension-system effect wasn’t just due to aerodynamics, but benefited from the mass in her wings. For reference, each of our upper limbs is about 5% of our body weight; for a bird it’s about double, and they use that mass to effectively absorb the gust,” said joint lead-author Dr Jorn Cheney from the Royal Veterinary College.

“Perhaps most exciting is the discovery that the very fastest part of the suspension effect is built into the mechanics of the wings, so birds don’t actively need to do anything for it to work. The mechanics are very elegant. When you strike a ball at the sweetspot of a bat or racquet, your hand is not jarred because the force there cancels out. Anyone who plays a bat-and-ball sport knows how effortless this feels. A wing has a sweetspot, just like a bat. Our analysis suggests that the force of the gust acts near this sweetspot and this markedly reduces the disturbance to the body during the first fraction of a second. The process is automatic and buys just enough time for other clever stabilising processes to kick in,” added joint lead-author, Dr Jonathan Stevenson from the University of Bristol.

Dr Windsor said the next step for the research, which was funded by the European Research Council (ERC), Air Force Office of Scientific Research and the Wellcome Trust, is to develop bio-inspired suspension systems for small-scale aircraft.

• PAPER – Bird wings act as a suspension system that rejects gusts. Jorn A. Cheney, Jonathan P. J. Stevenson, Nicholas E. Durston, Jialei Song, James R. Usherwood, Richard J. Bomphrey and Shane P. Windsor. Proceedings of the Royal Society B.

In this episode, Kate Zhou interviews Farrell Helbling, postdoctoral fellow at Harvard Microrobotics lab, who has worked on developing the RoboBee, an insect-inspired robot that is the lightest vehicle to achieve untethered flight. Farrell discusses challenges with building the robot at centimeter-scale as well as integration of sensors and power electronics particularly in considerations with weight trade-offs.

Farrell Helbling

Farrell Helbling is a post doctoral fellow at Harvard University, where she focuses on the systems-level design of the Harvard RoboBee, an insect-scale flapping wing robot. Her research looks at the integration of the control system, sensors, and power electronics within the strict weight and power constraints of the vehicle. Her work led to the first untethered flight of an insect scale vehicle and was recently featured on the cover of Nature. She is the recipient of a NSF Graduate Research Fellowship, a 2018 Rising Star in EECS, and co-author on the IROS 2015 Best Student Paper for an insect-scale, hybrid aerial-aquatic vehicle. Her work on the RoboBee project is also featured at the Boston Museum of Science, World Economic Forum, London Science Museum, and the Smithsonian, as well as in the popular press (PBS NewsHour, The New York Times, Science Friday, BBC, and Wired). She is interested in the codesign of mechanical and electrical systems for mass-, power-, and computation-constrained robots.

Links

• Harvard Biorobotics Lab
• Download mp3 (17.1 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

with Guy Hoffman
Robotics Researcher, Cornell University

An Art, Technology, and Culture Colloquium, co-sponsored by the Center for New Music and Audio Technologies and CITRIS People and Robots (CPAR), presented with Berkeley Arts + Design as part of Arts + Design Mondays.

As we continue to develop social robots designed for connectedness, we struggle with paradoxes related to authenticity, transience, and replication. In this talk, I will attempt to link together 15 years of experience designing social robots with 100-year-old texts on transience, replication, and the fear of dying. Can there be meaningful relationships with robots who do not suffer natural decay? What would our families look like if we all choose to buy identical robotic family members? Could hand-crafted robotics offer a relief from the mass-replication of the robot’s physical body and thus also from the mass-customization of social experiences?

About Guy Hoffman

Dr. Guy Hoffman is an Assistant Professor and the Mills Family Faculty Fellow in the Sibley School of Mechanical and Aerospace Engineering at Cornell University. Prior to that he was an Assistant Professor at IDC Herzliya and co-director of the IDC Media Innovation Lab. Hoffman holds a Ph.D from MIT in the field of human-robot interaction. He heads the Human-Robot Collaboration and Companionship (HRC2) group, studying the algorithms, interaction schema, and designs enabling close interactions between people and personal robots in the workplace and at home. Among others, Hoffman developed the world’s first human-robot joint theater performance, and the first real-time improvising human-robot Jazz duet. His research papers won several top academic awards, including Best Paper awards at robotics conferences in 2004, 2006, 2008, 2010, 2013, 2015, 2018, and 2019. His TEDx talk is one of the most viewed online talks on robotics, watched more than 3 million times.

About the Art, Technology, and Culture Colloquium

Founded by Prof. Ken Goldberg in 1997, the ATC lecture series is an internationally respected forum for creative ideas. Always free of charge and open to the public, the series is coordinated by the Berkeley Center for New Media and has presented over 200 leading artists, writers, and critical thinkers who question assumptions and push boundaries at the forefront of art, technology, and culture including: Vito Acconci, Laurie Anderson, Sophie Calle, Bruno Latour, Maya Lin, Doug Aitken, Pierre Huyghe, Miranda July, Billy Kluver, David Byrne, Gary Hill, and Charles Ray.

Fall 2019 – Spring 2020 Series Theme: Robo-Exoticism

In 1920, Karl Capek coined the term “robot” in a play about mechanical workers organizing a rebellion to defeat their human overlords. A century later, increasing popularism, inequality, and xenophobia require us to reconsider our assumptions about labor, trade, political stability, and community. At the same time, advances in artificial intelligence and robotics, fueled by corporations and venture capital, challenge our assumptions about the distinctions between humans and machines. To explore potential linkages between these trends, “Robo-Exoticism” characterizes a range of human responses to AI and robots that exaggerate both their negative and positive attributes and reinforce fears, fantasies, and stereotypes.

Robo-Exoticism Calendar

09/09/19 Robots Are Creatures, Not Things
Madeline Gannon, Artist / Roboticist, Pittsburgh, PA
Co-sponsored by the Jacobs Institute for Design Innovation and CITRIS People and Robots (CPAR)

09/23/19 The Copper in my Cooch and Other Technologies
Marisa Morán Jahn, Artist, Cambridge, MA and New York, NY
Co-sponsored by the Wiesenfeld Visiting Artist Lecture Series and the Jacobs Institute for Design Innovation

10/21/19 Non-Human Art
Leonel Moura, Artist, Lisbon
Co-sponsored by the Department of Spanish & Portuguese and CITRIS People and Robots (CPAR)

11/4/19 Transience, Replication, and the Paradox of Social Robotics
Guy Hoffman, Robotics Researcher, Cornell University
Co-sponsored by the Center for New Music and Audio Technologies and CITRIS People and Robots (CPAR)

01/27/20 Dancing with Robots: Expressivity in Natural and Artificial Systems
Amy LaViers, Robotics, Automation, and Dance (RAD) Lab
Co-sponsored by the Department of Theater, Dance, and Performance Studies and CITRIS People and Robots (CPAR)

02/24/20 In Search for My Robot: Emergent Media, Racialized Gender, and Creativity
Margaret Rhee, Assistant Professor, SUNY Buffalo; Visiting Scholar, NYU
Co-sponsored by the Department of Ethnic Studies and the Department of Comparative Literature

03/30/20 The Right to Be Creative
Margarita Kuleva, National Research University Higher School of Economics, Moscow
Invisible Russia: Participatory Cultures, Their Practices and Values
Natalia Samutina, National Research University Higher School of Economics, Moscow
Co-sponsored by the Department of Slavic Languages and Literature and Department of the History of Art and the Arts Research Center

04/06/20 Artist Talk
William Pope.L, Artist
Presented by the Department of Art Practice

04/13/20 Teaching Machines to Draw
Tom White, New Zealand
Co-sponsored by Autolab and CITRIS People and Robots (CPAR)

For more information:

http://atc.berkeley.edu/

Contact: info.bcnm [​at​] berkeley.edu, 510-495-3505

ATC Director: Ken Goldberg
BCNM Director: Nicholas de Monchaux
Arts + Design Director: Shannon Jackson
BCNM Liaisons: Lara Wolfe, Laurie Macfee

ATC Highlight Video from F10-S11 Season (2 mins)
http://j.mp/atc-highlights-hd

ATC Audio-Video Archive on Brewster Kahle’s Internet Archive:
http://tinyurl.com/atc-internet-archive

ATC on Facebook:
https://www.facebook.com/cal-atc

ATC on Twitter:
https://www.twitter.com/cal_atc

In this episode, Marwa ElDiwiny interview Peer Fisher, a Professor of Physical Chemistry at the University of Stuttgart and the Director of the Micro Nano and Molecular Systems Lab at the Max Planck Institute for Intelligent Systems.  Fischer discusses micro robots that has been designed to move inside of environments similar to the human body called, “micro swimmers.”  He talks about how they are fabricated, powered, and how they can move with light or “nano propellers.”  Fischer also discusses simulating human tissue and the future of micro and nano robots, including how they could be a replacement for certain surgeries.

Peer Fischer

Peer Fischer is a Professor of Physical Chemistry at the University of Stuttgart and he heads the independent Micro Nano and Molecular Systems Lab at the Max Planck Institute for Intelligent Systems in Stuttgart.

He received a BSc. degree in Physics from Imperial College London and a Ph.D. from the University of Cambridge. He was a NATO (DAAD) Postdoctoral Fellow at Cornell University, before joining the Rowland Institute at Harvard. At Harvard he held a Rowland Fellowship and directed an interdisciplinary research lab for five years. In 2009 he received an Attract Award from the Fraunhofer Society which led him to set up a photonics lab at the Fraunhofer Institute for Physical Measurement Techniques in Freiburg. In 2011 he moved his lab to the Max Planck Institute for Intelligent Systems in Stuttgart, and since 2013 he is a Professor at the University of Stuttgart.

Links

• Micro, Nano, & Molecular Systems Lab
• Micro Swimmers
• Download mp3 (13.7 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Audrow Nash interviews Juxi Leitner, a Postdoctoral Research Fellow at QUT; and Nicholas Panitz, Ben Wilson, and James Brett, from CSIRO.

Leitner speaks about the Amazon Picking challenge, a challenge to advance the state of robotic grasping, and their robot which won the challenge in 2017. Their robot is similar to a cartesian 3D printer in form and uses either a suction cup or a pinch gripper for grabbing objects. Their robot has a depth camera and uses a digital scale to determine if an object has been picked up successfully. Leitner discusses what their team did differently from other teams that helped them win the competition.

Panitz, Wilson, and Brett speak about their hexapod robots. Their hexapods are for several purposes, such as environmental monitoring and remote inspection. They choose to use hexapods because they are statically stable. They discuss the design of their hexapods and how research works at Commonwealth Scientific and Industrial Research Organization, or CSIRO.

A video of the robot Leitner discusses, #Cartman:

An example of CSIRO’s hexapod robots for inspection:

Links

• Download mp3 (12.9 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Audrow Nash speaks with Maja Matarić, a professor at the University of Southern California and the Chief Science Officer of Embodied, about socially assistive robotics. Socially assistive robotics aims to endow robots with the ability to help people through individual non-contact assistance in convalescence, rehabilitation, training, and education. For example, a robot could help a child on the autism spectrum to connect to more neurotypical children and could help to motivate a stroke victim to follow their exercise routine for rehabilitation (see the videos below). In this interview, Matarić discusses the care gap in health care, how her work leverages research in psychology to make robots engaging, and opportunities in socially assistive robotics for entrepreneurship.

A short video about how personalized robots might act as a “social bridge” between a child on the autism spectrum and a more neurotypical child.

A short video about how a robot could assist stroke victims in their recovery.

Maja Matarić

Maja Matarić is professor and Chan Soon-Shiong chair in Computer Science Department, Neuroscience Program, and the Department of Pediatrics at the University of Southern California, founding director of the USC Robotics and Autonomous Systems Center (RASC), co-director of the USC Robotics Research Lab and Vice Dean for Research in the USC Viterbi School of Engineering. She received her PhD in Computer Science and Artificial Intelligence from MIT, MS in Computer Science from MIT, and BS in Computer Science from the University of Kansas. 

Links

• Interaction Lab
• Embodied
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon
• Download MP3 (13.7 MB)

In this episode, Audrow Nash speaks with Monica Daley about learning from birds about legged locomotion. To do this, Daley analyzes the gaits of guineafowl in various experiments to understand the mechanical principles underlying gaits, such as energetic economy, mechanical limits, and how the birds avoid injury. She then tests her ideas about legged locomotion on legged robots with collaborators, including Jonathan Hurst from Oregon State University. Daley also speaks about her experience with interdisciplinary collaborations. 

Monica Daley

Monica Daley earned an HBSc in Biology with a Chemistry minor at the University of Utah, where she was inspired to pursue an academic career through her research on human locomotor-ventilatory integration with Dennis Bramble and David Carrier. Daley then spent a year as a research technician at the University of Utah, investigating motor control of singing in zebra finches in the lab of Franz Goller. These experiences initiated a long-standing fascination with the interplay of mechanics and neural control.

Daley went on to Harvard University, where she earned her MA and PhD in Organismic and Evolutionary Biology. Her research on muscle-tendon dynamics and biomechanics of avian bipedal locomotion was supported by a prestigious Predoctoral Fellowship award from the Howard Hughes Medical Institute and supervised by Andrew Biewener at the Concord Field Station of Harvard University (CFS Website).

After completing her PhD, Daley was awarded a Research Fellowship by the U.S. National Science Foundation to develop models of the dynamics and control of bipedal locomotion, working with Dan Ferris in the Human Neuromechanics Lab at University of Michigan, in collaboration with Auke Ijspeert in the Biologically Inspired Robotics Group at the Swiss Federal Institute of Technology in Lausanne.

Daley is a faculty member of the Structure and Motion Lab, where she leads research in Comparative Neuromechanics— a field that seeks to understand the interplay of morphology, mechanics and sensorimotor control that influences how animals move through their environment.

Links

• Structure and Motion Lab
• Download mp3 (15.5 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Audrow Nash interviews Elliott Rouse, Assistant Professor at the University of Michigan, about an open-source prosthetic leg—that is a robotic knee and ankle. Rouse’s goal is to provide an inexpensive and capable platform for researchers to use so that they can work on prostheses without developing their own hardware, which is both time-consuming and expensive. Rouse discusses the design of the leg, the software interface, and the project’s timeline.

Elliott Rouse

Elliott Rouse is an Assistant Professor in the Mechanical Engineering Department at the University of Michigan, where he directs the Neurobionics Lab. The vision of his group is to discover the fundamental science that underlies human joint dynamics during locomotion and incorporate these discoveries in a new class of wearable robotic technologies. The Lab uses technical tools from mechanical and biomedical engineering applied to the complex challenges of human augmentation, physical medicine, rehabilitation and neuroscience. Dr. Rouse and his research have been featured at TED, on the Discovery Channel, CNN, National Public Radio, Wired Magazine UK, Business Insider, and Odyssey Magazine.

Links

• Rouse’s lab website
• Rouse’s personal website
• Download mp3 (12.6 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this interview, Audrow Nash interviews Helen Huang, Joint Professor at the University of North Carolina at Chapel Hill and North Carolina State, about a method of tuning powered lower limb prostheses. Huang explains how powered prostheses are adjusted for each patient and how she is using supervised and reinforcement learning to tune prosthesis. Huang also discusses why she is not using the energetic cost of transport as a metric and the challenge of people adapting to a device while it learns from them.

Helen Huang

Helen Huang is a Professor in the Joint Department of Biomedical Engineering at the University of North Carolina at Chapel Hill (UNC) and North Carolina State University (NCSU) and the Director of the NCSU/UNC Closed-Loop Engineering for Advanced Rehabilitation (CLEAR) Core. Huang is interested in neural-machine interfaces for prostheses and exoskeletons, wearer-robot interaction, adaptive and optimal control of wearable robots, and human movement control. She has developed an EMG-based neural interface for robotic prosthetic legs. She was a recipient of the Delsys Prize for Innovation in Electromyography, the Mary E. Switzer Fellowship with the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR), and a NSF CAREER Award and was named NC State faculty scholar in 2015.

Links

• Huang’s research lab
• Download mp3 (19.9 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Jack Rasiel interviews Vijay Kumar, Professor and Dean of Engineering at the University of Pennsylvania.  Kumar discusses the guiding ideas behind his research on micro unmanned aerial vehicles, gives his thoughts on the future of robotics in the lab and field, and speaks about setting realistic expectations for robotics technology.

Vijay Kumar

Vijay Kumar is the Nemirovsky Family Dean of Penn Engineering with appointments in the Departments of Mechanical Engineering and Applied Mechanics, Computer and Information Science, and Electrical and Systems Engineering at the University of Pennsylvania.

Dr. Kumar received his Bachelor of Technology degree from the Indian Institute of Technology, Kanpur and his Ph.D. from The Ohio State University in 1987. He has been on the Faculty in the Department of Mechanical Engineering and Applied Mechanics with a secondary appointment in the Department of Computer and Information Science at the University of Pennsylvania since 1987. In his time at the university, Dr. Kumar has held numerous positions including director of the GRASP Laboratory, Chairman of the Department of Mechanical Engineering and Applied Mechanics, and Deputy Dean for Education in the School of Engineering and Applied Science. From 2012 to 2013, he served as the assistant director of robotics and cyber physical systems at the White House Office of Science and Technology Policy.

Links

• Vijay Kumar Lab
• Download (11.8 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Child using robot-driven TPAD training method to improve crouch gait, symptom of cerebral palsy.
—Photo courtesy of Sunil Agrawal/Columbia Engineering

In the U.S., 3.6 out of 1000 school-aged children are diagnosed with cerebral palsy (CP). Their symptoms include abnormal gait patterns which results in joint degeneration over time. Slow walking speed, reduced range of motion of the joints, small step length, large body sway, and absence of a heel strike are other difficulties that children with CP experience. A subset of these children exhibit crouch gait which is characterized by excessive flexion of the hips, knees, or ankles.

A team led by Sunil Agrawal, professor of mechanical engineering and of rehabilitation and regenerative medicine at Columbia Engineering, has published a pilot study in Science Robotics that demonstrates a robotic training method that improves posture and walking in children with crouch gait by enhancing their muscle strength and coordination.

Crouch gait is caused by a combination of weak extensor muscles that do not produce adequate muscle forces to keep posture upright, coupled with tight flexor muscles that limit the joint range of motion. Among the extensor muscles, the soleus, a muscle that runs from just below the knee to the heel, plays an important role in preventing knee collapse during the middle of the stance phase when the foot is on the ground. Critical to standing and walking, the soleus muscle keeps the shank upright during the mid-stance phase of the gait to facilitate extension of the knee. It also provides propulsive forces on the body during the late stance phase of the gait cycle.

“One of the major reasons for crouch gait is weakness in soleus muscles,” says Agrawal, who is also a member of the Data Science Institute. “We hypothesized that walking with a downward pelvic pull would strengthen extensor muscles, especially the soleus, against the applied downward pull and would improve muscle coordination during walking. We took an approach opposite to conventional therapy with these children: instead of partial body weight suspension during treadmill walking, we trained participants to walk with a force augmentation.”

The research group knew that the soleus, the major weight-bearing muscle during single stance support, is activated more strongly among the lower leg muscles when more weight is added to the human body during gait. They reasoned that strengthening the soleus might help children with crouch gait to stand and walk more easily.

To test their hypothesis, the team used a robotic system — Tethered Pelvic Assist Device (TPAD) — invented in Agrawal’s Robotics and Rehabilitation (ROAR) Laboratory. The TPAD is a wearable, lightweight cable-driven robot that can be programmed to provide forces on the pelvis in a desired direction as a subject walks on a treadmill. The researchers worked with six children diagnosed with CP and exhibiting crouch gait for fifteen 16-minute training sessions over a duration of six weeks. While the children walked on treadmills, they wore the TPAD as a lightweight pelvic belt to which several wires were attached. The tension in each TPAD wire was controlled in real time by a motor placed on a stationary frame around the treadmill, based on real-time motion capture data from cameras. The researchers programmed the TPAD to apply an additional downward force through the center of the pelvis to intensively retrain the activity of the soleus muscles. They used a downward force equivalent to 10 percent of body weight, based on the results of healthy children carrying backpacks. This was the minimum weight needed to show notable changes in posture or gait during walking.

“TPAD is a unique device because it applies external forces on the human body during walking,” says Jiyeon Kang, PhD candidate and lead author of the paper. “The training with this device is distinctive because it does not add mass/inertia to the human body during walking.”

The team examined the children’s muscle strength and coordination using electromyography data from the first and last sessions of training and also monitored kinematics and ground reaction forces continuously throughout the training. They found that their training was effective; it both enhanced the children’s upright posture and improved their muscle coordination. In addition, their walking features, including step length, range of motion of the lower limb angles, toe clearance, and heel-to-toe pattern, improved.

“Currently, there is no well-established physical therapy or strengthening exercise for the treatment of crouch gait,” Agrawal notes.

Heakyung Kim, A. David Gurewitsch Professor of Rehabilitation and Regenerative Medicine and Professor of Pediatrics at the Columbia University Medical Center, who treats these patients, added “Feedback from the parents and children involved in this study was consistent. They reported improved posture, stronger legs, and faster walking speed, and our measurements bear that out. We think that our robotic TPAD training with downward pelvic pull could be a very promising intervention for these children.”

The researchers are planning more clinical trials, to test a larger group and changing more variables. They are also considering studying children with hemiplegic/quadriplegic CP.

Click here to view the research paper.

By Taylor Kubota, Stanford News Service

Imagine rescuers searching for people in the rubble of a collapsed building. Instead of digging through the debris by hand or having dogs sniff for signs of life, they bring out a small, air-tight cylinder. They place the device at the entrance of the debris and flip a switch. From one end of the cylinder, a tendril extends into the mass of stones and dirt, like a fast-climbing vine. A camera at the tip of the tendril gives rescuers a view of the otherwise unreachable places beneath the rubble.

This is just one possible application of a new type of robot created by mechanical engineers at Stanford University, detailed in a June 19 Science Robotics paper. Inspired by natural organisms that cover distance by growing — such as vines, fungi and nerve cells — the researchers have made a proof of concept of their soft, growing robot and have run it through some challenging tests.

“Essentially, we’re trying to understand the fundamentals of this new approach to getting mobility or movement out of a mechanism,” explained Allison Okamura, professor of mechanical engineering and senior author of the paper. “It’s very, very different from the way that animals or people get around the world.”

To investigate what their robot can do, the group created prototypes that move through various obstacles, travel toward a designated goal, and grow into a free-standing structure. This robot could serve a wide range of purposes, particularly in the realms of search and rescue and medical devices, the researchers said.

A growing robot

The basic idea behind this robot is straightforward. It’s a tube of soft material folded inside itself, like an inside-out sock, that grows in one direction when the material at the front of the tube everts, as the tube becomes right-side-out. In the prototypes, the material was a thin, cheap plastic and the robot body everted when the scientists pumped pressurized air into the stationary end. In other versions, fluid could replace the pressurized air.

What makes this robot design extremely useful is that the design results in movement of the tip without movement of the body.

“The body lengthens as the material extends from the end but the rest of the body doesn’t move,” explained Elliot Hawkes, a visiting assistant professor from the University of California, Santa Barbara and lead author of the paper. “The body can be stuck to the environment or jammed between rocks, but that doesn’t stop the robot because the tip can continue to progress as new material is added to the end.”

Graduate students Joseph Greer, left, and Laura Blumenschein, right, work with Elliot Hawkes, a visiting assistant professor from the University of California, Santa Barbara, on a prototype of the vinebot. (Image credit: L.A. Cicero)

The group tested the benefits of this method for getting the robot from one place to another in several ways. It grew through an obstacle course, where it traveled over flypaper, sticky glue and nails and up an ice wall to deliver a sensor, which could potentially sense carbon dioxide produced by trapped survivors. It successfully completed this course even though it was punctured by the nails because the area that was punctured didn’t continue to move and, as a result, self-sealed by staying on top of the nail.

In other demonstrations, the robot lifted a 100-kilogram crate, grew under a door gap that was 10 percent of its diameter and spiraled on itself to form a free-standing structure that then sent out a radio signal. The robot also maneuvered through the space above a dropped ceiling, which showed how it was able to navigate unknown obstacles as a robot like this might have to do in walls, under roads or inside pipes. Further, it pulled a cable through its body while growing above the dropped ceiling, offering a new method for routing wires in tight spaces.

Difficult environments

“The applications we’re focusing on are those where the robot moves through a difficult environment, where the features are unpredictable and there are unknown spaces,” said Laura Blumenschein, a graduate student in the Okamura lab and co-author of the paper. “If you can put a robot in these environments and it’s unaffected by the obstacles while it’s moving, you don’t need to worry about it getting damaged or stuck as it explores.”

Some iterations of these robots included a control system that differentially inflated the body, which made the robot turn right or left. The researchers developed a software system that based direction decisions on images coming in from a camera at the tip of the robot.

A primary advantage of soft robots is that they can be safer than hard, rigid robots not only because they are soft but also because they are often lightweight. This is especially useful in situations where a robot could be moving in close quarters with a person. Another benefit, in the case of this robot, is that it is flexible and can follow complicated paths. This, however, also poses some challenges.

Joey Greer, a graduate student in the Okamura lab and co-author of the paper, said that controlling a robot requires a precise model of its motion, which is difficult to establish for a soft robot. Rigid robots, by comparison, are much easier to model and control, but are unusable in many situations where flexibility or safety is necessary. “Also, using a camera to guide the robot to a target is a difficult problem because the camera imagery needs to be processed at the rate it is produced. A lot of work went into designing algorithms that both ran fast and produced results that were accurate enough for controlling the soft robot,” Greer said.

Going big—and small

As it exists now, the scientists built the prototype by hand and it is powered through pneumatic air pressure. In the future, the researchers would like to create a version that would be manufactured automatically. Future versions may also grow using liquid, which could help deliver water to people trapped in tight spaces or to put out fires in closed rooms. They are also exploring new, tougher materials, like rip-stop nylon and Kevlar.

The vinebot is a tube of soft material that grows in one direction. (Image credit: L.A. Cicero)

The researchers also hope to scale the robot much larger and much smaller to see how it performs. They’ve already created a 1.8 mm version and believe small growing robots could advance medical procedures. In place of a tube that is pushed through the body, this type of soft robot would grow without dragging along delicate structures.

Okamura is a member of Stanford Bio-X and the Stanford Neurosciences Institute.

This research was funded by the National Science Foundation.

Spiders are truly amazing creatures. They have evolved over more than 200 million years and can be found in almost every corner of our planet. They are one of the most successful animals. Not less impressive are their webs, highly intricate structures that have been optimised through evolution over approximately 100 million years with the ultimate purpose of catching prey.

However, interestingly, the closer you look at spiders’ webs the more details you can observe and the structures are much more complicated than one would expect from a simple snare. They are made of a variety of different types of silks, use water droplets to keep the tension [see: citation 4], and the structure is highly dynamic [see: citation 4]. Spider’s webs have a great deal more morphological complexity than what you would need to simply catch flies.

Since nature typically does not spoil resources the question arises: why are spiders’ webs so complex? Might they have other functionalities besides being a simple trap? One of the most interesting answers to this question is that spiders might use their webs as computational devices.

How does the spider use the web as a computer?

Despite the fact that most spiders have a lot of eyes (the majority has 8, but some have even up to 12), a lot of the spiders have bad sight. In order to understand what is going on in their webs, they use mechanoreceptors in their legs (called lyriforms) to “listen” to vibrations in the web.  Different species of spiders have different preferred places to sit and observe. While some can be found right at the center, others prefer to sit outside the actual web and to listen to one single thread. It is quite remarkable that based only on the information that comes through this single thread the spider seems to be able to deduce what is going on in their web and where these events are taking place.

For example, they need to know if there is a prey, like a fly, entangled in their web. Or if the vibrations are coming from a dangerous insect like a wasp and they should stay away. The web is also used to communicate with potential mates and the spider even excites the web and listens to the echo. This might be a way for the spider to check if threads are broken or if the tension in the web has to be increased.

From a computational point of view, the spider needs to classify different vibration pattern (e.g., prey vs predator vs mate) and to locate its origin (i.e., where the vibration started).

One way to understand how a spider’s web could help to carry out this computational functionality is the concept of morphological computation. This is a term that describes the understanding that mechanical structures all over in nature are carrying out useful computations. For example, they help to stabilise running, facilitate sensory data processing, and helps animals and plants to interact with complex and unpredictable environments.

One could say computation is outsourced to the physical body (e.g., from the brain to another part of the body).

From this point of view, the spider’s web can be seen as a nonlinear, dynamic filter. It can be understood as some kind of pre-processing unit that makes it easier for the animal to interpret the vibration signals. The web’s dynamic properties and its complex morphological structure mix vibration signals in a nonlinear fashion. It even has some memory. This can be easily seen by pinching the web. It responds with vibrations for some moments after the impact echoing the original input.  The web can also damp unwanted frequencies, which is crucial to get rid of noise. On the other hand, it might even be able to highlight other signals at certain frequencies that carry more relevant information about the events taking place on the web.

These are all useful computations and they make it easier for the spider to “read” and understand the vibration patterns. As a result, the brain of the animal has to do less work and it can concentrate on other tasks. In effect, the spider seems to devolve computation to the web.  This might be also the reason why spiders tend to their webs so intensively. They constantly observe it and adapt the tension if it has changed, e.g. due to change in humidity, and repair it as soon a thread is broken.

From spider webs to sensors

People have speculated for a while that spider webs might have additional functionalities. A great article that discusses that is “The Thoughts of a Spiderweb“.

However, nobody so far has systematically looked into the actual computational capabilities of the web. This is about to change. We recently started a Leverhulme Trust Research project that will investigate naturally spun spider webs of different species to understand how and which computing might take place in these structures.  Moreover, the project does not only try to understand the underlying computational principles but will also develop morphological computation-based sensor technology to measure flow and vibrations.

The project combines our research expertise in Morphological Computation at the University of Bristol and the expertise on spider webs at the Silk Group in Oxford.

In experimental setups we will use solenoids and laser Doppler vibrometers to measure vibrations in the web with very high precision. The goal is to understand how computation is carried out. We will systematically investigate how filtering capabilities, memory, and signal integration can happen in such structures. In parallel, we will develop a general simulation environment for vibrating structures. We will use this to ask specific questions about how different shapes and materials others than spider webs and silk can help to carry out computations. In addition, we will develop real prototypes of vibration and flow sensors, which will be inspired by these findings. It’s very likely they will look different from spider webs and they will use various types of materials.

Such sensors can be used in various applications. For example, morphological computation based flow sensors could be used to detect anomalies in the flow in tubes. Or vibration sensors put at strategic places on buildings could be able to detect earthquakes or structural failure. Also highly dynamic machines, for example, a wind turbine, could be monitored by such sensors to predict failure.

Ultimately, the project will provide not only a new technology to build sensors, but we hope also to get a fundamental understanding how spiders use their webs for computation.

References

[1] Hauser, H.; Ijspeert, A.; Füchslin, R.; Pfeifer, R. & Maass, W.”Towards a theoretical foundation for morphological computation with compliant bodies.”Biological Cybernetics, Springer Berlin / Heidelberg, 2011, 105, 355-370

[2]  Hauser, H.; Ijspeert, A.; Füchslin, R.; Pfeifer, R. & Maass, W. “The role of feedback in morphological computation with compliant bodies”. Biological Cybernetics, Springer Berlin / Heidelberg, 2012, 106, 595-613

[3] Hauser, H.; Füchslin, R.M.; Nakajima, K.“Morphological Computation – The Physical Body as a Computational Resource” Opinions and Outlooks on Morphological Computation, editors Hauser, H.; Füchslin, R.M. and Pfeifer, R., Chapter 20, pp 226-244,  2014, ISBN 978-3-033-04515-6

[4] Mortimer, B., Gordon, S. D., Holland, C., Siviour, C. R., Vollrath, F. and Windmill, J. F. C. (2014), The Speed of Sound in Silk: Linking Material Performance to Biological Function. Adv. Mater., 26: 5179–5183. doi:10.1002/adma.201401027

Image: Alex Waller, MIT CSAIL

Being able to both walk and take flight is typical in nature – many birds, insects and other animals can do both. If we could program robots with similar versatility, it would open up many possibilities: picture machines that could fly into construction areas or disaster zones that aren’t near roads, and then be able to squeeze through tight spaces to transport objects or rescue people.

The problem is that usually robots that are good at one mode of transportation are, by necessity, bad at another. Drones are fast and agile, but generally have too limited of a battery life to travel for long distances. Ground vehicles, meanwhile, are more energy efficient, but also slower and less mobile.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) are aiming to develop robots that can do both. In a new paper, the team presented a system of eight quadcopter drones that can both fly and drive through a city-like setting with parking spots, no-fly zones and landing pads.

“The ability to both fly and drive is useful in environments with a lot of barriers, since you can fly over ground obstacles and drive under overhead obstacles,” says PhD student Brandon Araki, lead author on a paper about the system out of CSAIL director Daniela Rus’ group. “Normal drones can’t maneuver on the ground at all. A drone with wheels is much more mobile while having only a slight reduction in flying time.”

Araki and Rus developed the system along with MIT undergraduate students John Strang, Sarah Pohorecky and Celine Qiu, as well as Tobias Naegeli of ETH Zurich’s Advanced Interactive Technologies Lab. The team presented their system at IEEE’s International Conference on Robotics and Automation (ICRA) in Singapore earlier this month.

How it works

The project builds on Araki’s previous work developing a “flying monkey” robot that crawls, grasps, and flies. While the monkey robot could hop over obstacles and crawl about, there was still no way for it to travel autonomously.

To address this, the team developed various “path-planning” algorithms aimed at ensuring that the drones don’t collide. To make them capable of driving, the team put two small motors with wheels on the bottom of each drone. In simulations the robots could fly for 90 meters or drive for 252 meters before their batteries ran out.

Adding the driving component to the drone slightly reduced its battery life, meaning that the maximum distance it could fly decreased 14 percent to about 300 feet. But since driving is still much more efficient than flying, the gain in efficiency from driving more than offsets the relatively small loss in efficiency in flying due to the extra weight.

“This work provides an algorithmic solution for large-scale, mixed-mode transportation and shows its applicability to real-world problems,” says Jingjin Yu, a computer science professor at Rutgers University who was not involved in the paper.

The team also tested the system using everyday materials like pieces of fabric for roads and cardboard boxes for buildings. They tested eight robots navigating from a starting point to an ending point on a collision-free path, and all were successful.

Rus says that systems like theirs suggest that another approach to creating safe and effective flying cars is not to simply “put wings on cars,” but to build on years of research in drone development to add driving capabilities to them.

“As we begin to develop planning and control algorithms for flying cars, we are encouraged by the possibility of creating robots with these capabilities at small scale,” says Rus. “While there are obviously still big challenges to scaling up to vehicles that could actually transport humans, we are inspired by the potential of a future in which flying cars could offer us fast, traffic-free transportation.”

Click here to read the paper.

Image: Harvard Gazette

By Peter Reuell, Harvard Staff Writer

At the beginning of the decade, George Whitesides helped rewrite the rules of what a machine could be with the development of biologically inspired “soft robots.” Now he’s poised to rewrite them again, with help from some plastic drinking straws.

Inspired by arthropod insects and spiders, Whitesides and Alex Nemiroski, a former postdoctoral fellow in Whitesides’ Harvard lab, have created a type of semi-soft robot capable of standing and walking. The team also created a robotic water strider capable of pushing itself along the liquid surface. The robots are described in a recently published paper in the journal Soft Robotics.

Unlike earlier generations of soft robots, which could stand and awkwardly walk by inflating air chambers in their bodies, the new robots are designed to be far nimbler. Though real-world applications are still far off, the researchers hope the robots eventually could be used in search operations following natural disasters or in conflict zones.

“If you look around the world, there are a lot of things, like spiders and insects, that are very agile,” said Whitesides, the Woodford L. and Ann A. Flowers University Professor at Harvard. “They can move rapidly, climb on various items, and are able to do things that large, hard robots can’t do because of their weight and form factor. They are among the most versatile organisms on the planet. The question was, how can we build something like that?”

The answer, Nemiroski said, came in the form of your average drinking straw.

“This all started with an observation that George made, that polypropylene tubes have an excellent strength-to-weight ratio. That opened the door to creating something that has more structural support than purely soft robots have,” he said. “That was the building block, and then we took inspiration from arthropods to figure out how to make a joint and how to use the tubes as an exoskeleton. From there it was a question of how far can your imagination go? Once you have a Lego brick, what kind of castle can you build with it?”

What they built, he said, is a surprisingly simple joint.

Whitesides and Nemiroski began by cutting a notch in the straws, allowing them to bend. The scientists then inserted short lengths of tubing which, when inflated, would force the joints to extend. A rubber tendon attached on either side would then cause the joint to retract when the tubing deflated.

Armed with that simple concept, the team built a one-legged robot capable of crawling, and moved up in complexity as they added a second and then a third leg, allowing the robot to stand on its own.

“With every new level of systems complexity, we would have to go back to the original joint and make modifications to make it capable of exerting more force or to be able to support the weight of larger robots,” Nemiroski said. “Eventually, when we graduated to six- or eight-legged arthrobots, making them walk became a challenge from a programming perspective. For example, we looked at the way ants and spiders sequence the motion of their limbs and then tried to figure out whether aspects of these motions were applicable to what we were doing or whether we’d need to develop our own type of walking tailored to these specific types of joints.”

While Nemiroski and colleagues were able to control simple robots by hand, using syringes, they turned to computers to control the sequencing of their limbs as the designs increased in complexity.

“We put together a microcontroller run by Arduino that uses valves and a central compressor,” he said. “That allowed us the freedom to evolve their gait rapidly.”

Though Nemiroski and colleagues were able to replicate ants’ distinctive “triangle” gait using their six-legged robot, duplicating a spider-like gait proved far trickier.

“A spider has the ability to modulate the speed at which it extends and contracts its joints to carefully time which limbs are moving forward or backward at any moment,” Nemiroski said. “But in our case, the joints’ motion is binary due to the simplicity of our valving system. Either you switch the valve to the pressure source to inflate the balloon in the joint, and thus extend the limb, or you switch the valve to atmosphere to deflate the joint and thus retract the limb. So in the case of the eight-legged robot, we had to develop our own gait compatible with the binary motion of our joints. I’m sure it’s not a brand-new gait, but we could not duplicate precisely how a spider moves for this robot.”

Developing a system that can fine-tune the speed of actuation of the legs, Nemiroski said, would be a useful goal for future research, and would require programmable control over the flow rate supplied to each joint.

“We hit that limitation in the system, which I’m actually pretty proud of, because it means we pushed it to its absolute limit,” he said. “We took the basic concept and asked how far can we go before we would have to make radical alterations to how these limbs work, and we found that limit at the eight-legged robot. We were able to make it walk, but if you wanted to make it walk faster, or to add more limbs — for example, to support a load — you would have to start rethinking the system from the ground up.”

Though it may be years before the robots find their way into real-world applications, Whitesides believes the techniques used in their development — particularly the use of everyday, off-the-shelf materials — can point the way toward future innovations.

“I don’t see any reason to reinvent wheels,” he said. “If you look at drinking straws, they can make them at, effectively, zero cost and with great strength, so why not use them? These are academic prototypes, so they’re very light weight, but it would be fairly easy to imagine building these with a lightweight structural polymer that could hold a substantial weight.”

“What’s really attractive here is the simplicity,” added Nemiroski. “This is something George has been championing for some time, and something I grew to appreciate deeply while I was in his lab. For all the complexity of movement and structural integrity we get out of these robots, they’re remarkably simple in terms of construction and control. Using a single, easy-to-find material and a single concept for an actuator, we could achieve complex, multidimensional motion.”

This post was originally published on The Harvard Gazette. Click here to view the original.

This research was supported with funding from the U.S. Department of Energy, DARPA, the Natural Sciences and Engineering Research Council of Canada, the National Science Foundation, the Swedish Research Council, and the Wyss Institute for Biologically Inspired Engineering at Harvard University.

The device named “Spark” flew high above the man on stage with his hands waving in the direction of the flying object. In a demonstration of DJI’s newest drone, the audience marveled at the Coke can-sized device’s most compelling feature: gesture controls. Instead of a traditional remote control, this flying selfie machine follows hand movements across the sky. Gestures are the most innate language of mammals, and including robots in our primal movements means we have reached a new milestone of co-existence.

Madeline Gannon of Carnegie Mellon University is the designer of Mimus, a new gesture controlled robot featured in an art installation at The Design Museum in London, England. Gannon explained: “In developing Mimus, we found a way to use the robot’s body language as a medium for cultivating empathy between museum-goers and a piece of industrial machinery. Body language is a primitive yet fluid means of communication that can broadcast an innate understanding of the behaviors, kinematics and limitations of an unfamiliar machine.” Gannon wrote about her experiences recently in the design magazine Dezeen: “In a town like Pittsburgh, where crossing paths with a driverless car is now an everyday occurrence, there is still no way for a pedestrian to read the intentions of the vehicle…it is critical that we design more effective ways of interacting and communicating with them.”

So far, the biggest commercially deployed advances in human-robot interactions have been conversational agents by Amazon, Google and Apple. While natural language processing has broken new ground in artificial intelligence, the social science of its acceptability in our lives might be its biggest accomplishment. Japanese roboticist Masahiro Mori described the danger of making computer generated voices too indistinguishable from humans as the “uncanny valley.” Mori cautioned inventors from building robots that are too human sounding (and possibly looking) as the result elicits negative emotions best described as “creepy” and “disturbing.”

Recently, many toys have embraced conversational agents as a way of building greater bonds and increasing the longevity of play with kids. Barbie’s digital speech scientist, Brian Langner of ToyTalk, detailed his experiences with crossing into the “Uncanny Valley” as: “Jarring is the way I would put it. When the machine gets some of those things correct, people tend to expect that it will get everything correct.”

Kate Darling of MIT’s Media Lab, whose research centers on human-robot interactions, suggested that “if you get the balance right, people will like interacting with the robot, and will stop using it as a device and start using it as a social being.”

This logic inspired Israeli startup Intuition Robotics to create ElliQ—a bobbing head (eyeless) robot. The purpose of the animatronics is to break down barriers between its customer base of elderly patients and their phobias of technology. According to Intuition Robotics’ CEO, Dor Skuler, the range of motion coupled with a female voice helps create a bond between the device and its user. Don Norman, usability designer of ElliQ, said: “It looks like it has a face even though it doesn’t. That makes it feel approachable.”

Mayfield Robotics decided to add cute R2D2-like sounds to its newest robot, Kuri. Mayfield hired former Pixar designers Josh Morenstein and Nick Cronan of Branch Creative with the sole purpose of making Kuri more adorable. To accomplish this mission, Morenstein and Cronan gave Kuri eyes, but not a mouth as that would be, in their words “creepy.” Conan shares the challenges with designing the eyes: “Just by moving things a few millimeters, it went from looking like a dumb robot to a curious robot to a mean robot. It became a discussion of, how do we make something that’s always looking optimistic and open to listen to you?” Kuri has a remarkable similarity to Morenstein and Cronan’s former theatrical robot, EVA.

In the far extreme of making robots act and behave human, RealDoll has been promoting six thousand dollar sex robots. To many, RealDoll has crossed the “Uncanny Valley” of creepiness with sex dolls that look and talk like humans. In fact, there is a growing grassroots campaign to ban RealDoll’s products globally, as it endangers the very essence of human relationships. Florence Gildea writes on the organization’s blog: “The personalities and voices that doll owners project onto their dolls is pertinent for how sex robots may develop, given that sex doll companies like RealDoll are working on installing increasing AI capacities in their dolls and the expectation that owners will be able to customize their robots’ personalities.” The example given is how the doll expresses her “feelings” for her owner on Twitter:

Obviously a robot companion has no feelings, however it is a projection of the doll owners’. “To anthropomorphize their dolls to sustain the fantasy that they have feelings for the owner. The Twitter accounts seemingly manifest the dolls’ independent existence so that their dependence on their owners can seem to signify their emotional attachment, rather than it following inevitable from their status as objects. Immobility, then, can be misread as fidelity and devotion.” The implications of this behavior is that their female companion, albeit mechanical, enjoys “being dominated.” The fear that the Campaign Against Sex Robots expresses is the objectification of women (even robotic ones) reinforces problematic human sexual stereotypes.

Today, with technology at our fingertips, there is growing phenomena of preferring one-directional device relationships over complicated human encounters. MIT Social Sciences Professor Sherry Turkle writes in her essay, Close Engagements With Artificial Companionship, that “over-stressed, overworked, people claim exhaustion and overload. These days people will admit they’d rather leave a voice mail or send an email than talk face-to-face. And from there, they say: ‘I’d rather talk to the robot. Friends can be exhausting. The robot will always be there for me. And whenever I’m done, I can walk away.’”

In the coming years humans will communicate more with robots in their lives from experiences in the home to the office to their leisure time. The big question will be not the technical barriers, but the societal norms that will evolve to accept Earth’s newest species.

“What do we think a robot is?” asked robot designer Norm. “Some people think it should look like an animal or a person, and it should move around. Or it just has to be smart, sense the environment, and have motors and controllers.”

Norm’s answer, like beauty, could be in the eye of the beholder.

A researcher at the University of the West of England (UWE Bristol) is developing a bio-inspired ‘smart’ knee joint for prosthetic lower limbs. Dr Appolinaire Etoundi, based at Bristol Robotics Laboratory, is leading the research and will analyse the functions, features and mechanisms of the human knee in order to translate this information into a new bio-inspired procedure for designing prosthetics.

Dr Etoundi gained his PhD in bio-inspired technologies from the University of Bristol where he developed a design procedure for humanoid robotic knee joints.  He is now turning his attention to nature, a growing area in robotics known as Bio-mimicry, combining curiosity about how biological systems work with solving complex engineering problems, in order to develop a prototype smart knee joint for prosthetics.

Andy Lewis, a Paralympic Triathlon Gold Medallist (Rio 2016), who wears a lower limb prosthetic, will try out the new joint once developed, to compare its energy consumption and gait efficiency to current prosthetics. There are currently approximately 100,000 knee replacement operations performed every year in the UK.  Lower limb amputation has a profound effect on daily life, and prosthesis must be comfortable and adapted to people so they can maintain daily activities such as walking and running.

Looking for inspiration in nature, Dr Etoundi will examine how the human knee works, as well as looking closely at the design of knee replacements used in surgery and at current knee joints in prosthetic limbs.  These three areas of knowledge will inform a procedure for designing a knee that could give greater, more responsive movement, while offering the control and intelligence that comes from robotics.

Dr Etoundi says, “I have spent years designing knee joints for humanoid robots, but the human knee has evolved over millions of years and is incredibly successful.  The human knee is a very complex joint with ligaments, which guide the motion of the knee, and bones that perform the motion.  Current mechanisms in prosthetic knees have a straightforward pin joint with ball bearings that does not have the sophisticated range of motion and stability of the human knee with its cruciate ligaments.

“The complex interaction between the soft tissue (ligaments) and the bones in the knee joint is an area that has yet to be replicated in prosthetics.  We need to understand this better in order to provide a better knee joint for people to use. I will study the different mechanisms within the knee joint and look for ways to translate its beneficial functionalities into a design concept for prosthetics.

“I want to create a prosthetic knee that will give the greatest range of motion with the least friction, enabling walking, climbing stairs, squatting and stability, while also offering important attributes of current prosthetics and the benefits of robotic technology.”

Andy Lewis, who will try out Dr Etoundi’s nature-inspired design says, “I was pleased when Appo approached me. He understands the importance of a good prosthetic for sports people, and it will be interesting to see what he discovers that might make a better prosthetic which is more responsive.  I am looking forward to seeing his early designs next year and trying them out.”

The research team includes Professor Richie Gill (University of Bath), Dr Ravi Vaidyanathan (Imperial College London) and Dr Michael Whitehouse (University of Bristol).

Dr Etoundi is a Senior Lecturer in Mechatronics at UWE Bristol and is a member of the Medical Robotics group at Bristol Robotics Laboratory, which looks at the application of robotic technology in human-controlled and surgical applications.

Hi, I’m Ben. I was a member of the team that developed a new walking mechanism, TrotBot, that we eventually scaled up to the size of a mini-van (you can read my original post here). Now, at DIYwalkers, I’ve posted plans for TrotBot Ver. 2, designed to handle the weight of the EV3 brick. As you can see in the following videos, we were able to improve TrotBot by borrowing some ideas from a galloping horse.

Just like lunges are more tiring by simply walking, robots with bumpy gaits require more power to walk. This may not be a problem at small scales, but as a robot’s weight increases, smoother gaits are required.  TrotBot Ver. 1 has somewhat of a bumpy gait, but as you can see in my video below, TrotBot’s weight at LEGO-scale is low enough that it walks well:

However, when I added LEGO’s relatively heavy EV3 brick to TrotBot it didn’t perform so well.  To reduce its power requirements, I needed to smooth TrotBot’s gait, which I did by adding active feet that mimic the leg action of a galloping horse.

Background on TrotBot and its Feet

Our goal for TrotBot was to create a walking mechanism capable of walking on rough terrain and areas generally inaccessible to wheeled vehicles, so we designed its linkage to step high. We initially prototyped TrotBot in LEGO with 8 legs.  It walked well enough that we—somewhat naively—thought we could scale it up to mini-van size with only 8 legs.  A team of us spent most of that summer’s boiling hot weekends building our large TrotBot in my garage, only to find, during our first walking test, that too much torque was required to drive the robot up from the gait’s low point.  As we discovered, large walkers should always have at least one foot in contact with the ground at each corner, like how a car’s four wheels are always in contact with the ground. Below, I simulated one corner of a 12 leg version of TrotBot, and as you can see we should have scaled up TrotBot with 12 legs like Theo Jansen’s famous Strandbeest has:

It would have required too many new parts to switch to a 12 leg version of TrotBot, and we didn’t want to start over from scratch, so instead we explored ideas for active feet that could smooth TrotBot’s 8 legged gait.

Using a galloping horse’s gait as inspiration we sought to add some sort of second foot to each leg that would mimic how a horse’s rear then front legs land in pairs. This resulted in the additional linkage that we call TrotBot’s “heel” which increased TrotBot’s foot-contact with the ground by about 10%, reduced how much the feet skidded, and increased the step-height of TrotBot’s rear legs. Shown below is a video comparing TrotBot with these “heels” to a galloping horse:

Next, we explored adding some sort of toe that would push down on the ground as the foot begins to lift, just like how humans use their toes to walk. We installed one of these toe ideas on our large TrotBot but they didn’t smooth the gait enough, and since they were attached to the legs at a fixed angle they tended to catch on obstacles. Catching on obstacles occasionally caused the linkage to lock and gears to grind. In other words, this toe compromised our main goal of creating a mechanism that could walk on rough terrain! Here’s a photo of that toe:

TrotBot’s fixed angle toe

Looking again to a galloping horse for inspiration we started to experiment with linkages that mimicked how horses paw their hooves backward and then keep them folded back as they lift their legs to strike the ground again. We discovered a few options that mimicked this action, and they smoothed the gait by increasing TrotBot’s foot-contact by another 10% while maintaining its high foot-path. I added one of these toe options to TrotBot Ver. 2, and while I couldn’t make a toe with accurate dimensions using LEGO’s integer-based beams, it still smoothes TrotBot’s gait significantly:

Click here for instructions on how to build your own TrotBot Ver. 2!

Image courtesy of flora robotica, Photo by Anders Ingvartsen, CITA

Robots and plants are being intricately linked into a new type of living technology that its creators believe could be used to grow a house.

‘The growth is for free, but we have to control the plants to grow in the shapes we want,’ explained Professor Heiko Hamann, from the University of Lübeck in Germany, who coordinates the EU-funded flora robotica project. The plants grow through a network of sensors, computers and 3D-printed robotic nodes that are connected to each other and constantly monitor the plants. The team uses a white plastic scaffolding with black strips woven into it to guide the growth. The strips contain LED lights and sensors that can cause plants to grow into pre-programed shapes.

Plastic white scaffolding helps hold up plant-robot hybrids until they are strong enough to support themselves. Image courtesy of flora robotica

Once the plants are strong enough, the scaffolding is removed piece by piece and the hybrids can stand and support themselves. The idea is to create a society of plant-robot hybrids that functions as a self-organising system. As the system grows over long periods in interaction with humans, the research team hopes it will form meaningful architectural forms, including eventually a house.

Prof. Hamann believes a full house will take 40 years to grow, but his team is already starting to create parks where people can design their own plants by mixing seeds, scaffolds and electronics. The techniques required to create the plant-robot hybrids are already well advanced. ‘They can talk to each other … and they already know the pattern that they’re supposed to make, and they can decide which lights should turn on to grow the shape that you actually want,’ he said.

Images courtesy of flora robotica

The robots make use of the fact that plants respond differently to red and blue light. To test this out, researchers put coloured lights around bean plants and monitored their growth.

‘The blue light influences where the plant grows,’ explained Prof. Hamann.

The team is also testing out how to weave different braided structures. Some are loose and let light inside for plants to grow within the structure, while others are woven more tightly making the scaffold stronger for plants and sensors to sit outside.

Image courtesy of flora robotica

The different tightnesses of the braids could also help researchers make bendable arms that are controlled by robots to move side to side and extend and retract – similar to the way a puppet’s arms move when someone pulls a string.

The researchers hope that the arm will be able to interact with people, for example when someone walks by. It’s part of their overall goal to ‘bring technology and nature together’. The idea is that robots will work with people around them, for example by not growing in a place where people often walk by, or telling a passer-by that a plant needs more water.

‘The plant system should interact with human beings,’ explained Prof. Hamann.

Image courtesy of flora robotica

In this episode, Audrow Nash interviews Bradley Knox, founder of bots_alive. Knox speaks about an add-on to a Hexbug, a six-legged robotic toy, that makes the bot behave more like a character. They discuss the novel way Knox uses machine learning to create a sense character. They also discuss the limitation of technology to emulate living creatures, and how the bots_alive robot was built within these limitations.

Brad Knox

Dr. Bradley Knox is the founder of bots_alive. He researched human-robot interaction, interactive machine learning, and artificial intelligence at the MIT Media Lab and at UT Austin. At MIT, he designed and taught Interactive Machine Learning. He has won two best paper awards at major robotics and AI conferences, was awarded best dissertation from UT Austin’s Computer Science Department, and was named to IEEE’s AI’s 10 to Watch in 2013.

Links

• bots_alive Kickstarter page
• Download mp3 (16.8 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Chances are that you’ve never given much thought to how insects walk, or what combination of leg movements–or gaits–is most stable or fastest, but, if like a group of scientists from Ramdya, Floreano and Ijspeert labs, NCCR Robotics, you are trying to create fast and robust robots, taking inspiration some of nature’s most agile movers might give you just the inspiration you need.

Multiple insect species walk fastest using a tripod gait, that means that three “feet” are on the ground and three are moving at any given time. Because insects find food and mates by walking around on leaves and climbing up trees, it has long been thought that evolutionary processes discovered the tripod gait to best achieve this kind of climbing over challenging terrain. In contrast, when moving quickly, quadruped vertebrates like dogs and horses tend to only keep one or two legs on the ground at a time. This difference in fast locomotor strategies between insects and vertebrates was puzzling to the researchers and suggested that the tripod gait might not be the best to use in ground-based bioinspired six-legged robots. To test this possibility, the team used computer simulations to model the running gaits of the fruit fly, Drosophila melanogaster. The team then used a Particle Swarm Optimization (PSO) algorithm on the in-silico insect model to discover optimally fast gaits in different environmental scenarios and configurations (e.g. climbing, walking on flat ground).

Indeed, they discovered that the tripod gait is best used for fast climbing while new, unexpected bipod gaits with two legs on the ground were faster during ground locomotion – the typical requirement of six-legged robots. To move beyond the simulation, the then tested these new bipod gaits in a hexapod robot. They found that it also moved faster than when using the tripod gait. Finally, the team discovered that when leg adhesion was blocked in real Drosophila melanogaster, insects used gaits more similar to those discovered in the simulation. These results reveal the importance of environmental and biomechanical context in designing locomotor control strategies. Bioinspired methods are finely tuned to the biological challenges they address and may not be appropriate for all robotics applications.

Why does this interest roboticists? In creating previous bioinspired robots, the group from NCCR Robotics have developed robots that are capable of accessing multiple types of environments to locate victims and better focus rescue efforts. One failing of legged robots thus far, however, is that they are not quick as their biological, or flying, counterparts. By discovering how walking robots can more efficiently move, they can make the robots more robust and also reduce the energy requirements for a mission.

Reference

P. Ramdya, R. Thandiackal, R. Cherney, T. Asselborn, R. Benton, A. Ijspeert, and D. Floreano. Climbing favors the tripod gait over alternative faster insect gaits. Nature Communications. DOI 10.1038/NCOMMS14494.

If you enjoyed this article, you might also be interested in:

• QuadBot: The 3D printed crab-bot scuttling beyond its Kickstarter goal
• Design, simulate and build a custom drone
• A new quadrupedal robot, the ANYmal
• Make your own Dtto Modular Robot – Assembly instructions (Part 1 of 3)
• mROBerTO: The modular millirobot for swarm behavior studies

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

“Hydrogels are soft, wet, biocompatible, and can form more friendly interfaces with human organs,” says Xuanhe Zhao, associate professor of mechanical engineering and civil and environmental engineering at MIT. Photo: Hyunwoo Yuk/MIT Soft Active Materials Lab

Engineers at MIT have fabricated transparent, gel-based robots that move when water is pumped in and out of them. The bots can perform a number of fast, forceful tasks, including kicking a ball underwater, and grabbing and releasing a live fish.

The robots are made entirely of hydrogel — a tough, rubbery, nearly transparent material that’s composed mostly of water. Each robot is an assemblage of hollow, precisely designed hydrogel structures, connected to rubbery tubes. When the researchers pump water into the hydrogel robots, the structures quickly inflate in orientations that enable the bots to curl up or stretch out.

The team fashioned several hydrogel robots, including a finlike structure that flaps back and forth, an articulated appendage that makes kicking motions, and a soft, hand-shaped robot that can squeeze and relax.

Because the robots are both powered by and made almost entirely of water, they have similar visual and acoustic properties to water. The researchers propose that these robots, if designed for underwater applications, may be virtually invisible.

“Hydrogels are soft, wet, biocompatible, and can form more friendly interfaces with human organs,” Zhao says. “We are actively collaborating with medical groups to translate this system into soft manipulators such as hydrogel ‘hands,’ which could potentially apply more gentle manipulations to tissues and organs in surgical operations.”

Zhao and Yuk have published their results this week in the journal Nature Communications. Their co-authors include MIT graduate students Shaoting Lin and Chu Ma, postdoc Mahdi Takaffoli, and associate professor of mechanical engineering Nicholas X. Fang.

Robot recipe

For the past five years, Zhao’s group has been developing “recipes” for hydrogels, mixing solutions of polymers and water, and using techniques they invented to fabricate tough yet highly stretchable materials. They have also developed ways to glue these hydrogels to various surfaces such as glass, metal, ceramic, and rubber, creating extremely strong bonds that resist peeling.

The team realized that such durable, flexible, strongly bondable hydrogels might be ideal materials for use in soft robotics. Many groups have designed soft robots from rubbers like silicones, but Zhao points out that such materials are not as biocompatible as hydrogels. As hydrogels are mostly composed of water, he says, they are naturally safer to use in a biomedical setting. And while others have attempted to fashion robots out of hydrogels, their solutions have resulted in brittle, relatively inflexible materials that crack or burst with repeated use.

In contrast, Zhao’s group found its formulations leant themselves well to soft robotics.

“We didn’t think of this kind of [soft robotics] project initially, but realized maybe our expertise can be crucial to translating these jellies as robust actuators and robotic structures,” Yuk says.

Fast and forceful

To apply their hydrogel materials to soft robotics, the researchers first looked to the animal world. They concentrated in particular on leptocephali, or glass eels — tiny, transparent, hydrogel-like eel larvae that hatch in the ocean and eventually migrate to their natural river habitats.

“It is extremely long travel, and there is no means of protection,” Yuk says. “It seems they tried to evolve into a transparent form as an efficient camouflage tactic. And we wanted to achieve a similar level of transparency, force, and speed.”

To do so, Yuk and Zhao used 3-D printing and laser cutting techniques to print their hydrogel recipes into robotic structures and other hollow units, which they bonded to small, rubbery tubes that are connected to external pumps.

To actuate, or move, the structures, the team used syringe pumps to inject water through the hollow structures, enabling them to quickly curl or stretch, depending on the overall configuration of the robots.

Yuk and Zhao found that by pumping water in, they could produce fast, forceful reactions, enabling a hydrogel robot to generate a few newtons of force in one second. For perspective, other researchers have activated similar hydrogel robots by simple osmosis, letting water naturally seep into structures — a slow process that creates millinewton forces over several minutes or hours.

Catch and release

In experiments using several hydrogel robot designs, the team found the structures were able to withstand repeated use of up to 1,000 cycles without rupturing or tearing. They also found that each design, placed underwater against colored backgrounds, appeared almost entirely camouflaged. The group measured the acoustic and optical properties of the hydrogel robots, and found them to be nearly equal to that of water, unlike rubber and other commonly used materials in soft robotics.

In a striking demonstration of the technology, the team fabricated a hand-like robotic gripper and pumped water in and out of its “fingers” to make the hand open and close. The researchers submerged the gripper in a tank with a goldfish and showed that as the fish swam past, the gripper was strong and fast enough to close around the fish.

“[The robot] is almost transparent, very hard to see,” Zhao says. “When you release the fish, it’s quite happy because [the robot] is soft and doesn’t damage the fish. Imagine a hard robotic hand would probably squash the fish.”

Next, the researchers plan to identify specific applications for hydrogel robotics, as well as tailor their recipes to particular uses. For example, medical applications might not require completely transparent structures, while other applications may need certain parts of a robot to be stiffer than others.

“We want to pinpoint a realistic application and optimize the material to achieve something impactful,” Yuk says. “To our best knowledge, this is the first demonstration of hydrogel pressure-based acutuation. We are now tossing this concept out as an open question, to say, ‘Let’s play with this.’”

This research was supported, in part, by the Office of Naval Research, the MIT Institute for Soldier Nanotechnologies, and the National Science Foundation.

You might also enjoy the following articles:

• Wearable AI that can detect the tone of a conversation
• Soft exosuit economies: Understanding the costs of lightening the load
• Students develop cheap 3D printing technique for soft robotics
• MIT Media Lab to participate in $27 million initiative on AI ethics and governance

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Samsung Simband. Image: Jason Dorfman, MIT CSAIL

It’s a fact of nature that a single conversation can be interpreted in very different ways. For people with anxiety or conditions like Asperger’s, this can make social situations extremely stressful. But what if there was a more objective way to measure and understand our interactions?

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) say that they’ve gotten closer to a potential solution: an artificially intelligent wearable system that can predict if a conversation is happy, sad or neutral based on a person’s speech patterns and vitals.

“Imagine if, at the end of a conversation, you could rewind it and see the moments when the people around you felt the most anxious,” says graduate student Tuka Alhanai, who co-authored a related paper with PhD candidate Mohammad Ghassemi that they will present at next week’s Association for the Advancement of Artificial Intelligence (AAAI) conference in San Francisco. “Our work is a step in this direction, suggesting that we may not be that far away from a world where people can have an AI social coach right in their pocket.”

As a participant tells a story, the system can analyze audio, text transcriptions and physiological signals to determine the overall tone of the story with 83 percent accuracy. Using deep-learning techniques, the system can also provide a specific sentiment score for five-second intervals within a conversation.

“As far as we know, this is the first experiment that collects both physical data and speech data in a passive but robust way, even while subjects are having natural, unstructured interactions,” says Ghassemi. “Our results show that it’s possible to classify the emotional tone of conversations in real-time.”

The team is keen to point out that they developed the system with privacy strongly in mind: the algorithm runs locally on a user’s device as a way of protecting personal information. (Alhanai says that a consumer version would obviously need clear protocols for getting consent from the people involved in the conversations.)

How it works

Past studies in this area would often show participants “happy” and “sad” videos, or ask them to artificially act out specific emotive states. But in an effort to elicit more organic emotions, the team instead asked subjects to tell a happy or sad story of their own choosing.

Subjects wore a Samsung Simband, a device that captures high-resolution physiological waveforms to measure features like movement, heart rate, blood pressure, blood flow and skin temperature,. The system also captured audio data and text transcripts to analyze the speaker’s tone, pitch, energy, and vocabulary.

Mohammad Ghassemi and Tuka Alhanai converse with the wearable. Image: Jason Dorfman MIT CSAIL

“The team’s usage of consumer market devices for collecting physiological data and speech data shows how close we are to having such tools in everyday devices,” says Björn Schuller, professor and chair of Complex and Intelligent Systems at the University of Passau in Germany who was not involved in the research. “Technology could soon feel much more emotionally intelligent, or even ‘emotional’ itself.”

After capturing 31 different conversations of several minutes each, the team trained two algorithms on the data: one classified the overall nature of a conversation as either happy or sad, while the second classified each five-second block of every conversation as either positive, negative or neutral.

Alhanai notes that, in traditional neural networks, all features about the data are provided to the algorithm at the base of the network. In contrast, her team found that they could improve performance by organizing different features at the various layers of the network.

“The system picks up on how, for example, the sentiment in the text transcription was more abstract than the raw accelerometer data, says Alhanai. “It’s quite remarkable that a machine could approximate how we humans perceive these interactions, without significant input from us as researchers.”

Results

Indeed, the algorithm’s findings align well with what we humans might expect to observe. For instance, long pauses and a monotonous vocal tones were associated with sadder stories, while more energetic, varied speech patterns were associated with happier ones. In terms of body language, sadder stories were also strongly associated with increased fidgeting and cardiovascular activity, as well as certain postures like putting one’s hands on one’s face.

Graph showing real-time emotion detection. Image: MIT CSAIL

On average, the model could classify the mood of each five-second interval with an accuracy that was approximately 18 percent above chance, and a full 7.5 percent better than existing approaches. In future work, the team hopes to collect data on a much larger scale, potentially using commercial devices like the Apple Watch that would allow them to more easily deploy the system out in the world.

“Our next step is to improve the algorithm’s emotional granularity so it can call out boring, tense, and excited moments with greater accuracy instead of just labeling interactions as ‘positive’ or ‘negative’,” says Alhani. “Developing technology that can take the pulse of human emotions has the potential to dramatically improve how we communicate with each other.”

Click here to download the research paper.

You might also enjoy the following articles:

• Artificial intelligence and ethics: Who does the thinking?
• Soft exosuit economies: Understanding the costs of lightening the load
• MIT Media Lab to participate in $27 million initiative on AI ethics and governance
• How artificial intelligence is changing our Christmas shop
• White House report: Artificial intelligence, automation, and the economy

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The Harvard team’s exosuit is a wearable robot that directly targets the ankle joints and is made of functional textiles, cable-based actuation and a biologically-inspired control system. In this version, actuation, electronics and battery units (shown on the left) have been off-boarded to better allow the researchers to quantify the exosuit’s energy-saving potential in healthy wearers. Credit: Wyss Institute at Harvard University

Last year, Harvard’s soft exosuit team provided first proof-of-concept results showing that its wearable robot could lower energy expenditure in healthy people walking with a load on their back. Made of functional textiles, cable-based actuation and a biologically-inspired control system, the exosuit targets specific leg joints – instead of the full leg – and delivers assistance that is synchronized with the wearer’s walking mechanics in individuals ranging from people that carry heavy loads to people that are disabled by stroke or other health complications.

The team is led by Conor Walsh, a Wyss Institute Core Faculty member, John L. Loeb Associate Professor of Engineering and Applied Sciences at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) and the Founder of the Harvard Biodesign Lab.

But the question still remained. What is the effect of increasing assistance levels on the overall energy savings to wearers? These savings are the result of the energy spared due to the assistive force provided to the ankle motions, and this must be balanced with the additional energy expended by the wearer when carrying the exosuit’s mass.

Now, in a new study published as the cover article in Science Robotics, Walsh’s multidisciplinary team of roboticists, engineers, biomechanics experts and apparel designers isolated the beneficial effects of the exosuit. By removing the actuation, electronics and battery units, leaving only the exosuit’s light wearable textiles and pulling cables, the researchers were able to calculate the impact on the wearers’ joints. The study is selected as the cover article of the issue.

“In a test group of seven healthy wearers, we clearly saw that the more assistance provided to the ankle joints, the more energy the wearers could save with a maximum reduction of almost 23% compared to walking with the exosuit powered-off,” said Walsh. “To our knowledge, this is the highest relative reduction in energy expenditure observed to date with a tethered exoskeleton or exosuit.”

Interestingly, the team found that wearers significantly adapted their gait with increasing levels of assistance. The changes were most significant at the ankle joint but also at the hip as the exosuit included straps coupling the assistance from the back of the lower legs to the front of the hip in a beneficial manner.

“Other studies had reported that there can be an energy transfer between the ankle and other joints. However, by having a textile couple the ankle and hip with our soft exosuit, may have amplified this effect, contributing to the considerable energy savings we found,” said Brendan Quinlivan, a graduate student working with Walsh and one of the two first-authors on the study.

The exosuit’s soft textiles strategically position the actuating cables that assist with the ankle motion at the back of the lower legs and, through additional straps, transfer energy produced at the ankle to the front of the hip to also assist with the gait’s hip motion. Credit: Wyss Institute at Harvard University

The authors acknowledge that future studies are now required to compare the effects of fully body-worn versions of the exosuit, with the weight of the actuation and battery carried by the wearer, to walking with normal clothes to better quantify the true impact of the technology.

“This study represents another major step forward along a path that will bring this new type of ‘wearable robot’ into the marketplace where it will help soldiers and workers, as well as patients with disabilities,” said Wyss Institute Founding Director Donald Ingber, M.D., Ph.D., who is also the Judah Folkman Professor of Vascular Biology at Harvard Medical School and Professor of Bioengineering at Harvard SEAS.“The results from this study will help us find a sweet spot between the added system weight and the amount of force applied to achieve maximum benefits in energy expenditure in wearers. Defining these interdependencies in healthy people will ultimately also help us design versions suitable for people with gait abnormalities,” said Sangjun Lee, who also is a graduate student in Walsh’s team and a co-first author of the study.

The multi-disciplinary study was supported by the Wyss Institute for Biologically Inspired Engineering, the Harvard John A. Paulson School of Engineering and Applied Sciences, the Defense Advanced Research Projects Agency (DARPA)’s Warrior Web Program, the National Science Foundation, a Samsung Scholarship, the São Paulo Research Foundation and the Robert Bosch Stiftung.

If you enjoyed this article, you may also enjoy:

• Indego joins Ekso with FDA exoskeleton approvals
• Bioinspired robotics #3: Wearables, with Conor Walsh
• Robots Podcast #216: ExoAtlet: Exoskeleton for Rehabilitation, with Ekaterina Bereziy
• Exoskeletons: From helping people walk to controlling robots in space
• Are skintight suits the future of robotic exoskeletons?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Fabrication and complete assembly of a Geneva drive device using the iMEMS method. The left panel shows the layer-by-layer fabrication of support structures and assembly of gear components. The image on the right shows the complete device after the layers have been sealed. Image: SauYin Chin, Columbia Engineering.

A team of researchers led by Biomedical Engineering Professor Sam Sia at Columbia Engineering has developed a way to manufacture microscale machines from biomaterials that can safely be implanted in the body. Working with hydrogels, which are biocompatible materials that engineers have been studying for decades, Sia has invented a new technique that stacks the soft material in layers to make devices that have three-dimensional, freely moving parts. The study, published online January 4, 2017, in Science Robotics, demonstrates a fast manufacturing method Sia calls “implantable microelectromechanical systems” (iMEMS).

By exploiting the unique mechanical properties of hydrogels, the researchers developed a “locking mechanism” for precise actuation and movement of freely moving parts, which can function as valves, manifolds, rotors, pumps, and drug delivery systems. They were able to tune the biomaterials within a wide range of mechanical and diffusive properties and to control them after implantation without a sustained power supply, such as a toxic battery. They then tested the payload delivery in a bone cancer model and found that the triggering of releases of doxorubicin from the device over 10 days showed high treatment efficacy and low toxicity, at 1/10^th of the standard systemic chemotherapy dose.

“Overall, our iMEMS platform enables development of biocompatible implantable microdevices with a wide range of intricate moving components that can be wirelessly controlled on demand and solves issues of device powering and biocompatibility,” says Sia, also a member of the Data Science Institute. “We’re really excited about this because we’ve been able to connect the world of biomaterials with that of complex, elaborate medical devices.  Our platform has a large number of potential applications, including the drug delivery system demonstrated in our paper which is linked to providing tailored drug doses for precision medicine.”

Most current implantable microdevices have static components rather than moving parts and, because they require batteries or other toxic electronics, have limited biocompatibility. Sia’s team spent more than eight years working on how to solve this problem. “Hydrogels are difficult to work with, as they are soft and not compatible with traditional machining techniques,” says Sau Yin Chin, lead author of the study, who worked with Sia. “We have tuned the mechanical properties and carefully matched the stiffness of structures that come in contact with each other within the device. Gears that interlock have to be stiff in order to allow for force transmission and to withstand repeated actuation. Conversely, structures that form locking mechanisms have to be soft and flexible to allow for the gears to slip by them during actuation, while at the same time they have to be stiff enough to hold the gears in place when the device is not actuated. We also studied the diffusive properties of the hydrogels to ensure that the loaded drugs do not easily diffuse through the hydrogel layers.”

The team used light to polymerize sheets of gel and incorporated a stepper mechanization to control the z-axis and pattern the sheets layer by layer, giving them three-dimensionality. Controlling the z-axis enabled the researchers to create composite structures within one layer of the hydrogel while managing the thickness of each layer throughout the fabrication process. They were able to stack multiple layers that are precisely aligned and, because they could polymerize a layer at a time, one right after the other, the complex structure was built in under 30 minutes.

Sia’s iMEMS technique addresses several fundamental considerations in building biocompatible microdevices, micromachines, and microrobots: how to power small robotic devices without using toxic batteries; how to make small biocompatible moveable components that are not silicon, which has limited biocompatibility; and how to communicate wirelessly once implanted (radio frequency microelectronics require power, are relatively large, and are not biocompatible). The researchers were able to trigger the iMEMS device to release additional payloads over days to weeks after implantation. They were also able to achieve precise actuation by using magnetic forces to induce gear movements that, in turn, bend structural beams made of hydrogels with highly tunable properties. (Magnetic iron particles are commonly used and are FDA-approved for human use as contrast agents.)

In collaboration with Francis Lee, an orthopedic surgeon at Columbia University Medical Center at the time of the study, the team tested the drug delivery system on mice with bone cancer. The iMEMS system delivered chemotherapy adjacent to the cancer, and limited tumor growth while showing less toxicity than chemotherapy administered throughout the body.

“These microscale components can be used for microelectromechanical systems, for larger devices ranging from drug delivery to catheters to cardiac pacemakers, and soft robotics,” notes Sia. “People are already making replacement tissues and now we can make small implantable devices, sensors, or robots that we can talk to wirelessly. Our iMEMS system could bring the field a step closer to developing soft miniaturized robots that can safely interact with humans and other living systems.”

The study, “Additive manufacturing of hydrogel-based materials for next-generation implantable medical devices,” was supported by an NSF CAREER award, NIH R01 grant (HL095477-05), and NSF ECCS-1509748. Chin was supported by the National Science Scholarship (PhD) awarded by the Agency for Science, Technology and Research (Singapore). The researchers have a patent pending.

If you enjoyed this article, you may also enjoy:

• The rise of China’s medical robotics sector
• Robots at your service: Empowering healthy aging at European Robotics Week, 2016
• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• Survey: Evaluate ethics of health related privacy with care robots
• Care-O-bot 4 celebrates its premiere as a shopping assistant
• Robot companions are coming into our homes — how human should they be?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Image: Eunike Nugroho

What might the first western explorer have thought upon encountering whales, flamingos and iguanas? What would have they thought of these strange near-fantastical creatures?

What is it like for 21st-century people to encounter robots for the first time? Are they friend or foe? Where do they come from and what controls them? If we saw one for the first time tomorrow, what would we tell our friends?

This is the thinking behind Robot Life Survey – an alternative history series from design company After the flood that visualises Boston Dynamic’s Big Dog, Prox Dynamics’ Black Hornet Nano, and Harvard University’s Kilobot Swarm as if witnessed by an early explorer. What is our relationship with these strange, futuristic creations?

Max Gadney, After the flood director, comments: “In this series, mechanical intelligence is discovered for the first time by a mystery character and recorded for posterity and science. This explorer records their wonder, alluding to a wider picture of and cast of players. It was a fantastic project to work on at After the flood, combining the passion for art and complex mechanics that underpins much of what we do.”

This series of the Survey is available in three prints – each presenting a record of the explorer’s experiences along with snatches of field notes. A larger book based on Robot Life Survey and its explorer will follow in 2017. This will visualise more robots, notes and thinking around this new world of physical, connected intelligence.

The voice of the explorer is provided by Lloyd Shepherd, author of historical thrillers The English Monster, The Poisoned Island and Savage Magic published by Simon and Schuster. Lloyd has spent many years studying the age of enlightenment, and with a background in the tech industry he is the best person to breathe life into the world of Robot Life Survey. The art itself is by Indonesia-based artist named Eunike Nugroho, who specialises in botanical and animal life illustrations and paintings.

Enjoy the “Explorer Notes” by Lloyd Shepherd below:

Big Dog

Image: Eunike Nugroho

“Of all the creatures I’d encountered on my voyages, this was the one that most disturbed me. It stepped silently through the trees, its queasily familiar legs quick-stepping through the snow, as it sought to self-correct its balance at each contact with the ground. On its back was an enormous hessian sack, tied to its body with rope, the property, no doubt, of some Siberian tribesman who had captured this thing and adapted its programming for his own banal ends. It would have come up to above my waist, had I not been sitting in a tree watching it pass, and it was the biggest single machine I had yet seen. But this is not what disturbed me about it. What gave me pause was the memory of Lovelace’s wolf-hound, a great grey beast of enormous gentleness which had plodded this changing Earth with the exact same care as this silent machine. The dog must have been long-dead by that time, but my heart softened at the memory of it, as I watched this machine make its considered way through those deep, silent, freezing woods.”

Nano UAV

Image: Eunike Nugroho

The noise made by one of these tiny machines must be almost inaudible, but the little swarm we encountered down in the valley made an angry sound in its buzzing aggregate. I had seen plans for machines such as these in Lovelace’s study, and had marvelled at how the elegant lines had been squeezed and folded into such tiny mechanisms. Each was barely five centimetres in length, and they must be light indeed to hover in the air so effortlessly, though it is some time since anyone has caught one. Their yellow-and-silver sides reflected the harsh Afghan sun, and as they moved away from us their buzz mellowed, putting me in mind of English summer gardens, though no bee I ever saw cast a shadow such as these machines did on the orange rock, and no singular purpose guides a bee’s flight in the way it seemed to hold these little things in check, as they flew away from us, in inscrutable formation, for a destination at which I could only guess.

Kilobot Swarm

Image: Eunike Nugroho

There were too many of the little machines to count, but there may have been as many as a thousand. The herd moved like oil gliding down a steel ramp, slowly and with its only apparent purpose being adjacency, a need to hold itself together, to cohere into a shape that was a circle, then a triangle, then a square, a puddle of technology which could not make up its mind. Each of the little machines was five centimetres across, and moved on little feet which tapped on the concrete, making a sound like light rain. All each of them knew of the world was the location of its neighbours, and a simple task held in the code of no more than a dozen of them. So was this a single organism, a slime-mold made of plastic and silicon, its intention rippling across tiny invisible networks? Was each device’s awareness of its neighbours the beginnings of a kind of consciousness? And if Lovelace were standing with me now, watching these little machines ripple across the ground, would those machines detect a presence, and stop, and turn to their Creator? No answer came, and on the mechanistic puddle slid, aware only of itself.

If you enjoyed this article, you might also be interested in:

• When robots become art: Interview with Yi-Wei Keng
• SOCIAL ROBOTICS JAPAN: Robot Town Sagami’s strange, yet surprisingly hopeful, anime promo
• Why watching Westworld’s robots should make us question ourselves
• The new Westworld: Humanizing the un-human, or dehumanizing humankind?
• Ex Machina: When Turing meets Bechdel test
• President Obama discusses artificial intelligence with Media Lab Director Joi Ito

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In this episode, Audrow Nash interviews Domenico Prattichizzo, Professor of Robotics at the University of Siena and Senior Scientist at the Istituto Italiano di Tecnologia in Genova in Italy, about a device for assisting people who have lost the use of one of their hands, for example by a stroke. The device is an extra finger that functions to press an object into the paralyzed hand so that it can be grasped.

A video of the device being used for various tasks is below:

https://www.youtube.com/watch?v=spAXDXLCsKw&list=PLfYZRHo5ca5asEeY8b66xJ1ZuX3SFlCpR&index=1

Domenico Prattichizzo

Since 2015, Prattichizzo has been a Full Professor at the University of Siena. From 2002 to 2015, he was an Associate Professor of Robotics at the University of Siena. Since 2009, Prattichizzo has been a Scientific Consultant at Istituto Italiano di Tecnoloogia, in Genova, Italy. In 1994, he was a Visiting Scientist at the MIT AI Lab. Prattichizzo  received his M.S. degree in Electronics Engineering and the Ph.D. degree in Robotics and Automation from the University of Pisa in 1991 and 1995, respectively.

Links

• The Sixth Finger project page
• Download mp3 (15.3 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Image: Festo

Ask a child to design a robot, and they’ll produce a drawing that looks a little like you or I—the parts may be gray and boxy, but it will have two arms, two legs, and a head (probably with an antenna coming out of the top). Starting from the beginning of robotics, the human form has seemed like an excellent starting point. One of the best places to draw inspiration for robotic design, however, is the kingdom of insects, arachnids, snails, and slugs.

Challenges inherent in the human form

Here’s a simple question: how do you walk? It’s not by putting one foot in front of another. You walk by falling, and then catching yourself. This is a really complex movement, which has been best described by mathematical models that describe the human body as an inverted pendulum. Therefore, it’s safe to say that walking upright is not an easy challenge for robots to master. A recent DARPA robotics challenge—showcasing the most advanced humanoid robots on Earth—still saw nearly half of the contestants fall flat on their faces.

Designing robots to imitate humans—or any mammal—is a huge challenge. Our gait, our musculature, even the ways we see the world are all vastly more complicated than any machine we’ve ever made. Insects are simpler creatures by contrast, and therefore are much easier to replicate.

The insectoid advantage

Here are a few reasons why a budding robot designer should study insects. First of all, insects get a lot of mobility out of small bodies with simple joints. Better still, they’re robust—a ladybug can fly into a window, bounce off, and keep going. Small, mobile, and durable—all of these qualities represent admirable characteristics in a practical robot.

Small robots can be produced easily en masse, which can also lend itself to useful applications. By mimicking the swarming behavior of a cockroach, the makers of a robot known as Velociroach hope to use their tiny robots as rescue workers. Small robots can easily crawl through the cracks and crevices of a collapsed building in order to locate survivors and notify rescue workers.

Similarly, small insect bodies equate to small insect brains. While this isn’t likely to be much help to an insect in an IQ test, it’s a great way for engineers to study the secrets of robot autonomy. Despite their small brains, insects can avoid predators and display complex behavior. Even a creature as simple as a maggot—with just 10,000 neurons—can still learn relatively complex tasks, and can actually be trained to a certain extent.

The metamorphosis from insects to robots

Insects are simple enough that a myriad of insect-inspired robots have already made it from the drawing board into real life. Researchers from Seoul University have made a robot, for example, that takes its cues from the water strider—it can literally jump on water.  Sensors based on light-sensing organs found in insects have been used to stabilize drones.  Other robots have demonstrated the ability to swarm like ants.

When designing a robot, it makes sense to copy simpler organisms first, and then move on to more complex organisms. Robotics engineers have already learned a great deal from insects, and this knowledge lends itself to advancements in both mechanics and artificial intelligence.

If you enjoyed this bio-inspired article, you might also be interested in:

• QuadBot: The 3D printed crab-bot scuttling beyond its Kickstarter goal
• Robohub roundtable: Robotic bee swarms from Black Mirror – what’s hype, what’s real?
• New insect-inspired vision strategy could hasten development of mini-drones
• A drone with insect-inspired folding wings
• mROBerTO: The modular millirobot for swarm behavior studies

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In this episode, Abate De Mey interviews Edward Neff, founder of SMAC Corporation. Mr. Neff discusses how breakthroughs in his company have allowed them to develop linear actuators compact enough to be used to actuate robotic fingers. Companies like Apple and Samsung push for the development of robotic fingers to perform lifelike tests on their phones.

The robotic fingers developed using SMAC’s linear servo motors simulate a button press on a screen by employing a strong downwards force, as well as small sideways forces tangential to the screen. This mimics a realistic way that people interact with their touchscreens. The robotic fingers can also gently bump into surfaces, pull, and push objects. Other applications for these fingers include electronic circuit board assembly, and screwing bottle caps onto bottles.

Edward Neff

Edward Neff grew up in Birmingham, Michigan (USA) and is a graduate of the University of Michigan. He has a career history of managing automation companies in Belgium and Japan. Mr. Neff is the founder of SMAC Corporation in Carlsbad, California. SMAC is a leading manufacturer of precision programmable electric actuators. The company is planning a 2018 IPO. Mr Neff has over 40 patents in robotics and serves as a Director in BioTech and Medical Device companies.

Links

• SMAC Website
• Robotic Finger using a SmartPhone (YouTube)
• Robotic Finger at the Tokyo Robot Show (YouTube)
• Robotic Finger VS Human (YouTube)
• Smart Assembly Video (YouTube)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Robotics is becoming more accessible for many people, but the complexities of legged robots mean they remain beyond the reach of most consumers. The complex mechanics, electronics and code algorithms make these robots much more difficult to simply get working. Four legged robots are especially difficult, they might seem very stable but simply lifting a leg up can cause the robot to fall.

QuadBot is an open-source, 3D printed robot that aims to change this.

QuadBot is suitable for makers of any skill level, it works straight out the box, and can be programmed using scratch based blocks. And when you’re ready to explore robotics further, every aspect of QuadBot can be changed, from the mechanical design to individual lines of Arduino code. Once you’ve mastered walking, you can add sensors to the board, and make QuadBot avoid obstacles, follow light and even dance to music!

QuadBot was released on Kickstarter and comes either as a bare bones (3D Print it yourself) kit, or as a full kit. Within the first week, QuadBot reached 100% of the £12,500 goal, and is nearly at £20,000 at the time of writing!

QuadBot is the creation of Jack Scott-Reeve and Josh Elijah, two graduates of Engineering who felt that Robotics Education needed a change. Now that it is funded (and with 5 weeks still left on the campaign), they can focus on improving the product for our early backers. This means adding goals like extra colours, sensor packs and improved learning resources. However, among many Kickstarter campaigns there’s a tendency to overstretch and under deliver, so Elijah and Reeve are letting the product sell on its base features rather than promising massive additions to the design.

Nevertheless, QuadBot has massive future potential for expansion and increased capability, whether using modular design to create QuadBot sensor modules or integrating a Raspberry Pi onto the board to allow Python coding and vision processing.

One emerging market for this technology is robotics in schools. Teaching programming is becoming more common in UK schools, as are products to augment learning like the BBC Microbit. QuadBot could potentially be integrated with existing products like Microbit and the Raspberry Pi, to bring practical robotics into schools. With large distributors like RS Components interested, we could soon see QuadBot being distributed internationally, and with a curriculum to go along with it.

At present it’s hard to say in what direction Elijah and Reeve will go with QuadBot but, with a fully funded Kickstarter already, it’s likely we’ll see many new iterations and designs in the near future!

Click here to visit the QuadBot website.

If you enjoyed this article, you might also be interested in:

• Design, simulate and build a custom drone
• A new quadrupedal robot, the ANYmal
• Make your own Dtto Modular Robot – Assembly instructions (Part 1 of 3)
• mROBerTO: The modular millirobot for swarm behavior studies

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In this roundtable edition, we watched the Black Mirror episode “Hated in the Nation” and asked our Robohub team members: with many institutions focused on developing aerial drone technology, and in light of the pressing reality of climate change and bee colony collapse, do we see robotic bees in our future? Would swarms of artificial insects even be desirable?

In her book “The Ultimate Quotable Einstein”, Alice Calaprice asserted that Einstein probably never said anything close to: “If the bee disappeared off the surface of the globe then man would only have four years of life left. No more bees, no more pollination, no more plants, no more animals, no more man.”

Still. Whoever said it, it’s a haunting prophecy. With climate change thought to be forcing colony collapse disorder and threatening the existence of many pollinators, might swarm robotics hold the key to humanity’s survival? Could these robot bees ever be hacked to spy and become weaponised? In our roundtable we discussed the technology in the Black Mirror episode “Hate the Nation” and some of the pitfalls.

Swarms of autonomous, 3D printed drone bees pollinating the nation. Hackable AI, vulnerable to attack. Flying tools turned aerial enemy. Do our experts think any of this could happen? Is this type of technology likely or even feasible?

Our team share their thoughts:

Sabine: What’s interesting is that not everything in the episode is entirely crazy. We do want to create technology that addresses a specific need. If there are fewer pollinators, we might want to create artificial pollinators, though not necessarily artificial bees, to help us along the path to addressing the issue. But I think the technology in Black Mirror is very far from something we can do today.

There was a project called RoboBees, that was Harvard-based, that was all about creating artificial bees for pollination—so exactly this scenario.  There were multiple parts and problems: how do you make very small flyers; how do you drive them in a controlled way; and how do they sustain enough energy. All these things are extremely hard to address. They’ve managed to make some of the world’s best miniature flyers that can flap their wings and take off, and are making some great steps in controlling them. But when I say control, I really mean turning left and right in a laboratory setting. If you were to put them outside in a windy environments, it’d be a completely different story, and they don’t really have the energy to go very far.

And then there’s the algorithms. It’s not just the bees in this episode, they’re all connected to hives and they have all these control algorithms implemented that allow them to find flowers and people with image processing. How do you get such large numbers of drones to communicate with each other and their hubs? There are multiple communication issues that are very hard to address. What is interesting is that [Black Mirror] put together these bees with the swarm approach, and the pollinate in a way that’s very bio-inspired.

Kassie: Does anyone think that this episode did a disservice to swarm robotics in that someone was able to hack into the mainframe and essentially attack and kill people? That’s a very real fear that people have with regards to robotics and AI.

Sabine: For years we had to answer the question: Have you read “Prey”? Is this likely to happen? And now it’s: Have you seen “Dark Mirror”?

Ioannis: In fairness, there is some justification for this, as opposed to questions about terminators and things.

Sabine: And it’s not the technology [in Black Mirror] that’s the problem, it’s the humans. The technology starts by being portrayed as a real positive. All the tech in this episode, like the autonomous cars, is actually really cool and helpful. It’s the people who screwed up, ultimately, and the system got hacked. So I think the key message is that we need to make the technology secure.

Ioannis: Did you think it was a little short-sighted in the way it portrayed swarm robotics? It sounds very inefficient to me to have a central control that’s required to monitor the flight of every bee. All the bad guy needed to do was hack the central control to manipulate every bee. A swarm is something different than a million drones. In the episode, it looks like a swarm, but in reality it’s actually a lot of remotely operated robots.

Sabine: That’s right. And if it was properly decentralised, it would have been harder to hack because you wouldn’t be downloading massive amounts of code on every one of these bees.

Ioannis: To make the swarm go from controlled to bad, the hacker could have introduced some kind of memory or code to enable the swarm to evolve rapidly, and from that point on it’s out of control. That would have been interesting. Instead, the hacker just uses the bees as lots of tele-operated robots. I found the episode a little disappointing in this aspect.

Kassie: If you’re doubting robotics in any way, this episode is perhaps something you could look at and think: “Oh my God, this is something that could happen.” But I took some positives from this [with robotics] then only negatives.

Sabine: I felt the autonomous cars were positive; similar to what you’d expect them to be like in 30 years or so.

Ioannis: They portray autonomous cars as being cool, also, which is one aspect. But the fact that the cars are autonomous is nothing [to the story], the characters just use them, which I think is quite cool actually. They ignore the fact that these could be anything futuristic which I think is good.

Kassie: It was almost a precursor to something that could actually happen soon, in that some people were still driving, but the people with money had autonomous systems. And it wasn’t portrayed as scary! So I agree that that was a good aspect.

Sabine: I think the point of Black Mirror is very often not the technology but that people misuse it, and they make a very good point about online bullying [in this episode].

Ioannis: Yes, and they do not portray online bullies in a stereotypical way. For example, there is a teacher character who is a very nice lady in reality, but online is something completely different. Sabine, I’d like to ask you what you think about the potential of a weaponized swarm for defense purposes?

Sabine: I think it’s a very bad idea to weaponize swarms. You have to think about such things as stability and payload, accounting for wind and conditions etc. How would you do that? And why use a swarm as opposed to another type of robot with another type of payload?

Ioannis: I don’t have any specific examples for swarm robotics, but in nature a lion is dangerous, and so is a mosquito. Both in different way. So there could be an application.

Sabine: In that sense, you can weaponize anything. It’s just so much harder to weaponize a swarm than a more capable system, at least today.

Ioannis: What did you think of the robo-bee design?

Sabine: I think it was pretty cool, but too complex. If you were to design a robo-bee you would design something like a mini-copter that would sit on a plant, rather than something that looked like a bee. Or you might have an agriculture robot that simply goes across fields and does the pollination. Although the nice thing about the bee design is that the infrastructure is still already in place—plants etc.

If you enjoyed this article, you may also want to read our other Roundtables:

• Robohub roundtable: Public perceptions of robotics and AI, what’s your pet peeve?
• Robohub roundtable: Why do robotics crowdfunding projects fail?
• Robohub roundtable: Cybathlon and advancements in prosthetics
• Robohub roundtable: Job loss through automation, Foxconn controversy
• Robohub roundtable: Why is it so difficult to define ‘robot’?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In this episode, Audrow Nash interviews Dieter Fox, Professor in the Department of Computer Science and Engineering at the University of Washington, about the 100/100 Computer Vision Tracking Challenge. This is a self-imposed challenge to understand 100% of the pixels in an image 100% of the time in video footage; this includes understanding semantic information. Such understanding would allow robots to assist humans more naturally in environments like a home kitchen, wet lab, or in disaster response. To accomplish this challenge, Fox discusses challenges which include modeling, tracking, and detecting articulated objects.

Fox’s Keynote at ICRA 2016 on the 100/100 tracking challenge:

Dieter Fox
Dieter Fox is a German roboticist and a Professor in the Department of Computer Science & Engineering at the University of Washington, Seattle. He is most notable for his contributions to several fields including robotics, artificial intelligence, machine learning, and ubiquitous computing. Together with Wolfram Burgard and Sebastian Thrun he is a co-author of the book Probabilistic Robotics.

Links

• Download mp3 (13.5 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

In this episode, Ron Vanderkley interviews Jürgen “Juxi” Leitner, a researcher at the ARC Centre of Excellence in Robots Vision in the Queensland University of Technology in Australia. Leitner speaks about a system being developed for the Google Lunar XPrize, called LunaRoo.

The Google Lunar challenge calls for privately funded spaceflight teams to land a robotic spacecraft on the Moon, travel 500 meters, and transmit back high-definition video and images. LunaRoo is a hopping robot designed to leap up to 20 meters in the Moon’s lower gravity. While in the air (well, not really), LunaRoo will use computer vision to track the movement of a rover, that is trying to drive 500 meters, navigate the Moon’s surface.

Below is a video showing some prototypes of LunaRoo.

Jürgen Leitner

Jürgen “Juxi“ Leitner is a researcher at the ARC Centre of Excellence in Robotic Vision. Before coming to Australia, he worked and studied at various research institutes in Europe and Japan. From 2010-2014, he worked in a robotics lab at the Dalle Molle Institute for AI (IDSIA) in Lugano, Switzerland. Prior to IDSIA, he worked at the European Space Agency‘s Advanced Concepts Team for a year.

Links:

• Download mp3 (8.3 MB)
• LunaRoo Proposal
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

In this episode, Audrow Nash interviews Hugh Herr, Director of the Biomechatronics Group at MIT. Herr talks about the accident that led to the amputation of both of his legs below the knee and how this shaped his rock climbing and academic career. Herr also discusses orthoses and exoskeletons developed by his research group, as well as the future of bionic technology.

Transcript below.

Among topics discussed is the lower-body exoskeleton below. This exoskeleton is the first lower-body exoskeleton to decreases the user’s energy expenditure when walking (measured by oxygen consumption). It works by assisting the calf muscle and using the body’s joints, rather than heavy mechanical joints. Hugh Herr says, “Subjects noticed that their legs felt heavier and awkward when they took the exoskeleton off.”

Below is a TED talk Hugh Herr gave in 2014. In this talk, Herr describes disability as a failure of technology, he discusses the design of his own prostheses and his lab’s research, and there is a dance performance by Christian Lightner and Adrianne Haslet-Davis, who lost her lower left leg in the Boston Marathon bombings.

Hugh Herr

His research program seeks to advance technologies that promise to accelerate the merging of body and machine, including device architectures that resemble the body’s musculoskeletal design, actuator technologies that behave like muscle, and control methodologies that exploit principles of biological movement. His methods encompass a diverse set of scientific and technological disciplines, from the science of biomechanics and biological movement control to the design of biomedical devices for the treatment of human physical disability.

His research accomplishments in science and technology have already made a significant impact on physically challenged people. The Transfemoral Quasipassive Knee Prosthesis has been commercialized by Össur Inc., and is now benefiting amputees throughhttp://biomech.media.mit.edu/people-hugh-herr/out the world.  In 2006, he founded the company iWalk Inc. to commercialize the Powered Ankle-Foot Prosthesis and other bionic leg devices.  Professor Herr’s work impacts a number of academic communities. He has given numerous invited and plenary lectures at international conferences and colloquia, including the IVth World Congress of Biomechanics, the International Conference on Advanced Prosthetics, the National Assembly of Physical Medicine and Rehabilitation, World Economic Forum, Google Zeitgeist, Digital Life Design, and the TEDMED Conference. He is Associate Editor for the Journal of NeuroEngineering and Rehabilitation, and has served as a reviewer for the Journal of Experimental Biology, the International Journal of Robotics Research, IEEE Transactions on Biomedical Engineering, and the Proceedings of the Royal Society: Biological Sciences. He has been invited to participate in joint funding proposals from other universities and corporations, and has served on research review panels including the National Institute of Health, the National Institute on Disability and Rehabilitation, and the Department of Veterans Affairs. In 2007, He was presented with the 13th Annual Heinz Award for Technology, the Economy and Employment. His work has been featured by various national and international media, including Scientific American Frontiers, Technology Review, National Geographic, the History Channel, and CNN.

Links:

• Download mp3 (18.3 MB)
• Link to the Biomechatronics Group’s webpage
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

The transcript is edited for clarity.

Hugh Herr:
My name is Hugh Herr and I’m a professor at MIT. I co-direct the Center for Extreme Bionics. I also am the founder of a company called BionX Medical Technologies.

Audrow Nash:
Can you tell me a bit about how you have lost your legs from the knee down?

Hugh Herr:
I was an avid mountain climber. I started mountain climbing when I was seven years old. When I was in my early teenage years I was considered a child prodigy in climbing. When I was seventeen I was out with my climbing partner at the time. Our goal was to climb Mt. Washington in the winter by ascending Huntington’s Ravine, which is about an eight hundred foot, very steep ice wall. We ascended very quickly the eight hundred foot wall, and then decided to continue towards the summit. The weather conditions got worse and worse. Even though we went just five minutes above the head-wall of Huntington Ravine, we were not able to retrace our tracks and we descended the mountain.

By the time we got to tree line it was clear that we had gone off course, but we were stuck at that point. We really didn’t have the rational choice of retracing our tracks and going towards the summit. At that point, wind speeds were so high that one could not even stand. We were forced to go down this ravine system, it’s called the Great Gulf Region. It’s the wilderness side of Mt. Washington. It was there that we spent several days in most extreme bushwhacking that one can imagine. The average depth of snow was to the waist. Sometimes it was to the chest. We were just simply trapped in a white maze.

We dug in when we were not able to move any more out of exhaustion by creating kind of these snow caves. During the day we would walk. We would probably make two miles of progress in a complete kind of marathon effort in the deep snow. We made it within a few miles of the roadway and then couldn’t walk anymore because of severe frostbite. We were later discovered, or found by a person out snowshoeing for the day, and we were plucked from the mountain via helicopter and then treated for severe frostbite and hypothermia.

Audrow Nash:
Then, so from there because of the severe frostbite you had both of your legs amputated from the knee down?

Hugh Herr:
That’s right. After a few months of effort by my medical team to “save” my biological limbs, they were amputated. We were on the mountain late January. My legs were amputated mid-March.

Audrow Nash:
Afterwards, you wanted to return to rock climbing. You were told otherwise, though.

Hugh Herr:
Yeah. I didn’t know what I would be able to do with my new body, and my father said, “If you want to climb, you should climb.” I really had no clear examples of what life would be using prostheses. My limbs are amputated. I went through about a month or two of healing, and then I was fitted with my first pair of artificial limbs. I was shocked at their lack of technological sophistication. They were actually made of plaster of Paris and it was recommended that I not walk without canes or crutches because the limbs might actually break. They were kind of these crazy trainer limbs. The first weekend I went home from the rehabilitation center they didn’t allow me to take my legs because they knew what I was capable of. The next weekend they were stupid enough to allow me to take my legs and I went climbing with my brother, Tony.

Audrow Nash:
How did you begin to adjust your prosthetic limbs for climbing and various purposes?

Hugh Herr:
I quickly abandoned the notion that the limbs needed to look human, and I quickly focused on function. I thought to myself what would be optimal designs for the vertical world of rock and ice climbing. What emerged was a series of different feet for rock and ice surfaces, so feet that could stand on small rock edges the width of a coin, feet that could wedge into rock fissures even where the human foot could not penetrate, feet that could penetrate ice walls. I made my legs exceedingly lightweight to increase my strength to weight ratio. Through design, I was able to etch out a few advantages. There were disadvantages, but I gained enough advantages that it was only twelve months after my limbs were amputated that I was climbing better with artificial limbs than I’d ever achieved with normal biological limbs before the accident.

Audrow Nash:
How did you transition into academia from this, and how did you know that that is the way you should pursue, or continue?

Hugh Herr:
To be honest, the men in my family, my father and brothers and grandfathers, were in the business of construction. Given how uncomfortable artificial limbs were at the time, I couldn’t imagine being on a construction site for the rest of my life. I also couldn’t imagine being on the construction site for the rest of my life. I decided to go to college, which I’d never plan to do. My intent was to be the best climber in the world before the accident. I went to college and I signed up for very basic math courses and computer science courses. I developed an extraordinary passion for the topics and couldn’t stop studying. I just absolutely loved the material. It’s interesting because science and a passion for science and math actually replaced somewhat my passion for climbing mountains.

Audrow Nash:
Then you began to apply what you were learning to your own prostheses with what you were learning? When did you start doing that?

Hugh Herr:
I did have the goal of continuing to design prostheses. The experience of designing new climbing legs and succeeding so wildly beyond everyone’s expectation, including my own, was very inspirational for me. I realized the power of technology to heal, to rehabilitate, even to extend human capability beyond innate capabilities. That was also a clear motivation in my acquisition of knowledge and going back to school. My first patent was obtained, issued very close to the time when I was graduating from undergraduate school in physics. It was on the mechanical interface between the residuum and the artificial limb. The idea was to use fluid bladders in certain configurations. I later proposed to my future MIT advisor that I would build this interface and have actuation pumps and pressure sensing and continually modulate the pressure field around the residuum.

Audrow Nash:
On the spot where the prosthetic limb interfaces with the residual limb is what you’re talking about?

Hugh Herr:
Yes.

Audrow Nash:
So you would use some sort of fluid to make a vacuum so that it would stay very well attached?

Hugh Herr:
Yes, a vacuum offering suspension, but also offering loading support.

Audrow Nash:
Loading support.

Hugh Herr:
A pressure field that would be very comfortable around the end of the limb, enabling a person to walk without discomfort.

Audrow Nash:
Can you tell me a bit about the prostheses that you’re wearing now?

Hugh Herr:
I’m wearing two bionic legs. Both legs have three microprocessors and twelve sensors. The microprocessors are brains, if you will. The devices control a muscle-like actuator that moves and stiffens and powers my synthetic ankles.

Audrow Nash:
What sensors are on board?

Hugh Herr:
The sensors measure position, speed, acceleration, force and temperature, as well as joint position and speed.

Audrow Nash:
What sensing are you doing, contact with the ground?

Hugh Herr:
No. The device has tendon-like series springs, and also a parallel elasticity. The torque sensing is of the torque that the series elastic actuator sees and the parallel spring sees.

Audrow Nash:
I see.

Hugh Herr:
Either one can produce net torque on a joint.

Audrow Nash:
Can you describe how they look a little bit?

Hugh Herr:
They look interesting. There’s two black batteries that sit on top, and there’s black anodized metal shields coming around the ankle. Then inside those shields are all the electronics and then the motor system. Then attached to that is a synthetic foot made of again, black carbon composite.

Audrow Nash:
What kind of grip on the bottom?

Hugh Herr:
I wear the carbon foot, and glued to that carbon foot is a typical rubber that you’d find on the bottom of a shoe.

Audrow Nash:
What kind of battery life?

Hugh Herr:
We get a few thousand fast walking steps. For me and my lifestyle, that’s fine to get through the day. If one wants to walk say, ten thousand steps, one would just take a spare battery or two. The batteries are modular. You just snap them in like a power tool.

Audrow Nash:
I see. How does this compare to other prostheses?

Hugh Herr:
All other foot/ankle prostheses in the world are human powered, meaning the energy of the human attached to them drives the movement. The legs that I’m wearing are bionic. They inject mechanical energy into a person’s stride in a way that’s similar to what the muscles used to do that were lost on amputation.

Audrow Nash:
Can you talk a bit about how they are fit to your residual limbs?

Hugh Herr:
At MIT I have a team of people thinking about fit. What is the nature of comfort? What are the mathematics of comfort? What is the science of comfort? We build mathematical descriptions of the tissues of the ends of the residuum and theories of how the shape of the structure that comes around the residuum and the stiffness of that shape to optimize comfort. Basically, the end of my limb goes into a cup-like device that supports my weight, and so the shape of that cup is critically important to comfort. We mathematically derive that shape, and then we 3D print the structure.

Audrow Nash:
You’re identifying where there are hard and soft spots in the residual limb so that you can match them or match them with their complement, so soft to hard, hard to soft so that it’s more comfortable?

Hugh Herr:
Yeah. Then what’s very complex is the shape, the equilibrium shape of the synthetic skin, if you will. That’s based on tissue, how compliant the tissues are, that shape. Mathematically we understand the compliance of the tissues in all the regions.

Audrow Nash:
What do you mean, compliance of the tissue?

Hugh Herr:
How soft or how stiff they are.

Audrow Nash:
Okay. What about stretch of say, the skin? Does that apply as well?

Hugh Herr:
In a different way. I wear a second skin silicone liner, so the design of that second skin relates to the skin’s strain field, or the amount of stretch in the skin.

Audrow Nash:
I see. Are they attached by this vacuum principle or how would you attach them to the residual limb?

Hugh Herr:
There’s a number of ways to attach. You can have a pin and a lock at the base of the limb. You can have suction. You can have a sleeve that spans from the biological leg across the socket. There’s a number of ways to hold it on.

Audrow Nash:
What have been some challenges in designing this?

Hugh Herr:
The interface. If you ask a thousand persons with limb amputation what’s the number one problem they want solved, probably all thousand would say the mechanical interface, the mechanical attachment. Why? Because it’s uncomfortable. It’s a very, very important problem. It’s a very complex problem because no one understands what comfort is for any device, or there’d be a shoe or a prosthesis. We’re developing that science, and it’s very exciting. We want to be able to produce sockets that are comfortable. We want to produce them fast at low cost.

Audrow Nash:
What are a means of doing this? You 3D print the part. How do you model the residual limb?

Hugh Herr:
We take an MRI image of the limb, and that tells us where the shape of the bones and where the skin is and where the muscles are. We then use robotic palpation tools to compress the tissues and measure displacement speeds and forces, which tell us the fundamental constants of the tissues. We then build a continuum mechanical model. If you have a mathematical description of the biological segment, you can then apply pressure fields. The model will tell you how the tissues deform. Then we define particular tissue deformations that are healthy and comfortable.

Audrow Nash:
Is the goal is to make this cheap and replicable for the general person?

Hugh Herr:
Yeah. My vision of the future is that each human individual will have a digital representation of their body. When they need any type of a bionic device, that digital body will be used to compute an optimal personalized bionic device for the human.

Audrow Nash:
This can refer to clothing as well.

Hugh Herr:
Correct. Clothing, shoes, bras, bike seats, bionic limbs, exoskeletons, neural implants that go inside the body. Everything will be personalized.

Audrow Nash:
Everything is optimized, and this relies heavily on 3D printing or additive manufacturing?

Hugh Herr:
3D printing is a tool. It’s not the dominant science. That is to say, one could build these interfaces with molding processes and not using digital fabrication. Digital fabrication is sometimes nice because it reduces the fabrication frequency, but it’s not necessary. It’s not the secret sauce.

Audrow Nash:
What’s the role of industry and academia in producing these prostheses?

Hugh Herr:
As stated, I’m a professor. The typical model of a professor in the US is they receive some type of grant monies and they conduct research with a team comprising both staff and students. There are inventions. Patents are filed. Patents are owned by the university. The university then licenses the intellectual property to founders of a company. Those founders can be the professors and students that invented, and often that’s the model, the most successful model of translation. In my case that’s exactly what happened. There was patents generated. I’m the inventor. My students are the inventors. A company’s established and then there’s a contractual relationship between the company and the university, licensing the IP within a certain domain of use. That’s what was done with my company, BionX Medical Technologies.

Audrow Nash:
What role does the company play in developing the technology?

Hugh Herr:
At university we do science. We do research that’s publishing. We test hypotheses. We do things that have never been done before, never been tested before. At the company, basically crude prototypes that we build at the university are commercialized. What that means is getting them light enough, most important strong enough, durable enough, manufacturable, at the right price point. Then all the sales and clinical services necessary to distribute that product globally.

Audrow Nash:
What does it look like when someone puts on the BiOM ankle for the first time?

Hugh Herr:
The design in the ankle is biomimetic, inspired by nature, by the human body. The algorithms that are running on these small computers on the ankles control the motor as if the motor were made of muscles and tendon and the whole structure was bone and whatnot. The whole thing moves as if it was made of flesh and bone, even though it’s made of synthetics. Because of that, when it’s fit to a human, the human is used to those dynamics, so there’s little to no training required. Often in minutes the person is saying things like they have their life back, they have their leg back, and they’re either crying out of happiness or laughing out of happiness. It’s fun to go to these fittings, actually.

Audrow Nash:
I’d like to move into talking about exoskeletons. First, would you define what an exoskeleton is and then tell us how it relates to your work with prostheses.

Hugh Herr:
My definition of an exoskeleton is a device that attaches to the body, intimately to the body, that augments physicality. What I mean by augments, it enables a human being to do something that’s beyond natural capability for an innate, healthy body. The word orthoses is a medical term for a robot that attaches to the body that enables a person with a disability or some condition to move more naturally. Exoskeletons are wearable robots that augment.

Audrow Nash:
Can you tell me a bit about metabolic cost as a parameter for designing exoskeletons?

Hugh Herr:
One goal in exoskeletal design is to augment a human by pedal locomotion. An excellent metric of evaluation is the amount of food energy a person requires when using the exoskeleton versus not using the exoskeleton. Why? My view is that if the exoskeleton increases energy levels of the human, the human won’t want to use it. It won’t have sufficient value. It’s a very important metric of evaluation.

Audrow Nash:
Basically they’ll discard it and not use it if the metabolic cost is higher than the task, or higher than it would be to accomplish otherwise?

Hugh Herr:
You know, unless they want to get exercise. If they want to actually be augmented and enhance physicality, no they won’t use it.

Audrow Nash:
How do you measure this?

Hugh Herr:
Typically for aerobic exercise it’s measured by monitoring how much oxygen a person uses when they breathe and how much carbon dioxide is expelled. With those two rates one can compute the calories burned per unit of time.

Audrow Nash:
Would you describe the exoskeleton used to decrease metabolic costs for walking?

Hugh Herr:
The first exoskeleton that was envisioned and published in the world that I’m aware of was in the nineteenth century, over a hundred years ago. It was a Russian inventor called Jagen. His dream was to augment the Russian Army, Russian soldiers. It wasn’t until 2014 that someone succeeded in building an exoskeleton that augments walking and running. My group was the first to accomplish that goal. It’s an exoskeleton that spans the foot and comes up about mid-calf, just below the knee. Fundamentally it’s an artificial calf muscle. The calf muscle is the most important muscle in bi-pedal walking. It supplies about eighty percent of the power to walk. It’s where humans are most inefficient, so the exoskeleton adds an artificial calf muscle and injects energy into the gait like a calf muscle to reduce the metabolic cost of the biological calf.

Audrow Nash:
What does it look like and what kind of actuators are you using?

Hugh Herr:
The actuators are electric based. It looks very interesting. It’s like, this crazy structure, this crazy shoe that goes higher than is normal and has these springy struts that go up on the side of the leg, and then a motor that sits on kind of a shin guard that pulls on that strut and powers movement.

Audrow Nash:
Yes, and so it’s using some sort of tether and you’re pulling that, and that’s how you flex the calf to augment walking?

Hugh Herr:
Flex the ankle, yes.

Audrow Nash:
You’re using the body as its own joint for this. Correct?

Hugh Herr:
It’s very important that exoskeletons are very, very, very light weight. Whenever you add weight to the legs, metabolic cost increases. In an effort to reduce the weight or the mass of the device, we actually use the body’s joints as the bearing instead of putting a synthetic bearing in the device.

Audrow Nash:
What kind of sensing is done with the device?

Hugh Herr:
All the electronics are up high by design so you can walk through a deep, deep puddle of water and not affect the system. We measure positions and speeds and accelerations, essentially. Then from those data we can compute torques, forces, given a model of the structure.

Audrow Nash:
You can infer where the person is in their gait and then determine if you should pull to assist the gap?

Hugh Herr:
Essentially.

Audrow Nash:
Why did you choose to revolve around the ankle?

Hugh Herr:
Again because the calf muscle is the most important muscle in walking.

Audrow Nash:
Are there other advantages to it as well, being that there’s not so much squishy …

Hugh Herr:
Yeah. The shin bone is where the leg is most stiff, most rigid, and it’s a tremendous plate of bone to apply forces on from an exoskeleton.

Audrow Nash:
Can you only use this for walking? Could someone use it for running or other activities?

Hugh Herr:
The device is very versatile. It’s powered, it has smart computation, so yes you can walk, run up and down hills, whatever you want to do. It’s as versatile as the human leg itself.

Audrow Nash:
What kind of efficiency gains are we getting?

Hugh Herr:
There’s a number of peer-reviewed manuscripts. Again, we were the first in 2014 to augment the human in a peer review publication. That first publication, the human subjects were wearing a twenty-three kilogram backpack and we augmented by eight percent. We later conducted a separate study on unloaded humans, just regular walking, and augmented by an average ten percent. The variation is huge, so you have N subjects. Some subjects are as high as almost thirty percent, some are low, and everything in between.

Audrow Nash:
What do you believe creates that variation?

Hugh Herr:
It’s a really interesting question. No one really knows, but everyone in the field has observed this phenomenon, that when you put a single exoskeletal design on N people, you get huge variation. My belief is you take ten people and you could distribute them in terms of athletic capability. Some people have a very smart spinal cord and they can exploit any tool or device very, very quickly. Others are clueless. That’s a possibility where training may be effective to mitigate that variance.

Audrow Nash:
Now, people that use it, what do they say it feels like?

Hugh Herr:
The human body is extraordinary, so a person begins to use it, they very quickly … Initially they feel the energy that it provides, but very quickly the human body gets used to it. Then you take it off and you kind of stumble, and your own biological normal legs feel heavy, awkward and slow. This tells us that wearing bionic structures will just be commonplace in the future, and innate, normal bodies will just be boring.

Audrow Nash:
Now, what kind of upper limit do you think we can get on efficiency with this device?

Hugh Herr:
With the foot/ankle device it’s not known. I would say at least twenty-five percent, but it may be even higher, which is very, very exciting.

Audrow Nash:
Then will you benefit, you think, from augmenting other muscles or other movements?

Hugh Herr:
We don’t know. What’s intriguing is our recent paper on the device. What’s intriguing is when you add exoskeletal torque and power at the ankle, not only do you reduce the power requirements of the biological ankle, but also you reduce the power requirements of the knee and hip. Even though the exoskeleton doesn’t span the knee and hip we nonetheless, reduce muscular effort at those joints tremendously. It begs the question do you really need an exoskeleton that spans the whole leg, or is the foot/ankle sufficient?

Audrow Nash:
Why would you think that it decreases the expenditure at those muscles?

Hugh Herr:
It’s just the dynamics of the system. The body, when you give it power, it uses that power distributively across the entire leg in an optimal way. It’s a new gait. It’s a different gait … than normal.

Audrow Nash:
How does it look compared to normal?

Hugh Herr:
It looks normal to an untrained eye. It doesn’t look like a bizarre walk. It is a distinctly new gait. You give the human energy from a bionic structure and the body quickly figures out how to ultimately use that energy to minimize its own metabolic food energy. That’s what humans are really good at. We’re inherently lazy and we minimize energy really, really well. We’re terrible at detecting high stress levels and probabilities of injury. We injure our knee and we’re surprised. We injure our hip and we’re surprised. An incredible value of exoskeletons is to give the human something that they’re bad at. One class of exoskeletons in the future will be exoskeletons that enable us to do athletic performances without a threat of injury.

Audrow Nash:
How would those work?

Hugh Herr:
I don’t know, but they would have to detect high levels of stress and fatigue in biological structures and then tell the human to stop or to move in a different way. Imagine a world where our top athletes never injure. Imagine the impact to human athletic performance. Performance would just go through the roof, because often what mitigates top performance is injury and recovering from injury and not training harder because you might injure, yada, yada, yada.

Audrow Nash:
What other future applications of exoskeletons do you see?

Hugh Herr:
Another form of augmentation is maintaining one’s inherent physicality independent of age. Imagine a world when you run your best marathon time when you’re sixty, and you’ve been running marathons since the age of fifteen. Why? Because you’re a more skilled athlete. As you age, technology eliminates the impact of age related degeneration by neural implants, maybe by regenerative medicine, maybe by exoskeletons or various forms of augmentation. That’s a very palpable form of augmentation. Most people ethically would be fine with that and most people would be excited about maintaining quality of life as they age.

Audrow Nash:
What are some bottlenecks in exoskeletons? What technological bottlenecks are there?

Hugh Herr:
I’m not sure there are technological bottlenecks. I think that largely the components are sufficient. It’s really a problem with design and architecture. The human machine interaction is something that’s unknown, so putting a bionic structure on a human and adapting its control in real time in a kind of human machine optimization is critically important to the future of this area of design. Imagine putting on a device and you just starting to walk and run and it adjusts its behavior to optimize its performance with you in a collaborative effort.

Audrow Nash:
Is this one of your research interests moving forward?

Hugh Herr:
Absolutely.

Audrow Nash:
What are some of your near-term research goals?

Hugh Herr:
I want to do three things first and foremost. One is I want to advance better muscle-like actuators. Perhaps actuators that are better than biological ones, kind of the engine of BionX. I want to understand electrical interface between the peripheral human nervous system and devices, so how to talk to nerve endings, essentially. I also want to understand how to attach machines to the body mechanically in a comfortable way. Those three innovations will really define bionics in this century. If you solve all three, you’re more or less done. You’ve solved bionics, limb bionics that is.

Audrow Nash:
Thanks for being on Robots Podcast.

Hugh Herr:
Thank you.

Myon walking with human support. Source: Neurorobotics Research Lab

The idea of connecting brain-inspired models of computation to robots is probably as old as the discipline of robotics itself; as far back as 1950, neurophysiologist William Grey Walter had already conceived of an autonomous mobile platform that was controlled by brain-inspired analog circuitry[1]. Today, researchers are connecting robotics with neuroscience in order to both build intelligent robots and to better understand the brain. The workshop Advances in Biologically Inspired Brain-Like Cognition and Control for Learning Robots at IROS (Hamburg) brought together experts from diverse fields in brain-based robotics, neurorobotics, artificial neural networks and machine learning to discuss the state of the art.

Learning About Animal Locomotion Control using Robots

Salamander CPG model tested with a amphibious salamander-like robot (Ijspeert et al., 2007).

In contrast to studying animal locomotion though simulation, biorobots allow researchers to observe physical interaction with the real world, while at the same time avoiding the ethical issues of animal experimentation. In the first talk, Auke Ijspeert from EPFL in Switzerland gave an introduction to his research on animal locomotion and biorobotics, and spoke about the advantages of using robots as an experimental tool.

After introducing the four essential ingredients of animal motor control (descending modulation from higher brain areas, reflexes, the musculoskeletal system, and central pattern generators – CPGs) Ijspeert’s talk focused on his group’s efforts to implement walking and swimming in a salamander-like robot using CPGs. CPGs are assumed to be neural circuits that are implemented in the spinal cord of vertebrates, and while there is no clear experimental evidence, researchers in the field suspect that they are responsible for implementing motion primitives that are controlled via modulation signals from the brain. In the salamander robot, this modulation enabled a CPG modeled as a system of coupled oscillators to switch between walking and swimming locomotion.

Further studies on the salamander robot include the incorporation of sensory feedback, which considerably improves performance in the presence of external perturbations. Ijspeert concluded by presenting recent research that showed how adding a CPG to a sensory-driven model of human bipedal locomotion can simplify the control of walking speed.

Employing Neural and Physical Dynamics for Robot Control

The second major theme of the workshop featured talks from three invited speakers. Manfred Hild from the Neurorobotics Research Laboratory at Beuth Hochschule für Technik Berlin, Germany presented on Self-Exploration of Autonomous Robots Using Attractor-Based Behavior Control. Rather than developing explicit models for his Myon [2] and Semni [3] robots, the goal of Hild’s research is to acquire implicit body representations through self-exploration.

The information gathered during this process is stored in sensorimotor manifolds that allow for behavior control. Learning these manifolds through naïve motor babbling is infeasible due to the complexity and size of the resulting data. In Hild’s approach, all joints are controlled independently by relatively simple cognitive sensorimotor loops. The embodied interplay of these individual loops enables attractor-based behavior control through transitions between different attractor states of the system formed by the robot and the environment. By varying the parameters of the individual cognitive sensorimotor loops, it is possible to learn sensorimotor manifolds in a highly efficient manner.

By contrast, Joni Dambre from Reservoir Computing Lab at Ghent University in Belgium leverages dynamical systems for robot control. In her talk Practical approaches to exploiting body dynamics in robot motor control, she provided an overview of her group’s research on applications of the reservoir computing paradigm in robotics. In particular, the focus was on low-cost compliant robots whose dynamics are not analytically tractable and therefore cannot be controlled using standard methods from classical robotics. The idea of reservoir computing was originally conceived independently by Jaeger[4] and Maas[5] in 2001 and 2002, respectively. In the original publications, the reservoir was modeled as a randomly generated recurrent network of artificial neurons or spiking neurons. Rather than training all connections of the network individually, only a small set of linear readout units is adapted with a simple learning algorithm-like linear regression. The internal weights of the complex reservoir network remain constant. Based on this concept, Dambre’s group has developed a method for embedding complex tunable CPGs into neural networks as well as an approach for creating inverse models using reservoir computing. In another study, the reservoir network was replaced by the physical dynamics of a tensegrity robot, which was then able to learn CPGs. Combining reservoir computing with morphological computation therefore provides an easy means of obtaining embodiment. If the body dynamics are not rich enough for the desired control task, it is possible to couple them with a neural reservoir. As an example, Dambre presented new results on learning to walk with the Oncilla robot.

The last presentation on neural dynamics was given by Herbert Jaeger from the MINDS group at Jacobs University in Bremen, Germany. In his talk Generating and Modulating Complex Motion Patterns with Recurrent Neural Networks and Conceptors, Jaeger (one of the inventors of the reservoir computing approach) outlined a novel computational principle called conceptors. Conceptors are an extension to Jaeger’s echo state network approach to reservoir computing and enable a recurrent network of artificial neurons to store and reproduce dynamical patterns. For each pattern stored in the neural network, a corresponding conceptor is generated. By later inserting this conceptor as a neural filter into the state update loop of the reservoir network, the corresponding pattern can be retrieved. Since a single reservoir can store several different motion patterns at the same time, it is also possible to morph patterns by linearly blending between different conceptors. This was demonstrated using the example of recorded human motion patterns.

Training and Understanding Deep Neural Networks

The recent successes of deep learning techniques have brought new momentum to the field of machine learning, and have heralded a renaissance in artificial neural networks. Deep neural networks can achieve superhuman performance in traffic sign recognition[6], learn to play computer games[7], and automatically generate appropriate responses to e-mail messages[8] — rendering them a highly interesting tool for commercial applications ranging from autonomous driving to automated image tagging. It is thus no surprise that companies like Facebook, Google and Microsoft have founded their own deep learning research laboratories.

In his talk on Training and Understanding Deep Neural Networks for Robotics, Design, and Perception, Jason Yosinski from the Creative Machines Lab at Cornell University shed light on some quite different applications and properties of deep neural networks. At the beginning, he demonstrated his work on central pattern producing networks (CPPNs) for generating 3D models. The shape of the generated forms is controlled by an evolutionary algorithm. On endlessforms.com, users can try out this algorithm live by selecting their preferred forms as a starting point and see how they evolve over different generations. Yosinski has also applied CPPNs for evolving gaits on the four-legged robot, Aracna.

The second part of Yosinksi’s talks was dedicated to methods for visualizing and understanding the inner workings of deep neural networks like AlexNet[9]. He demonstrated how CPPNs can be used to generate artificial images that are falsely recognized with high confidence by deep convolutional neural networks. This approach was used in the deep visualization toolbox[10], an open source tool for visualization and analysis of deep neural networks.

The Neurorobotics Platform of the Human Brain Project

In the last talk of the workshop, Stefan Ulbrich from FZI Forschungszentrum Informatik in Karlsruhe, Germany presented The Neurorobotics Platform[11] of the Human Brain Project[12]. This platform is developed within the Neurorobotics Subproject of the Human Brain Project and will provide both roboticists and neuroscientists with tools to connect robot models to large-scale brain simulations running on supercomputers or neuromorphic hardware. All functionality is made accessible via a modern web-based user interface that displays a live visualization of the simulated robot, its environment, and live data such as brain activity or robot state parameters. A separate closed-loop engine connects and synchronizes the robot and brain simulation components. The data exchanged between brain and robot is processed by user-definable transfer functions. Dedicated designers will enable users to define their own robots, environments and experiments. Alternatively, ready-to-run setups will be made available in a catalogue.

After outlining the roadmap for developing new features for the Neurobotics platform, Ulbrich closed his presentation with a video showing the platform in action. Interested users will be able to access and try out the Neurorobotics platform after its public release in April 2016.

https://www.youtube.com/watch?v=b_RjzdlN0y4

Poster Session

The invited talks were complemented by a poster session that presented range of topics, including a review of the role of time in neurobiological learning algorithms, an approach to cognitive walking, and a neural architecture for autonomous task recognition, learning and execution. Topics with a special focus on neurorobotics included the cerebellar control of a compliant robot arm and select experiments from the Neurorobotics Platform.

Final thoughts

The variety of different topics in the workshop and diverse backgrounds of the invited speakers demonstrates that there is a lot of research being done in machine learning and biologically inspired cognition, and much of it goes beyond standard deep convolutional neural networks. Instead of asking ‘which of the proposed methods will outperform all of its competitors,’ the question should rather be ‘how can these methods and their individual strengths be combined to build better robots?’ This workshop was a first important step on this way.

To foster future collaboration, a call for contributions to a special issue of the journal Frontiers in Neurorobotics on the topic of the workshop will soon be announced. Further information about the workshop and abstracts of all talks and posters are available on the www.neurorobotics.net.

[1] http://www.nature.com/scientificamerican/journal/v182/n5/pdf/scientificamerican0550-42.pdf

[2] http://www.neurorobotik.de/robots/myon_de.php

[3] http://www.neurorobotik.de/robots_de.php

[4] H. Jaeger (2001): The “echo state” approach to analysing and training recurrent neural networks. GMD Repo

[5] Maass, W., Natschläger, T., & Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural computation, 14(11), 2531-2560.

[6] http://people.idsia.ch/~juergen/superhumanpatternrecognition.html

[7] http://arxiv.org/abs/1312.5602

[8] http://googleresearch.blogspot.ch/2015/11/computer-respond-to-this-email.html

[9] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105).

[10] http://yosinski.com/deepvis

[11] http://www.neurorobotics.net

[12] http://www.humanbrainproject.eu

Last month we caught up with Dario Floreano, the head of the Laboratory of Intelligent Systems at the Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland. Here we continue our discussion, covering acoustic sensing, multi-drone operations and more. Missed Part 1? Check it out here.

Last time we spoke, we discussed your work on visual perception, but you also mentioned acoustic research. So this means we’re talking about ‘listening drones’?

Yes, that’s right. This is particularly interesting in the area of drone swarming, whereby multiple robots need to coordinate their flights very carefully.

The first big problem with swarming is perceiving the other drones. If you have multiple drones in flight, each robot is only a very small spot in the other robots’ visual fields of view. They’re very hard to spot against the background. So we thought, what about sound detection?

The question is, how can a robot hear other robots when it is in flight? Well actually, this is a simple trick. You simply switch the engine off, very briefly, listen using three tiny microphones pointing in different directions, then restart the engine. Just like the eBee does when it is capturing aerial photos. Using this trick we developed drones that can detect the presence and relative direction of other drones acoustically.

Outside of the lab, what do you see are some of the potential end applications of your work?

Sticking with the acoustic approach, this could be useful in search and rescue, being used to detect people’s locations on the ground. If people are lost in snow or in a forest, they often have emergency whistles. We have shown that we can locate a person using a standard emergency whistle and broadcast its GPS coordinates to a rescue team.

With swarming, we’re also looking at using drones as relay networks for communication between rescue and security teams on the ground.

In the event of an earthquake, a fire or a landslide, the first thing an emergency team usually does is to set-up its communication infrastructure. This typically consists of a set of trucks with radio signal emitters and receivers, which operate on a specific frequency. It can take one or several hours to complete this set-up before rescuers can start working on the ground, using their receivers.

However drones can be equipped with emitters and receivers. Therefore a swarm of drones could cover the area, pick up the signals of the rescue team on the ground, and relay these, connecting people via very low power, short-range emitters. Currently with this technique we can connect multiple people, and not only with audio but with video signals too.

We’re also experimenting with multiple people moving around on the ground, with robots following their movement, and we’re considering operations that last longer than a single battery duration of around half an hour. We’re developing an algorithm that allows drones to land for a battery change, while new drones with a full battery launch to continue the service.

So on one hand your work is bio-inspired research, but you’re obviously also thinking about the potential real-world applications of the technology you develop. Which point do you start from—do you look at biology first and then consider how to use the tech that comes out of that research, or do real-world problems drive the direction your biological research takes?

It’s a mix. When it comes to drones, 15 years ago it was more blue sky research. These days however I have more projects on flying robots that could solve problems that people might face, not necessarily next year but in five years’ time. For example, if you will have lots of drones all in the air, you then think about safety. That means thinking about AI and sensing.

The specific applications are not always there though. For example, when the Gimbal was designed we didn’t have an application in mind, just a problem: how to design a robot that could withstand falls and fly again. We started by adding retractable legs, as insects have, but this wasn’t optimal, so we came up with a rolling cage around the drone. Then we realised the application: being able to go where other drones couldn’t go.

Prof. Floreano’s work at the Laboratory of Intelligent Systems (LIS) is bio-inspired; he and his team develop flying robots with animal characteristics, which they also use to learn more about animals.

In swarming there are lots of interesting problems to think about. In the future, as I mentioned, we’ll have a very high number of drones sharing our airspace. We’ll have a lot of crowding in areas such as landing spots and drone airways.

The big question is this: how will these drones avoid each other? Commercial GPS isn’t good enough to enable this avoidance when you fly very close to other drones. So the question is: what are the best sensors and the best control algorithms, both for drones flying in cooperative conditions—meaning the same types of drone flying in the same area—and non-cooperative conditions featuring drones from different manufacturers using different sensors? This is a real problem that some of our research is addressing, regarding the sensors and algorithms we’re developing.

Another real problem comes in transportation: how can we transport parcels very safely? I can’t talk about the solution just yet, but our work there builds on a lot of the topics we’ve been discussing.

Which seems a logical place to wrap up! Thank you so much for all these insights. How can readers who are interested in learning more follow your lab’s work?

The homepage of the Laboratory of Intelligent Systems is at lis.epfl.ch. That page includes our most recent publications. We also have a LIS YouTube channel that people can subscribe to and I am on Twitter at @DFloreano. 

You are also involved with the RobotsPodcast, is that correct?

Well, I was involved in its creation, back when it was called Talking Robots, and I supported it from my lab for two years. The podcast grew and grew, and the students that were preparing it then set-up what is known today as theRobotsPodcast. Many members of LIS still work on this on a voluntary basis. It became so successful that it also spurned another, wider organisation, calledRobohub.

Thanks for your time!

You’re very welcome.

Watch Floreano’s ROBOTS14 presentation:

===========================
About Dario Floreano

Dario Floreano (M.A. 1988; M.S. 1992; Ph.D. 1995) is the director of the Laboratory of Intelligent Systems and director of the Swiss National Center of Competence in Robotics. He is co-founder of the company senseFly S.A., of the International Society for Artificial Life, and founder of the popular robotics podcast series Talking Robots (which later became RobotsPodcast).

He has been on the advisory board of the Future and Emerging Technologies division of the European Commission, vice-chair of the Global Agenda Council on Robotics and Smart Devices of the World Economic Forum, and on the board of governors of the International Society for Neural Networks. Floreano is a senior member of the IEEE and received several national and international awards for professional and cultural achievements. Floreano has published four books, more than 300 technical articles, and more than 10 patents in drone technologies.
===========================

The research found that drones with an insect-inspired vision strategy become unstable as they approach an object, and that this happens at a specific distance. Turning this weakness into a strength, drones can use the timely detection of that emerging instability to estimate distance.

Two landing trajectories over time, starting from 4 meters. The light blue drone has stronger reactions to optical flow deviations than the magenta drone, and hence starts to oscillate further away from the landing surface.

In an effort to make ever smaller drones navigate by themselves, researchers increasingly turn to flying insects for inspiration. For example, current consumer drones for indoor flight use an insect-inspired strategy for estimating their velocity. They use a downward looking camera that determines optical flow – the speed at which objects move through the camera’s field of view. However, optical flow only provides information on the ratio between distance and velocity. Hence, an additional sonar is generally added to indoor flying drones. The sonar provides the distance to the ground, after which the velocity can be calculated from the optical flow. With the new strategy it becomes possible to get rid of the sonar, allowing indoor flying drones to become even smaller.

Soft landings

Although flying insects such as honeybees have two facet eyes, these are placed so close together that they cannot use stereo vision to estimate distances useful for navigation. They heavily rely on the distance-less cue of optical flow, and yet they lack the sensors to retrieve the actual distance to objects in their environment. So how do insects navigate successfully? Previous research has shown that simple optical flow control laws ensure safe navigation. For instance, keeping the optical flow constant during descent ensures that a drone makes a soft landing.

Further reading:

A new artificial compound eye

Fully autonomous flapping-wing micro air vehicle weighs about as much as 4 sheets of A4 paper

Parrot AR.Drone app harnesses crowd power to fast-track vision learning in robotic spacecraft

DALER: A bio-inspired robot that can both fly and walk

Bioinspired robotics: Materials, manufacturing & design, with Robert Wood

Sign up for our newsletter.

However, implementing this strategy on a real drone turned out to be very difficult as it is extremely difficult to recreate fast and smooth optical flow landings because the drones tend to oscillate up and down as they approach the ground when landing.

While I initially thought that this was due to the computer vision algorithms not working well enough when close to the ground, I eventually determined that the effect is still there even when the drone has perfect vision.

Using instability

In fact, theoretical analysis of the drone’s control laws showed that a robot that is trying to keep the optical flow constant will start to oscillate at a specific distance from the landing surface. The oscillations are induced by the robot itself, because a movement that occurs close to the ground has a much quicker and larger effect on the optical flow than a movement that occurs further away from the ground.

The core idea is that timely detection of such oscillations allows the drone to know how far it is from the landing surface. In other words, the robot exploits the impending instability of its control system to perceive distances. This could be used to determine when to switch off its propellers during landing, for instance.

Of course I had many strange looks in the flight arena when I was cheering on a flying robot that seemed to be on the verge of going out of control!

A Parrot AR drone making a constant optical flow landing outdoors.

Biological hypothesis

A closer look at the biological literature shows that flying insects do have some behaviours that are triggered at specific distances. For example, both honeybees and fruitflies extend their legs at a fairly consistent distance from the landing surface [1,2]. The new theory provides a hypothesis on how insects may perceive the distances necessary for such behaviors.

More info

‘Monocular distance estimation with optical flow maneuvers and efference copies: a stability-based strategy’ in Bioinspiration & Biomimetics, article published online on 7 January 2016.

http://mavlab.tudelft.nl/
http://www.bene-guido.eu/

Contact
Guido de Croon: +31 (0)15 278 1402, G.C.H.E.deCroon@tudelft.nl

References 

[1] Van Breugel F and Dickinson M H 2012 The visual control of landing and obstacle avoidance in the fruit fly Drosophila melanogaster J. Exp. Biol. 215 1783–98

[2] Evangelista C, Kraft P, Dacke M, Reinhard J and Srinivasan M V 2010 The moment before touchdown: landing manoeuvres of the honeybee apis mellifera J. Exp. Biol. 213 262–70

Precisely controlled elastic actuators allow for temporary energy storage – in fact this robotic system consumes 10 times less power than other hydraulic systems. Weighing in at just 26 kg, it can be handled by a single operator, but operates autonomously at a speed of 2km per hour walking or running. Potential applications for such a highly mobile robot include: inspection of industrial, construction, or polluted environments, search and rescue operations, security, or even the entertainment industry.

Further reading:
http://www.leggedrobotics.ethz.ch/doku.php?id=robots:starleth:starleth
http://www.leggedrobotics.ethz.ch/doku.php?id=robots:scarleth
http://www.leggedrobotics.ethz.ch/lib/exe/fetch.php?media=robots:starleth:description:201401_descriptiontransporter.pdf

If you liked this article, you may also be interested in:

• Air and ground robot collaborate to map and safely navigate unknown, changing terrain
•  Robots Podcast #187: Cheetah 2, with Sangbae Kim
• Introducing Spot, a new smaller 4-legged robot from Boston Dynamics
• Firmer footing for robots with smart walking sticks

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

by Jennifer Chu | MIT News Office

Harbor seals have an amazingly fine-tuned sense for detecting prey, as marine biologists have noted for years. Even when blindfolded, trained seals are able to chase the precise path of an object that swam by 30 seconds earlier. Scientists have suspected that the seal’s laser-like tracking ability is due in part to its antennae-like whiskers.

Now engineers at MIT have fabricated and tested a large-scale model of a harbor seal’s whisker, and identified a mechanism that may explain how seals sense their environment and track their prey.

The team found that a seal’s whiskers serve two main functions in sensing the environment: first remaining still in response to a seal’s own movements through the water, and then oscillating in a “slaloming” motion in response to the turbulence left by a moving object.

In their experiments, the researchers observed that once the fabricated whisker enters the wake left by a passing object, it starts vibrating at the same frequency as the wake’s passing vortices. Careful visualizations show that the whisker “slaloms” among the vortices, like a skier zigzagging between flags.

The research shows that this slaloming allows the whisker to extract energy from the wake, causing it to vibrate at the precise frequency of the wake — a mechanism that may give seals a clue to an object’s path, its size, and even its shape.

Michael Triantafyllou, the William I. Koch Professor in MIT’s Department of Mechanical Engineering, says that biologically inspired sensors, modeled after the harbor seal’s whiskers, may aid underwater vehicles in tracking schools of fish, as well as sources of pollution — a goal that he is currently working toward.

He and former graduate student Heather Beem, whose PhD thesis formed the basis of the work, have published their results in the Journal of Fluid Mechanics.

A “quieting effect”

The harbor seal’s whiskers are unique in shape: Even to the naked eye, an individual whisker appears not uniform, but wavy. Under a magnifying glass, the pattern is more intricate, with an elliptical cross-section that varies in size along its span.

“It’s marvelous to see this intricate pattern, it’s not just a straight antenna — it’s a perfect sinusoid,” Triantafyllou says.

He and Beem proposed that a whisker’s curiously geometric morphology may play a part in a seal’s exceptional sensitivity.

Using 3-D printing techniques, Beem reproduced the seal’s wavy morphology at a much larger scale, in order to accurately measure its response to various wakes. She tested the whisker’s vibration properties in a 30-meter-long tank of water with a moving track suspended above the water.

In her experiments, Beem first attached the artificial whisker to the moving track, allowing the whisker to freely vibrate in the water as it moved down the length of the tank.

While most long, thin rods tend to create large vortices, or eddies, as they move through water, forming a pattern that’s well-known in fluid mechanics, Beem found that the wavy pattern of the whisker’s geometry created much weaker vortices, enabling the whisker to move silently, with very little vibration, through the water.

The whisker’s morphology, the researchers found, may help the seal block out its own disturbance as it moves through water.

“It’s like having the ability to stick your head out of a car window, and have there be no noise, so that your ears don’t ring: It’s a quieting effect,” Triantafyllou says.

A whisker sensor

To test how a whisker reacts to external stimuli, Beem conducted a second set of experiments in which she attached a large, long circular cylinder ahead of the whisker. As the cylinder moved down the tank, it created large eddies, similar to the patterns generated by a passing fish.

In response, she found that the whisker, when following the cylinder, vibrated significantly, moving in a slaloming pattern among the wake vortices. As she varied the speed of the moving track, the whisker quickly adapted, vibrating at precisely the frequency of the cylinder’s changing vortices.

“The geometry of the whisker allows for this phenomenon of being able to move very silently through the water if the water’s calm, and extract energy from the fish’s wake in order to vibrate a lot,” Beem says. “Now we have an idea of how it’s possible that seals can find fish that they can’t see.”

Triantafyllou says artificial whiskers may be useful as low-power sensors for underwater vehicles.

“We already have a few sensors that can detect velocity, but now that we know better what they can do, we can use them to track sources of pollution and the like,” Triantafyllou says. “By having several whiskers on a vehicle, like the seal, you can, for example, detect a faraway plume, and track it all the way to the end.”

This research was supported in part by the Office of Naval Research, the Singapore-MIT Alliance for Research and Technology, and the MIT Sea Grant program.

Reprinted with permission of MIT News.

Transcript included.

In this episode, Ron Vanderkley speaks with Dr. Eleanor Sandry of Curtin University about her new book Robots and Communication. In the interview, we explore human to animal communication and what we can learn from it; human to humanoid robots interaction; and human to non-humanoid robots interactions. Also, we discuss Western and Eastern perceptions of robotics.

Dr. Eleanor Sandry

Eleanor Sandry is a Lecturer in the Department of Internet Studies at Curtin University in Perth, Western Australia. Her research is focused on developing an ethical and pragmatic recognition of, and respect for, otherness and difference in communication. Much of her work explores communication theory and practice by analysing human-robot interactions in science fact, science fiction and creative art.

Links:

• Download mp3 (16.6 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Transcript

Ron Vanderkley: Good morning Eleanor, if I can first ask you to introduce yourself to The Podcast listeners.

Eleanor Sandry: Hi there, well I’m Eleanor Sandry. I’m a lecturer in Internet Studies at Curtin University in Western Australia. My research is about communication in general, not just on theinternet and I’m particularly interested in human-robot interactions so that we can understand communication more broadly.

Ron Vanderkley: What got my attention to your subject was an up and coming book that appeared at a conference listing, if you can briefly tell us a little bit about the work on this book?

Eleanor Sandry: Yeah, so the book is called Robots in Communication and it’s been published by Paul Grave in the Pivot Series, which means it’s available online and print on demand. It’s a short format book, 45,000 words. It’s shorter than your average monograph and that’s because I wanted it to be accessible to a range of different people. It’s pitched towards people who are interested in robots, people who are interested in communication – human communication theories – so it’s trying to cater for quite a large audience. What it does is look at human-robot interactions and it tries to unpick them in some detail from a number of different theoretical perspectives.

I’m particularly interested in broadening the size of communication beyond just the transmission of information and ideas of success that surround accurately transmitting it. I’m interested in non-verbal communication; the idea that it isn’t just about what we say, but it’s about us using facial expressions and also movements more generally. I found that robots were just agreat way of looking at a huge range of different types of communication because they are sometimes built to look as human-like as possible, but on other occasions are completely machine-like, and yet what you find is the people who interact with them often develop some kind of relation to them. Even to a Roomba and other iRobot floor cleaners. Just looking at that gives me a whole range of different forms of communication to consider and things I can say about communication in general.

Ron Vanderkley: You speak about this idea of humanoid robots and the non-robots as a point of contact. What actually steers you towards the humanoid? Is it the facial features, the gestures, themovements?

Eleanor Sandry: I think that most people are studying humanoid robots because of particular facial features and expressions, in some cases. But, in others, it’s to do with physical ability to move around in human-like environments. One ofthereasons the Atlas robot – which is about to compete in the darker challenges – is human-like is because it’s expected to go and work in environments that are tailored for humans, but where humans can no longer enter, for example because of some catastrophic disaster.

In fact, I am much more drawn to non-humanoid robots and I’m particularly interested in the way that people will still read pretty much anything that moves as somehow trying to communicate with them. I’m also conscious of the fact that people who are trying to work with robots do better if they do develop this kind of relationship with them. Part of what I have been writing in the book is actually about looking at how those relationships develop and seeing those, say, between humans in ordinance explosive disposal and their robots. Relationshipsbetween people and robots are not necessarily a bad thing or something that we should be worried about; it actually allows those teams to operate better in real world environments.

Although a lot of people say that a sociable robot needs to be able to make gestures, maybe to also have facial expressions; that depends on the structure of the robot. I think there is a lot to be learnt from human-animal interactions. The things that humans and animals are capable of doing together open up possibilities for lots of other forms, for robots to be seen as effective teammates, basically.

Ron Vanderkley: For instance, canine interactions, this kind of trust and understanding. That is a whole different facet and we are still coming to termswith how wecan do that in robotics. Are you picking that apart and trying to figure out how wecan do that?

Eleanor Sandry: Yeah, I’m particularly interested in the idea of the need to have, almost, a trust and respect relationship in order for the team to operate effectively together. In the human-canine relationship, there would always be an overarching power relation. The human is almost always in control of the overallsituation and is working with the dog to achieve a particular purpose, for instance tracking a scent trail. The same is true with humans and robots and, at the moment, say with ordinance disposal robots, broadly, the human is in control almost 100 per cent of the time: they’re still mainly radio controlled or controlledvia wire connection. There are some things that they can perform autonomously like righting themselves if they fall over and maybe also returning to base.

What I’ve been trying to unpick, I suppose, is the idea that if a robot is going to become more autonomous, it will be out of contact with its human controller. There needs to be a more complex relation between the human andthe machine in order for that team to operate well. Thehuman effectively has to relinquish control to the machine at the point of which it moves out of radio contact, they have to trust in the robot to do its job. All of these ideas – even the terminology of me saying to do its job – isproblematic, historically, in understanding human-animal relations. Because many people think that a dog doesn’tknow what its job is, or isn’t going to take responsibility, and yet people who train dogs will tell you that this is precisely the way that you need to think about the dog in order tounderstand what is happening in the team, because you don’t have the nasal senses that the dog does.

It’s actually important that you trust a dog to do its job, so when it appears to be moving off a trail, in fact its followinga path that you have no idea exists and it will bring you to the final point; will complete the task effectively. I think thereare possibilities of the same thing between human robot teams, where the robot has senses that a human doesn’t. You have to learn to trust in those senses and I think it’s an interesting way to think about these relations. I think it has practical possibilities in the robotics community, but I also think it tells me all sorts of interesting things about how humans and other things interact.

Ron Vanderkley: The idea that we could try to use speech recognition between robot andobject, that being the only means of communication in the animal or non-animal world, is not entirely true. It’s looking at gestures, our voice.Is this also something that we may not have looked at in the robotic world?

Eleanor Sandry: Well I think there is a tendency. It’s not so much that it hasn’t been looked at but that the final goal is seen as being able to create a robot that it’s possible to activate entirely with just your voice and that will understand human expressions and gestures perfectly. I think that what’s being missed is that the robot could understand things differently from us. It’s not actually necessary to embed the robot in the world ina human-like way. It can be embedded in its own way and understand us in particular ways, even if that’s using sensors on hands that help direct the robot. Actually looking at those as real possibilities, not a problem that needs to be overcome, has potential forincreasing the flexibility of the way robots work.

The other reason I’m particularly interested in that is because I hope to start writing more about critical disability studies. That has really opened my eyes to the fact that technologies that are currently being directed towards very human-like interfaces may actually make it harder for people with disabilities to interact with a robot that might otherwise be very helpful to them in living their everyday lives independently. Really looking at a whole range of different interfaces for machines like this is important; not just thinkingof trying to perfect this notion of the human-like interface. I just don’t want them to cut down possibilities I suppose.

When I talk to people sometimes about robots, I get two reactions: normally, I get the people who are really firmly interested in human-like forms and want to make sociable robots as human-like as possible, and then I sometimes get other people that say “we never make robotshuman-like.” I get this divergent reaction and, really, I’m somewhere in the middle, trying to say “well, can’t we think about this differently? Can’t we think about this froma communication perspective, where you start seeing all the different ways that people and things and animals can communicate?” Because you’re quite right, of course, dogs don’t just respond to voice, they respond to tone of voice. They also understand things in ways that we can’t really possibly fully appreciate because wejust don’t have the senses that a dog does.

Ron Vanderkley: The other aspect I was considering is that we are talking about robots actually understanding us. I spoke to a guy in the UK about, I think, a robot called Hector, where it’s entertainment. It’s a robot telling the general public what is going on, with hand gestures, eye movements, et cetera. It’s a teaching tool, almost. I found that very interesting. Is that also something that you’re looking at?

Eleanor Sandry: I suppose it’s not my primary area of interest but that doesn’t mean I don’t think it’s important. I think that sometimes when I talk it’s easy for me to get obsessed with all the non-humanoid robots and it makes it sound a bit like I’m actually saying that we shouldn’t have humanoid robots and I don’t think that’s correct at all. I’m sure there are going to be situations where a humanoid robot works much better and that is probably a very good example. If you have a robot that you want to actually present to people and you want, effectively, to present it in a human-like way, then that is what you’re aiming for. You’re aiming for something that inspires the audience to pay attentionto it because of its abilities – that may sometimes be very important.

I think that thinking that that’s the only way to build a robot – that can be captivating, or attract attention, or draw people into communication – is probably incorrect and there are other examples. Certainly robots in art, for example, where a totally non-humanoid machine completely captivates its audience through its movements and the things it does, the sound it makes. It’s a very different idea; it’s not to try to transmit information directly but it’s still building a kind of relation between the human andthe machine that may lead us to question the ways that things exist in the world, which I think is usually quite a good thing. Also when things are framed by particular tasks, so anything where people and the robot aretrying to complete a task together kind of frames the communication that takes place.

I think it offers many more possibilities for non-humanoid communication in that particularsituation because you know what you’re aiming to do. And yourcapability of reading non-verbal communication is probably much greater in that situation. With the example of the robot up on stage, often a humanoid robot is going to win the game there.

Ron Vanderkley: When you look at the history of, say, robot interaction targeted at children or things like the first robot dog, where children actually relate to that almost as a real dog, and treat it as a real animal, do you think that’s the precursor to a closer relationship with non biological entities?

Eleanor Sandry: Well, I think its one direction in which to go. I’m slightly concerned by the idea of trying to create robots that effectively replace a relationship that someone would have with something that already exists in the world. It’s one of the reasons that I’m not so interested in humanoid developments, and also in the very animal-like robot developments, because it may notbe a good thing to replace the relationships that people already have with living things with a robot equivalent. Children are actually a fascinating example of humans who are freed from the need to justify all the decisions they make about how they’re going to interact with something and so, yes, theyreact to things, like a robotic dog,strongly. Their understanding, if something goes wrong with it, that it needs to rest or sleep, shows just how strong their understanding of the robot is.

I think that’s one of the reasons why I’m interested in robots that appear more overtly machine-like, because they remind you that you’re in quite acomplex new relationship with something that’s different. It’s different froman animal, it’s different from a human; it is its own thing.For example, I looked at interactions between humans and Guy Hoffman’s robot, which is a robotic desk lamp, a lighting assistant, and they did experiments together. People’sinteractions during the interaction were very much as if it was somewhat alive, not necessarily like an animal or anything else, it was an alive desk lamp. But you could still switch it off; it was still a machine as well.

Humans are capable of holding both of those ideas. I think children may be less worried about that, so as a specific example they do show just how far those relations can go. I suppose I’m interested in how you can signify that those relationships are actually different, that it is still a machine. However, I don’t really think that robots should replace human-animal relations. I just read something recently about a different researcher in Australia who has written about how he thinks, in the future, humans will have robotic animals and that immediately made me think of Doandroids dream of electric sheep? The original book has a very strong section in the front that is about the ownership of robotic animals being all that was possible for most people. The richest people had a real animal. It seemed to be that future idea and I hope and pray that we are not going to go that way.

Ron Vanderkley: No. There’s a concern throughout the entireinternetaboutthe future of autonomous robots, AI being this Skynet-kind of world, by those who I think are quite smart people. It kind of sends the wrong signals. I just wanted to see your take on things.

Eleanor Sandry: Well, my take on it is artificial intelligence has a long way to go before it ever getsto that kind of level. I also think that the way that it’s being developed at the moment, it’s not necessarily ever going to be the same as a human-like intelligence. An artificial intelligence is going to be different almost certainly. Yes, I get frustrated at the hype around the technology, and you’re right, it’s very respected voices that are raising these kinds of issues,and in the near future as well. I simply don’t think it’s going to happen in the near future and I don’t actually think it’s going to happen the way that some people are suggesting it might either.

I would definitely want to draw back from that and point out just how far away artificial intelligence is from being at that level. I mean Google Drive. These cars don’t drive everywhere; they drive in one particular place, where it’s been mapped sufficiently for them to be able to operate. I don’t think that people are necessarily aware of that when it’s reported in the mainstream media.

Ron Vanderkley: Not at all, and humanoid robots actually cost an awful lot of money to build …

Eleanor Sandry: Huge amount.

Ron Vanderkley: I don’t even know if Sony is actually able to clear a real amount of money.

Eleanor Sandry: I doubt it very much. I mean, that robot has been in a series of developments, way over a decade now, and still has issues being placed into a normal physical environment and operating successfully. It has to be ina controlled environment. I think that the general public probably isn’t aware of that. The difference between seeinga robot acting in a laboratory, in a controlled setting, and actually being able to interact in the normal physical world that we live in is something completely different. They are streets apart.

I saw something about the darker challenge basically explaining that they would be very impressed if one of the Atlas robots, or any of the other bipedal robots, actually manages to make it through the course without falling down. I think that that’s a realistic understanding of what’s possible at the moment. The idea that robots with four legs might actually operate more easily in those types of environments is fair and should be perceived as well. It’s a very interesting question.

Ron Vanderkley: Absolutely, and just in a previous podcast I was talking to some guys about soft robotics. I mean that’s completely going …

Eleanor Sandry: Totally, a very fascinating area.

Ron Vanderkley: Yeah, completely, a different direction and its early days.

Eleanor Sandry: Yeah, very early days, but there is so much potential for soft robotics but also mixtures of soft and hard. I saw ages ago that, rather than using a hand-like gripper to pick things up, someone had developedsomething that could actually… it was almost like something filled with coffee.

Ron Vanderkley: Like a bean bag.

Eleanor Sandry: Like a bean bag yes, that could grip. Those sorts of innovative ideas that are thinking of totally different ways of solving the task, rather than trying to make something that has a human-like hand, seem to me to offer tremendous potential and so I’m fascinated by the idea of what a robot might end up looking like in the future. I’m kind of encouraged that maybe robots in the future won’t look like us and actually that would be a really good thing.

Ron Vanderkley: That’s another human concern. Someone once put it that if we try and design something that looks so close to a humanand almost mimics the face, there’s total distrust. Whereas if it looks like a robot there is more trust.

Eleanor Sandry: I think there definitely is potential. I mean whenever I write about robots I always end up having to utilize the uncanny valleyexample. I’m kind of quite frustrated with it now: that idea that, as a robot becomes more and more human-like,eventually it’s actually read as zombie-like and there is a strong drop off in the level of trust in the robot. Some people suggest that you could climb the valley on the other side and eventually end up with aperfect human-like machine. Then, as has been written about in many different science fiction books, you have the question of what is the difference between the robot and the human and …

Ron Vanderkley: Silicon versus Calvin.

Eleanor Sandry: I think, really, we’re nowhere near achieving that at the moment. Even the very, very human-like faces you can see pretty easily from their expressions and from other things about them. I don’t know why we want to aim for that. I think that we have …

Ron Vanderkley: Perfection.

Eleanor Sandry: Yes, but it seems to be a perfection that’s driven by a100 percent understanding of humans being the pinnacle of evolution and, therefore, that should be the pinnacle of robotic development. That just seems to me an unfortunate decision to make. There are many more options for robotics than trying to achieve human-like status. I think that there are many other ways that machines could be far more interesting, far more useful but also maybe even develop new relations with people that don’t have to be like something that already exists.

Ron Vanderkley: One of the ideas in modern day sci-fi, the cyborg…

Eleanor Sandry: Yes, the idea of augmenting human ability, of fixing people, which is a very hot topic in disability studies because,in general, there aremany people with disabilities who do not wish to be fixed.Again, it’s driven by this idea of the human being the current pinnacle of evolution. Let’s see howfar we can take it. I hesitate to mention this but, as Dona Harrower said many, many years ago, in 1985 actually, we are already cyborg. I’m effectively a cyborg: I wear contact lenses; that’s the only way I can exist in this world. I would not be alive without those developments – of glasses or contact lenses. But where people begin to decide to amputate limbs, in order to have blades, because that means they can ran faster, that’s another thing science fiction is written about. There are all sorts of – not just ethical – questions, but bigger questions around that.

Ron Vanderkley: You brought up sci-fi as a sounding board. Do you use that as a sounding board for some of your ideas or your thoughts?

Eleanor Sandry: Often I do, yes: in fact, in robots and communication. Science fiction to me is a really great place to look because it’s sometimes overlooked as being a kind of subgenre. For some people, for many years, it was not real literature and not something that was studied much in scholarly circles. But it provides a real sounding board and some of the best science fiction writers write about embedding technology in societiesin ways that allow you to really think about some of these questions ina more developed way,because someone has written a story about it. It seems to provide a reallygreat way of thinking through some of the questions that are very difficult to picture in your head, whereas science fiction provides you with those kinds of thought experiments that can be so important.

In my thesis – what I was doing for my PhD- I did write more about science fiction: the idea of culture and human-machine interactions in culture were to me very interesting because they push way into the far future I guess.

Ron Vanderkley: Coming back on the idea that technology is going to take over. Is that something that you think that’s going to be more and more of a problem, as sci-fi seems to dwell on the bad things and not the positives, or is that just good for entertainment?

Eleanor Sandry: Well, it’s definitely good for entertainment. It’s also a very Western perspective on robots andtechnology. Whereas in Japanese Manga, the whole idea of technology, and robots in particular, is far more positive. Also, it’s a great pity that we lost [Iain] Banks because that was someone writing about a utopian feature, problematic in lots of ways, but at least he offered some kind of positive vision of the way things could work. You’re right, in general, there will still often… they’re not exactly disaster movies are they? But they’re kind of …

Ron Vanderkley: The good guy always seems to win in the end.

Eleanor Sandry: In the end but there is usually a potential for total disaster in the middle, isn’t there?

Ron Vanderkley: Okay, so in closing, what do you think is going to happen in the field of robotics, well, in Australia?

Eleanor Sandry: Australia is interesting and I’m still trying to get to grips with what’s happening in different parts of Australia. I think that the level of funding here may be very different from other countries. There are interesting things going on and I really need to spend more time getting to grips with them because I keep on reading things about what people are doing in Melbourne, starting with robotic arms, the disabled, people with disabilities, and I’m interested that there is stuff going on in Sydney. I’m seeing people from the Center for Social Robotics there, and so I’m particularly interested in what they’re doing and the cost as well.

I think robotics will continue to develop here. There is probably quite astrong drive towards thinking of robots in terms of self-driving machines, mind technology. That’s always going to be a big thing in this country. There are definitely innovative thinkers in Australia who produce interesting solutions to different problems and I would like to carry on finding out what they’re doing and writing about it.

Ron Vanderkley: Okay and to cap that off, if I can firstly thank you for giving the time to chat. On behalf ofThe Podcast, thank you very much.

Eleanor Sandry: Well, thank you for having me. It’s been great to talk.

All audio interviews are transcribed and edited for clarity with great care, however, we cannot assume responsibility for their accuracy.

Transcript included.

In this episode, Ron Vanderkley speaks with Dr. Lei Cui from Curtin University about his team’s work on 3D printable hand orthosis for rehabilitation, a  task-oriented 4-DOF robotic device for upper-limb rehabilitation and a 3-DOF platform providing multi-directional perturbations for research into balance rehabilitation. They also discuss a high-speed untethered robotic fish for river monitoring and an amphibious robot for monitoring the Swan-Canning River System.

A 3D Printable Parametric Hand Exoskeleton for Finger Rehabilitation

ComBot: a Compact Robot for Upper-Limb Rehabilitation

A 3-DOF Robotic Platform for Research into Multi-Directional Stance Perturbations

Curtin Robotic Fish

AmBot: A Bio-Inspired Amphibious Robot for Monitoring the Swan-Canning Estuary System

Lei Cui

Dr. Lei Cui completed his PhD degree in Mechanical Engineering in July 2010 from the Centre for Robotics Research, King’s College London (UK). He continued his employment as Postdoctoral Research Associate after graduation. In February 2011 he moved to the Robotics Institute, Carnegie Mellon University(US) and worked as Postdoctoral Fellow until July 2012. He joined the Department of Mechanical Engineering, Curtin University (AU) and was appointed Lecturer in Mechatronics, which he currently holds.

Links:

• Download mp3 (11.0 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Transcript

Ron Vanderkley: Dr. Lei, we can start by asking you to introduce yourself to the listeners.

Dr. Lei Cui:  Sure. My name is Lei Cui, I’m a lecturer in Mechatronics. I’m with the department of Mechanical Engineering at Curtin University. I joined Curtin about two-and-a- half years ago. Before that I got my PhD from King’s College, London, and I did my second Post-Doc at Carnegie Mellon University.

Ron Vanderkley: Let’s talk about the research that you’re doing at Curtin University. I know there are two facets to it. We can start with the first one: robot rehabilitation.

Dr. Lei Cui: I worked on robot rehabilitation when I was doing my PhD. After I joined Curtin, I decided it was a good time to pick this research up again, because the Mechanical Engineering Department is now shifting its research focus to rehabilitation and medical engineering. In the last two and a half years I’ve developed a 3D printable hand exoskeleton for rehabilitation, in collaboration with a colleague. We patented a finger design just three months ago. The unique feature of this hand exoskeleton is that each finger has three active joints, and, so, three active outputs, but with only one actuator – there’s one input. It’s the first design of its kind in the world.

It’s a joint linkage, I don’t know if you’re familiar with the concept of linkage? We applied the same linkage to all five fingers, so each can be operated by one linear actuator, but has three output joints corresponding to the three joints in each finger. Another feature is the whole assembly can be printed out using a 3D printer. No assembly work needs to be done, it’s for mass customization.

Ron Vanderkley: That’s a general 3D printer, printing in any kind of polymer plastic?

Dr. Lei Cui: No, we need a high precision printer. We need high resolution.

Ron Vanderkley: It’s not a hobbyist or …

Dr. Lei Cui: No, it’s not, any low level 3D printer cannot do that.

Ron Vanderkley: You told us that it’s at the point of being patented or patented already, are you collaborating with any other organization?

Dr. Lei Cui: Yes, we’re now working with a company in the US, from MIT. They’re also developing this hand exoskeleton, but our design is much better than theirs so we’ve been talking about how they could buy our design, or we could just print some collaboration frames so we can work with them.

Ron Vanderkley: You also spoke about the second project that you’re currently working on?

Dr. Lei Cui: Another project is an upper limb, for arm rehabilitation. On this project I have been working with Kerry Allison, a professor at Curtin from the Health Science School. Mainly, we’ve concentrated on task-oriented training. When patients do training they don’t focus on each individual joint in terms of rehabilitation. Usually, they focus on the task. For example, they want to move the hand from point A to point B: they normally look at the target first then move the hand from A to B. They don’t focus on each individual joint.

So that’s the idea behind our design. The ideal training would be for the user, or the patient, to look at the target, move to it and it would have false feedback to test the patient’s performance.

Ron Vanderkley: So it’s a variable device?

Dr. Lei Cui: No, it’s not variable, it’s a hand infector type. Not an exoskeleton type.

Ron Vanderkley: The primary use for that is to teach stroke victims?

Dr. Lei Cui: Yes, mainly for stroke patients.

Ron Vanderkley: What stage are you at with that process?

Dr. Lei Cui: We have developed a prototype and, currently, I’ve got one PhD student working on this project, but he’s only at the first stage. Now, we are combining force torque sensing and EMG sensing to do a patient intention estimation, it’s the first step. Previous work has used a predefined trajectory, meaning that the subjects first perform the task and they record the predefined trajectory; so they’ve try to force or assist the patient to follow the same trajectory. Our idea was to do a patient intention estimation first and combine EMG and force sensing. So, we don’t rely on the predefined trajectory, we want to generate it on the fly. This is the first step.

The second step would be to apply adaptive impedance control to assist training, as needed.

Ron Vanderkley: Thirdly, you are working on a three directional platform, is that correct?

Dr. Lei Cui: Yes, a three dimensional platform. That’s also a collaboration between Professor Gary Alison and myself. Basically, it’s his idea; he’s been working on this project for many years. What he wanted was a platform that could be multidirectional for the patients. This platform destabilizes the user. So, the user will be standing on top of the platform, it will move and the user has to move his or her foot to regain stability. That’s the part we need to control well.

Ron Vanderkley: What’s the end goal of it? Why are you creating it?

Dr. Lei Cui: Professor Alison’s research focuses mainly on how to prevent old people from falling. To study human posture and balance, you need to provide a well-controlled multidirectional platform for patients. That’s why it’s pretty large: 600mm in diameter. A person with an average weight of 100kg can stand on top of it comfortably. Also, they can shift a foot and regain their balance; we’ve provided enough space for that, and also enough speed and acceleration.

Ron Vanderkley: The other area of expertise you’re looking at is Biomimicry, could you tell us where you are going with that?

Dr. Lei Cui: Biomimicry is a fascinating research field; it’s also quite a hot topic. If we take the normal approach, using mathematical modeling, it’s very hard to realize some functions, that’s why we take inspiration from nature. Because nature – fish and other animals – has evolved over millions and millions of years. I’m focusing on applying Biomimicry design to monitoring.

The first project I worked on was called Amphibious Robot. We took inspiration from amphibious centipedes because they have multiple feet. These feet provide buoyancy and actuation, our design was decided with this in mind. It’s a track driven vehicle, but at each point of the two tracks, we’ve fixed ballpoint blocks so that, when the robot is in water, they provide buoyancy and actuation. When it’s on land, it can also effectively drive the robot.

Ron Vanderkley: At what stage are you at with that?

Dr. Lei Cui: We have finished the first stage, that’s the mechanical design and the empirical study; the paper’s been accepted by the ASME Journal of Mechanical Design. Now, some of my students are designing autonomous navigation systems, using an android form that also provides sensors, an accelerometer, gyros and a video camera.

Another project is the Robotic Fish.

Ron Vanderkley: Yes, we’ve heard a lot about the fish.

Dr. Lei Cui: In fact, this robotic fish is quite fast. It’s the fastest robotic fish in the world in terms of body length, per second and we’ve achieved three body lengths per second. A review paper in 2012 held the previous record: 1.6 body lengths per second.

Ron Vanderkley: So, it’s a first here in Western Australia?

Dr. Lei Cui: Yes, definitely, we just haven’t published anything yet. We want to wait until we collect enough data and it’s accepted by a good journal.

Ron Vanderkley: The propulsion of the fish…Biomimicry, is that using actuators?

Dr. Lei Cui: Yes. In fact, those kind of fish are in [the local] Swan Canning River, they’re called Bream. It’s quite a popular fish in this river. We effectively morphed this fish. If you put our robotic fish in the river, it would be very difficult to discern which is the real fish. The appearance is the same and the size.

Ron Vanderkley: Is there a plan to monitor the schools of fish?

Dr. Lei Cui: Yes. Our idea is that our fish could mix with biological fish seamlessly, because they could be made to look real. We hope, in the future, to use it to monitor the fish population in the Swan Canning River, for example.

Ron Vanderkley: Would you be able to mount sensors on them to take salinity tests, for example?

Dr. Lei Cui: Yes. Now we’re studying the actuation of the fish; we hope to further improve the speed, that’s the first task. The second task is to develop electrical field sensing technology for the fish, because they are un-tethered, small in size and battery powered. Currently, the batteries last for about 45 minutes but we need to make the fish autonomous. We’ve taken inspiration from some fish using electrical fields to sense their environments. I’ve got one PhD student and several undergrad students developing this form of bio-sensing, using electrical field sensing.

[0:17:53]

Ron Vanderkley: That’s very fascinating and impressive. I haven’t read anything on that for quite a while.

Dr. Lei Cui: Thank you.

Ron Vanderkley: We’d be very interested to find out how that’s developing, once you get going on it.

Now, the question that we ask everybody is this: Where do you think robotics will be, say, in 10 or 20 years from now? What kind of direction do you think we are heading in?

Dr. Lei Cui: I think robotics covers a lot of areas: artificial intelligence, mechanical design, sensing technologies; many of them are a combination of these. In my opinion, currently, sensing technology is holding back development in robotics. I would assume that, in the next 10 or 20 years, more people will be focusing on sensing. In terms of artificial intelligence, IBM has built machines to beat world chess champions. Artificial intelligence to do a single task is, I think, quite mature, but to multitask? Developing robotic animals that behave like the real thing still has a long way to go, it needs 50 -100 years, I’d guess.

Ron Vanderkley: Obviously, we’ve seen lots of examples, but predefined scripts give the audience the idea that some robots are more intelligent than they are?

Dr. Lei Cui: I’ll give just one example. While I was working at CMU, on the DARPA sponsored ARM [Autonomous Robotic Manipulation] project, one of my tasks was on the robot arm to hold a key, insert it into a lock and then unlock the door. If you watch the video, it seems quite intelligent. It appears as if it can feel what it’s doing, but my approach was to use force torque sensing on the wrist of the robot and then to insert the key into the hole. It looks intelligent but, in fact, as you’ve said, it’s defined by algorithms.

Ron Vanderkley: To me, that always confuses people and detracts from the research, that should be the focus. It presents the wrong impression. Similarly, movies like Terminator, also paint the wrong picture.

Dr. Lei Cui: Yes, definitely. I think it would take 200 years, at least, for Terminator to be a threat.

Ron Vanderkley: So, I’ll wrap it up by saying thank you very much for taking the time to do the podcast, and we hope to chat with you again in the future.

Dr. Lei Cui: Sure, thank you.

All audio interviews are transcribed and edited for clarity with great care, however, we cannot assume responsibility for their accuracy.

Image: MIT d’Arbeloff Laboratory

In this episode, Audrow Nash interviews Federico Parietti, a PhD candidate at the Massachusetts Institute of Technology, about his research on supernumerary robotic limbs that can be used in manufacturing and for rehabilitative purposes, among other uses.

The videos below demonstrate how supernumerary limbs can be used to assist in tasks. This research was done in the same lab that Federico works in.

Federico Parietti

Federico Parietti is currently a PhD candidate at the Massachusetts Institute of Technology, where his research focuses on the design and control of wearable robots and man-machine interfaces. Previously, Parietti was a Research Associate and Visiting Scholar at Carnegie Mellon University and an International Student at ETH Zurich, in Switzerland.

Links:

• Download mp3 (10.5 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Transcript

Audrow Nash: Hi, welcome to Robots Podcast. Can you introduce yourself?

Federico Parietti: I work on wearable robots at the Mechanical Engineering Department at MIT. I’m happy to be here with you today.

Audrow Nash: We are too. Can you tell me the goals and motivation behind your research?

Federico Parietti: I work on wearable robots – essentially robots that you can put on. These robots are built to give humans something new with respect to what they have in their natural bodies: they can make you stronger, can make you faster, or can provide you additional information, like for example, the Google Glass. What I’m working on right now is physical augmentation; by wearing my robots, a human can essentially become stronger, or avoid fatigue, or they can acquire skills that were not possible before.

Audrow Nash: Can you tell me what it looks like?

Federico Parietti: Essentially, it’s two additional arms. When you’re wearing the robot, you’re essentially wearing a type of backpack harness and you’re carrying around two additional arms that can help you do tasks, coordinate with your hands. It can also reach for the ground or reach for the environment and support you.

Audrow Nash: And also they’re located at the wearer’s hips?

Federico Parietti: Yes. And since as you point out these two additional arms are essentially springing out of your hips, they can act as legs, but they can also act as arms. They can reach where you can’t reach, even behind you almost like a tail, so there’s really no limit. That’s why we call them additional limbs, because there’s not really a particular definition.

Audrow Nash: So how is this different from exoskeletons, with the extra limbs?

Federico Parietti: That’s a very important point, because these limbs are independent. They can move on their own without having to copy the motion of the human. In an Ironman style exoskeleton, the robot has to follow the human, so that as you move the robot follows you because the robot is built as a suit.

In our case, the robot is wearable, but it’s also independent. So you can move, you can do whatever you want, and the robot is free to coordinate in an independent way. Some people say this is a “Doctor Octopus” kind of robot, because their arms are independent.

Audrow Nash: What kind of sensors and actuators are on the wearable system?

Federico Parietti: Each one of the two robotic limbs has three actuators. We have two rotation motors in the shoulder of the robot and they can pick the direction where the robot moves. And then the legs are telescopic so they stand and retract. This means that the robot can essentially reach any point in space, within a workspace that is essentially a sphere.

On the robot we have four sensors and position sensors so the robot knows where it is and what it’s touching and how strong it’s pushing. And then we also use sensors on the human, which is very important because we have to detect human motion, and sometimes also human muscle activity and brain activity. And we also have some experiments on gaze, so we can detect where the human is looking so the robot coordinate.

Audrow Nash: The arms are attached using a ball-and-socket method. Can you tell me a bit about that and the advantages of it?

Federico Parietti: The arms are attached to the base of the robot through a ball and socket – a joint that is a sphere. And it’s what we have essentially in our shoulder that allows us to rotate our shoulder around in all directions, because it’s like a sphere. The advantage is that whenever the robot is pushing on the ground, the forces go from the ground through the center of that sphere, and this means that you can sit on the robot arms without wasting energy. The robot stays there and you don’t really need the motors to lift you.

Audrow Nash: Can you give me some applications and situations where these supernumerary limbs might be used?

Federico Parietti: Our first scenario of application was aircraft assembly. We visited aircraft assembly factories and realized that for some tasks there were two or three workers collaborating for the same task, but only one of the two or three was doing the important task, for example drilling. The other two were just helping with boring and very tiring tasks like lifting parts or following the first worker with a vacuum cleaner. So we asked, why can’t a robot do that? Why can’t a wearable system do that and help the two workers?

Then we also realized that a lot of workers get tired because they have to work in uncomfortable positions. A robot like this can compensate for the weight of the user and make working much easier and less tiring. And then finally we also realized that there are a lot of people that, either because they’re rehabilitating or because they’re old, they need additional legs or additional arms in order to keep a comfortable lifestyle to avoid the risk of falling down. And so this robot can also help them.

Audrow Nash: So when they’re taking their step one leg is in front of the other and the weak directions are perpendicular or diagonal to the way they’re heading?

Federico Parietti: Yes, this is is a very interesting point. When we walk we are unstable and so we risk falling down, and falling is very dangerous for us because we are bipedal – it’s catastrophic essentially. The curious thing is that even in martial arts, like in judo for example, when you have both feet on the ground, if you push in the direction that is perpendicular to the line that joins your feet, you fall down. So a lot of martial arts techniques block your legs and then push you in a perpendicular direction so that you fall down easily.

That’s the weak point, but how can we use a robot to improve it, to actually remove that weakness? If we put the robot in the weak direction, then you can’t fall anymore because even if you slip in that direction, there is a robotic leg already in the ground there. So the idea is that when we walk, the robotic legs swing where our legs are not and so we’re always very stable.

Audrow Nash: So they swing similar to our arms being opposite to the foot that’s stepping forward?

Federico Parietti: Yes, it’s something similar to that essentially.

Audrow Nash: And they plant themselves on the ground in theory?

Federico Parietti: Actually for the purpose of balance assistance, they don’t even need contact with the ground; they just need to be in the right place close to the ground.

Audrow Nash: How does this research compare to similar work?

Federico Parietti: The fact that our limbs can be both arms and legs makes us both similar and different to a lot of robots, both robotic manipulators that are typically arms, and to exoskeletons that are typically legs.

Let’s talk first about the legs. There are a lot of exoskeletons right now that are used by paralyzed people to walk again so that they don’t have to use a wheelchair. These exoskeletons follow the legs, and they move the legs along a trajectory. They essentially do only one step after the other, but they can’t really keep balance so users still have to use their arms and crutches to balance. Our robot would be complementary with these exoskeletons, so that if you’re paralyzed, you can use a simple exoskeleton to move your legs and these additional “robotic crutches” to avoid falling down.

There are also robotic walkers that have been developed for elderly people, but these are essentially robots with wheels and so you can’t go up and down stairs, and they still require the elderly person to use their hands. In our case you don’t really need to use your hands, the robot just follows you.

Audrow Nash: What are some challenges of this research and what are some lessons learned from these challenges?

Federico Parietti: There are so many challenges. One of them is to make the robot comfortable. People have to wear this robot, so you have to make it as easy to wear and as light as possible. Weight might seem an easy thing to do, but if you need a robot that can lift you, you’re going to need some mass, and some pretty strong actuators. Right now, the goal of having the robot light, but still functional and powerful is really driving us towards exotic materials like carbon fiber or fiberglass. We can’t use metal anymore; it’s too heavy. That was one lesson learned: try to make it as light as possible, because that’s essential for users.

Another challenge is the interaction with the human, because we have independent arms – so just copying what the human is doing is not enough. We need to have some kind of independent source of information from the human. We have to find a way so that the human can control these legs without sacrificing anything. So for example, if we use our arms to control robotic arms, additional robotic arms, that would essentially miss the point, because we’d still be using our two arms, right?

How can we use human actions to extract independent signals so you can drive additional arms without limiting your own? For example, we’ve strapped an accelerometer to a person’s legs so that when they are just walking normally, we are detecting the frequency of the steps. The robot knows when they are stepping without them having to tell the robot, and the robot can just synchronize with their steps autonomously. The idea is that you can still do whatever you want with your arms and you can walk at any speed with your legs, but the robot synchronizes without you telling it how. Of course this is a simple periodic example, but we are interested in how we can we keep this a very intuitive interaction.

Audrow Nash: Once you have an intuitive interaction, do you think you’ll have the arms and legs swapping back and forth? Like being used as both, within the same system?

Federico Parietti: Yes, there’s really no limit. Once you have a good interface, you can do whatever you want. You can use the robot to pass you objects that you need, but if you’re using your hands you can’t really reach for them. You can use it for locomotion, for safety … We actually don’t know how many uses there are for additional arms, because that’s not in our nature; no one has ever had additional arms. I bet that if we provide a reliable human-machine interface people will just come up with new uses.

Audrow Nash: Going forward, what are your plans and goals?

Federico Parietti: It would be good to bring this robot out of the lab and into society, either for manufacturing applications and helping workers, or for rehabilitation, or assisting elderly people.

We already have a lot of patents either filed or about to be filed on these robots. It’s a pretty solid and unique technology because of the independence of the arms, and we think it’s very promising. This is a very good moment for start-ups and ventures, and this could be a good application.

Audrow Nash: What advice do you have for beginning researchers?

Federico Parietti: Do what you like. Refuse to do something that you don’t like, because it’s hard. This is the fourth product I’ve built in the last three years, so it’s a lot of work, it’s a lot of learning, there’s a lot of obstacles and sometimes nothing works. The only way to survive is if you love what you’re doing. So if you love it, there is really no problem, you can do whatever you want. I would say to researchers to pick the field that you really love, stick with that even if people tell you to go elsewhere, because doing what you like is essential to actually doing it well.

Audrow Nash: It’s interesting that you brought up how many iterations you’ve done with this. Can you talk a bit about that? The rapid turn-around time you’ve had?

Federico Parietti: I think it is a little bit extreme in our lab; no single product for me has ever lasted more than six months – and this is not because they break, it’s because we move on to the next thing. We review a product to test a new control idea, or to test a new design idea, and then we see if it works or if it doesn’t, and then we start immediately to build a new product that takes in account those lessons. That’s how we realized the robot that we have now, which is much, much lighter with respect to the first one. We realized, for example, that joint torque is not that important if the robot is heavy. It’s much better to have a slightly weaker robot but also lighter robot IF you use those torques in a smart way, like with the ball and socket joint for example. So right now we are talking about the latest product, but in six months we’ll have a new one.

Audrow Nash: Do you think all these iterations are beneficial?

Federico Parietti: Yes, it’s very beneficial. Sometimes we come up with applications that are new, and if I was stuck using my first product, I wouldn’t be able to use the robot for it because it was designed to do something else. All these iterations allow us to be very active and follow the hottest topic; we can incorporate the feedback that we receive at conferences very quickly.

Audrow Nash: Wrapping up, what do you believe is the future of robotics?

Federico Parietti: I see the future of robotics as divided in two huge areas: there are autonomous robots and there are wearable robots. The goal of autonomous robots is to be as intelligent as possible in order to act as a support or helper to the human, but an independent helper – a kind of robot butler. And then there are wearable robots, and their goal is not really to help as an external agent, but to make the wearer better. It’s essentially human augmentation, and that’s the field I’m working in.

The challenge for us is not really to make the robot smart or to give it artificial intelligence. Our challenge is to figure out how to make the robot as useful as possible to a human. It’s not an easy challenge, because we don’t currently have sensors that tell us what the human is thinking, or what the user wants to do. Our robot must be very smart in understanding what the user wants to do and doing it in the best possible way. Essentially we’re in the process of building extensions to the human body, so we need to understand both the mechanical and the neural part of the process, and that’s the challenge that I think is unique to wearable robots.

Audrow Nash: Thank you.

Federico Parietti: You’re welcome.

All audio interviews are transcribed and edited for clarity with great care, however, we cannot assume responsibility for their accuracy.

The Pleurobot (Photo: Hillary Sanctuary & BioRob).

The Pleurobot is a bioinspired robot being developed by the BioRob at EPFL and NCCR Robotics. Taking it’s cues from the salamander, the Pleurobot is a walking robot that can change its gait to help it to navigate uneven terrain, and is currently learning to swim.  Watch the video to see the researchers discuss what they are doing with the Pleurobot and how they hope to improve it in future.

For further information on the Pleurobot please see:

The project’s homepage

Introducing Pleurobot (video)