Last month we caught up with Dario Floreano, the head of the Laboratory of Intelligent Systems at the Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland. Here we continue our discussion, covering acoustic sensing, multi-drone operations and more. Missed Part 1? Check it out here.

Last time we spoke, we discussed your work on visual perception, but you also mentioned acoustic research. So this means we’re talking about ‘listening drones’?

Yes, that’s right. This is particularly interesting in the area of drone swarming, whereby multiple robots need to coordinate their flights very carefully.

The first big problem with swarming is perceiving the other drones. If you have multiple drones in flight, each robot is only a very small spot in the other robots’ visual fields of view. They’re very hard to spot against the background. So we thought, what about sound detection?

The question is, how can a robot hear other robots when it is in flight? Well actually, this is a simple trick. You simply switch the engine off, very briefly, listen using three tiny microphones pointing in different directions, then restart the engine. Just like the eBee does when it is capturing aerial photos. Using this trick we developed drones that can detect the presence and relative direction of other drones acoustically.

Outside of the lab, what do you see are some of the potential end applications of your work?

Sticking with the acoustic approach, this could be useful in search and rescue, being used to detect people’s locations on the ground. If people are lost in snow or in a forest, they often have emergency whistles. We have shown that we can locate a person using a standard emergency whistle and broadcast its GPS coordinates to a rescue team.

With swarming, we’re also looking at using drones as relay networks for communication between rescue and security teams on the ground.

In the event of an earthquake, a fire or a landslide, the first thing an emergency team usually does is to set-up its communication infrastructure. This typically consists of a set of trucks with radio signal emitters and receivers, which operate on a specific frequency. It can take one or several hours to complete this set-up before rescuers can start working on the ground, using their receivers.

However drones can be equipped with emitters and receivers. Therefore a swarm of drones could cover the area, pick up the signals of the rescue team on the ground, and relay these, connecting people via very low power, short-range emitters. Currently with this technique we can connect multiple people, and not only with audio but with video signals too.

We’re also experimenting with multiple people moving around on the ground, with robots following their movement, and we’re considering operations that last longer than a single battery duration of around half an hour. We’re developing an algorithm that allows drones to land for a battery change, while new drones with a full battery launch to continue the service.

So on one hand your work is bio-inspired research, but you’re obviously also thinking about the potential real-world applications of the technology you develop. Which point do you start from—do you look at biology first and then consider how to use the tech that comes out of that research, or do real-world problems drive the direction your biological research takes?

It’s a mix. When it comes to drones, 15 years ago it was more blue sky research. These days however I have more projects on flying robots that could solve problems that people might face, not necessarily next year but in five years’ time. For example, if you will have lots of drones all in the air, you then think about safety. That means thinking about AI and sensing.

The specific applications are not always there though. For example, when the Gimbal was designed we didn’t have an application in mind, just a problem: how to design a robot that could withstand falls and fly again. We started by adding retractable legs, as insects have, but this wasn’t optimal, so we came up with a rolling cage around the drone. Then we realised the application: being able to go where other drones couldn’t go.

Prof. Floreano’s work at the Laboratory of Intelligent Systems (LIS) is bio-inspired; he and his team develop flying robots with animal characteristics, which they also use to learn more about animals.

In swarming there are lots of interesting problems to think about. In the future, as I mentioned, we’ll have a very high number of drones sharing our airspace. We’ll have a lot of crowding in areas such as landing spots and drone airways.

The big question is this: how will these drones avoid each other? Commercial GPS isn’t good enough to enable this avoidance when you fly very close to other drones. So the question is: what are the best sensors and the best control algorithms, both for drones flying in cooperative conditions—meaning the same types of drone flying in the same area—and non-cooperative conditions featuring drones from different manufacturers using different sensors? This is a real problem that some of our research is addressing, regarding the sensors and algorithms we’re developing.

Another real problem comes in transportation: how can we transport parcels very safely? I can’t talk about the solution just yet, but our work there builds on a lot of the topics we’ve been discussing.

Which seems a logical place to wrap up! Thank you so much for all these insights. How can readers who are interested in learning more follow your lab’s work?

The homepage of the Laboratory of Intelligent Systems is at lis.epfl.ch. That page includes our most recent publications. We also have a LIS YouTube channel that people can subscribe to and I am on Twitter at @DFloreano. 

You are also involved with the RobotsPodcast, is that correct?

Well, I was involved in its creation, back when it was called Talking Robots, and I supported it from my lab for two years. The podcast grew and grew, and the students that were preparing it then set-up what is known today as theRobotsPodcast. Many members of LIS still work on this on a voluntary basis. It became so successful that it also spurned another, wider organisation, calledRobohub.

Thanks for your time!

You’re very welcome.

Watch Floreano’s ROBOTS14 presentation:

===========================
About Dario Floreano

Dario Floreano (M.A. 1988; M.S. 1992; Ph.D. 1995) is the director of the Laboratory of Intelligent Systems and director of the Swiss National Center of Competence in Robotics. He is co-founder of the company senseFly S.A., of the International Society for Artificial Life, and founder of the popular robotics podcast series Talking Robots (which later became RobotsPodcast).

He has been on the advisory board of the Future and Emerging Technologies division of the European Commission, vice-chair of the Global Agenda Council on Robotics and Smart Devices of the World Economic Forum, and on the board of governors of the International Society for Neural Networks. Floreano is a senior member of the IEEE and received several national and international awards for professional and cultural achievements. Floreano has published four books, more than 300 technical articles, and more than 10 patents in drone technologies.
===========================

As head of the Laboratory of Intelligent Systems at the Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland, Dario Floreano knows a thing or two about flying robots. In the first segment of a two-part conversation, Waypoint caught up with Floreano to discuss his team’s drone research, the potential applications of such innovations, and more.

Could you start by giving our readers a little background on the Laboratory of Intelligent Systems (LIS) at EPFL?

Yes of course. I started the laboratory in the year 2000 and today we have around 20 researchers. senseFly’s CEO [Jean-Christophe Zufferey] was actually one of my first PhD students. 

I had always been interested in bio-inspired artificial intelligence and robotics. I had two interests in particular. First was designing systems with animal characteristics; robust systems capable of solving problems that couldn’t be predicted in a changing environment. And the other interest, by contrast, is using these tools to better understand biology.

At LIS we work closely with biologists to extract principles from animals, such as how their vision works, then we translate these principles into control algorithms to manage a robot’s performance as the corresponding animal does. In other words, we use biology to design better robots and we use robots to understand more about biology.

Dario Floreano is a leading researcher in the field of flying robots, as well one of senseFly’s original co-founders.

Where does this bio-inspiration lead you in practice? What are some key themes?

Our activities focus on three core themes: flying robots, soft robots and evolutionary robots.

Since Waypoint is all about the professional use of flying robots, or drones, could we dig down into this area some more? Why flying robots? What’s the specific interest there?

Why flying robots? Because they’re very challenging!

They need to be very lightweight but at the same time you have to include energy, computing power, sensors… this is a really challenging job. This challenge is why I thought bio-inspired solutions would provide an advantage compared to traditional approaches. That’s the point we started from when Jean-Christophe Zufferey was working in my lab originally.

So what has your drone research typically looked into, and how do you see this work evolving in future?

If you want to have robots capable of flying in cluttered environments, they need to be able to ‘see’ in these spaces. So right now we’re looking at different sensor modalities: vision, in particular compound vision, which insects use, and acoustic sensing. The latter is mainly used for coordinating the flight of multiple drones, which we call swarming. We also do research in drone communication for coordinated flight and aerial telecommunication networks, but this work is less bio-inspired.

Other thing one needs to look at is how the body of a robot is designed. They need to be very agile to fly in cluttered environments so we’re looking at robots with adaptive morphology, drones that can change their shapes to transition between fixed wing and rotary formats, which each have their own pros and cons in terms of flight times and agility.

On this morphology front, you’ve been looking into folding drones?

Yes. So with morphology we’re also looking at size. In order to carry larger payloads, typically this means a larger drone. You have two choices when looking to carry larger payloads: either fly faster or accelerate propeller rotation, or increase the surface of the aerial surfaces. What people typically do is to increase the size of the props or the wings. This ends up with larger drones, which leads to problems transporting them and when making them fly through narrow passages. So we’ve been looking into the concept of foldability. We are looking at origami technology to make drones that you can easily carry on your pocket. There’s also the issue then of controlling such systems; how to make such drones stable after they open.

We’re also studying artificial muscles made of soft actuators, which could be used one day for foldable drones.

When you fly in really difficult situations— with things like cables around—then sooner or later there will be contact. This brings the question: how do you protect the robot, the environment and the people who come into contact with it?

One very simple way is to add a cage or a gimbal (such as Flyability’s Gimbal, which is another spin-off of my laboratory), and another way we’re now looking at is using new types of materials. Materials that minimise damage, absorb collision and then revert back to their original shape.

I’d like to go back to perception. You mentioned working on two sensor modalities: vision and acoustic sensing. Can we explore the first a little further?

Sure. When it comes to a robot’s perception, we’ve been working on vision for a long time. Most recently in this area we’ve been working on designing new types of vision sensors that mimic the compound vision of insects.

Conventional cameras are designed on the same principles of the vertebrate eye whereby an image is perceived via a very dense, high number of photoreceptors under a single lens. The advantage of this conventional approach is very fine picture quality, but on the downside you have a very limited field of view.

Now, when you have a drone, you want to have a very wide field of view because you need to see in many more directions than a terrestrial animal. But if you simply increase the number of cameras to achieve this, then you increase the weight. Insects on the other hand, with their compound eyes, cover a very wide field of view by using hundreds of simpler miniature eyes, with just a few photoreceptors under each lens pointing in the desired directions. Their perception is the mosaic image that results from these thin eyes.

So we decided to design new miniature sensors made from very simple cameras, each with between one and three photoreceptors, and a very narrow view lens. Only instead of three or four of these cameras we have hundreds of them. This insect-like approach gives a very wide field of view and the insect-inspired photoreceptors give us very fast reaction time; we can extract optic flow three times faster than insects can! However the drawback is that the images are not very fine, so these cameras are for navigation, not for taking pictures. (Read the paper: Miniature curved artificial compound eyes, or see the Robohub article here. )

The team at LIS has adopted an insect-like approach to vision – designing new miniature sensors made from very simple cameras, each with between one and three photoreceptors and a very narrow view lens.

What stage of development are you at with these multi-camera, insect-vision robots?

We have produced a few first prototypes. The cameras themselves are less than a millimetre thick and each weighs less than a milligram. We can also arrange them on fully flexible substrates, which I call “vision tapes”. These are packed full of sensors and you can wrap them around parts of the robot.

The challenge with our current prototype is that it has a very limited vision range—it can only see up to a maximum of 50 cm. So we’re working on designing of new type of sensor, which has a longer range but is still very simple to produce.

Thank you very much for your time!

You’re very welcome.

Read Part 2.

===========================
About Dario Floreano

Dario Floreano (M.A. 1988; M.S. 1992; Ph.D. 1995) is the director of the Laboratory of Intelligent Systems and director of the Swiss National Center of Competence in Robotics. He is co-founder of the company senseFly S.A., of the International Society for Artificial Life, and founder of the popular robotics podcast series Talking Robots (which later became RobotsPodcast).

He has been on the advisory board of the Future and Emerging Technologies division of the European Commission, vice-chair of the Global Agenda Council on Robotics and Smart Devices of the World Economic Forum, and on the board of governors of the International Society for Neural Networks. Prof. Floreano is a senior member of the IEEE and received several national and international awards for professional and cultural achievements. Prof. Floreano has published four books, more than 300 technical articles, and more than 10 patents in drone technologies.

The foldable quadrotor is small enough to fit in a hand (Photo: LIS, EPFL, NCCR Robotics).

The use of robots to find victims after natural disasters is fast becoming commonplace, with well documented cases where robots have been sent into areas too dangerous for rescue workers.  While the issues surrounding robustness, control and autonomy are frequently cited as key areas for research, a  team from LIS, EPFL and NCCR Robotics is working on another important aspect, how to make flying robots easily transportable and quick to deploy.

In a paper, which will be presented at the International Conference on Robotics and Automation (ICRA) 2015, the team describes how they made a quadrotor with rapidly self-deploying, folding arms. Foldable aerial platforms currently use either beams or detachable appendages (e.g. DJI spreading wings series and SenseFly eBee), which require manipulation by the user, slowing the process down and introducing the possibility of human error. The presented prototype has self-deploying arms that go from completely stowed to ready-to-use in 0.3 seconds, meaning that multiple drones can be utilized at once in a very short period of time.

The origami arms are stored using a system of strategically placed magnets, which can use the torque and thrust generated by the propellers to break the magnetic seal, thus allowing the arms to deploy when the quadrotor is switched on. In order to make use of such a simple mechanism, the design team carefully considered the placement of the folds in the arms; to take advantage of the propeller’s torque, folds are parallel to the its axis.

The origami structures that make up the arms allow them to fold around the central core, reducing its volume to about a third of commercial quadrotors of a similar weight. To make the origami structures, the arms are constructed from a 0.3mm layer of fiberglass over Icarex, a lightweight and inextensible fabric. Two vertical folds in the 2D structure allow the arms to lie in the wrapped configuration. When the motor begins, the action of the propellers forces the magnets apart and allows the arm to straighten in its flat configuration, whereupon a second set of magnets attract and pull the horizontal fold (down the centre of the arm) closed, creating a stiff, 3D structure with no need for heavy locking mechanisms.

First author Stefano Mintchev holds the quadrotor (Photo: LIS, EPFL, Alain Herzog).

The folding quadrotor currently doesn’t have an automatic “putting away” mode, meaning that the arms must be manually returned to the wrapped configuration. However, at the deployment stage, where speed is of the essence when trying to locate vulnerable people, this mechanism may just be a lifesaver.

Reference

S. Mintchev, L. Daler, G. L’Eplattenier, L. Saint-Raymond & D. Floreano, “Foldable and Self-Deployable Pocket Sized Quadrotor,” In 2015 IEEE International conference on Robotics and Automation (ICRA 2015), Seattle, Washington, USA, May 26-30, 2015.  A pdf of the full article can be downloaded from infoscience.

The DALER is a winged robot that can also walk.

The issue of how to use one robot across multiple terrains is an ongoing question in robotics research. In a paper published in Bioinspiration and Biomimetics today, a team from LIS, EPFL and NCCR Robotics propose a new kind of flying robot that can also walk. Called the DALER (Deployable Air-Land Exploration Robot), the robot uses adaptive morphology inspired by the common vampire bat, Desmodus rotundus, meaning that the wings have been actuated using a foldable skeleton mechanism covered with a soft fabric so that they can be used both as wings and as legs (whegs).

The DALER is being designed to be used in search and rescue operations, and its dual modes of locomotion will enable it to fly long distances to survey large spaces in a short timespan, and then to walk into dangerous or inaccessible areas.

One of the main problems in trying to get a flying robot to walk in small spaces is the surface areas of the wings. The wings of a flying robot must be wide in order to maintain flight, but this makes it difficult for them to fit into smaller spaces. After observing D. rotundus, the research team used a foldable actuator that allows the wings to fold into a smaller space, and to rotate around a hinge that attaches the whegs to the body in order to enable walking. D. rotundus doesn’t look very pretty when it walks, but it has certainly perfected the art of using all four limbs for two purposes.

The DALER using its wings to walk. Wingerons at the end of the wings rotate to facilitate walking.

To design the robot, the team had to first designate the primary mode of locomotion – in this case flight, as the DALER will cover the longest distances this way. The wings were then adapted for walking, taking into consideration the need to not add extra weight when the DALER is in flight mode. The resulting wing is equipped with wingerons, with an axis of rotation close to their centre of aerodynamic pressure and physical centre, so as to satisfy the constraints of both ground locomotion and flight. The best shape found for these wingerons is a triangular one.

The addition of wingerons poses a problem regarding the use of actuators. Flight maneuverability requires rapid wingeron movements and low torque, whereas walking requires the opposite conditions. To cope with this problem, the DALER uses a terrestrial actuator configuration, but with an optimized wingeron size, to force the working point of flight control inside the operating range of the terrestrial actuator, while not compromising flight maneuverability.

A second issue in creating a fixed wing drone capable of walking on the ground is with the morphology, thanks to the different centre of mass requirements for the two modes of locomotion. For this reason the DALER is equipped with foldable wings, so, much like the bat that inspired it, the DALER can walk on its haunches. These adaptations mean that the DALER can reach 20m/s in the air and 6cm/s on the ground, with a maximum step distance of 6cm.

Researchers hope that in the future the DALER might be used to find victims in dangerous areas after a natural disaster, for example. They envision the DALER being remotely deployed to fly to an affected area, and then walking through a disaster zone (e.g. a damaged building) to locate victims so that human rescue teams can concentrate their efforts where they are needed, rather than using time to search for victims in a dangerous environment. Future development of the DALER will include the ability to hover and to take off autonomously from the ground – necessary features if the robot is to return safely to its home base after a mission.

If you liked this article, you may also be interested in:

• Multi-purpose wings allow flying robot to walk across rough terrain
• Robots Podcast: Origami robots
• Robot bodies and how to evolve them
• ShanghAI Lectures: Josh Bongard “Morphological Change in Machines Accelerates the Evolution of Robust Behavior”
• Robots Podcast: Bio-inspired locomotion

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Our colleagues at the LSRO at Switzerland’s EPFL have sent a great video of their robot Thymio converted into a sushi-go-round for their XMas party:

Colleagues at the AI Lab’s Robotics and Perception Group also sent a video:
http://www.youtube.com/watch?v=dT_UISs6Vgw

Imina Technologies brings us a choir of miBot micromanipulators.

Previously featured on Robohub, General Motors and Kiva Systems feature high-tech robots pretending to be “Santa’s little helpers”.

http://www.youtube.com/watch?v=zHwGCiUOfgs&feature=share&list=PL5UWrgmuniqLsszErT_s9Gm74MGhx5AyA

Kiva’s video can be found here:
http://www.kivasystems.com/santas-high-tech-helpers/

While the industrial robots work away, Nao has been busy with his xylophone and well wishes.

Aisoy robotics brings us this fun video showing the challenges when filming robots with “personality”.
http://www.youtube.com/watch?v=XYfXxlNZhGM&feature=share&list=PL-NcJDWCD71gWi4Oc1nxGHfpUePcPEP7v

And in case you missed the holiday video from ETHZ featuring balancing, flying and legged robots, here it is.

For more holiday videos, have a look at the ROBOTS Podcast’s Annual Holiday Season Video Special Youtube Playlists. To submit your own, just send us a message at christmas@robotspodcast.com!