Source: Microsoft Research

On the surface, the challenge sounds rather easy – at least for a human. Given a cluttered cafeteria-style room as shown on the right, teams would have to program their robot to navigate autonomously to five specific locations all while avoiding tables, chairs, sofas, people and other objects. Each team would be given a limited time to first map the room with their robot and make note of the five locations. Each location was designated by a marker on the ceiling which would never be visible to the robot itself but could be used by the programmers during mapping to know when the robot was at a target location.

During the test run, the five location numbers would be given in a specific order to each team and the challenge was for the robot to move autonomously (i.e. without further control from the programmers) to each of these locations in the correct sequence and as quickly as possible without running into things or getting stuck. To make the task even more challenging (seemingly impossible when we first read about it), during the interval between the mapping and test phases, furniture and other objects could be moved, added or removed from the room. As if that weren’t enough, a number of people would be walking around the room during the test phase periodically crossing the robot’s path or even standing directly in its way for up to 30 seconds.

Source: Microsoft Research

All teams would be given the same robot, a Pioneer 3-DX (shown on the left) from Adept Mobile Robots. The contest was co-sponsored by Microsoft who provided the Kinect for Windows depth camera located on the vertical post attached to the back of the robot and facing directly forward. The camera was the only sensor that would be available to the programmers. There were no sonar or IR sensors and no laser scanners. So all mapping, navigation, and obstacle avoidance would have to be based on vision alone.

To understand why this task seemed nearly impossible, it helps to keep in mind two important points: (a) the robot’s camera was only 18 inches (46 cm) off the ground and (b) the most widely used automated mapping technique (SLAM), typically relies on a laser scanner with a wide field of view (e.g. 180 degrees) while the Kinect has a visual field of view of only 45 degrees. So unlike a person’s view of the room, the robot would be looking at objects from roughly the same height as someone crawling on their knees with a forward facing cone tied to their head blocking all peripheral vision. And even if we did have a proper laser scanner, imagine what it would see when sweeping across the room: basically just a large number of narrow chair legs and table pedestals that could be moved around anyway before the testing phase.

Even so, we would at least have the entire RGB video stream from the Kinect camera as well as its depth data. So what would a human do with such data while moving around the room trying to remember the five target locations?

Psychologists and biologists have known for some time that people and animals use visual landmarks to help localize themselves in a given environment, and fortunately for us, robotics researchers have been working on visual mapping and localization (“Visual SLAM” or “RGBD-SLAM”) for a number of years now. The most promising of these efforts appears to be the work coming out of the IntRoLab at the Université de Sherbrooke in Quebec. In particular, Mathieu Labbé has developed a Real-Time-Appearance-Based Mapping algorithm (RTAB-map) that stitches together visual features from an RGB-Depth camera like the Kinect and creates a truly amazing three-dimensional representation of rooms or other surroundings as shown in the video below:

Armed with RTAB-map, the SV-ROS team managed to borrow a Pioneer robot ahead of time and got to work programming all the control scripts, launch files, and configuration parameters needed to handle the requirements of the contest. In the meantime, Pi Robot had joined the party and was able to test the mapping procedure using his own Kinect camera. I also created a simulated cafeteria setting in Gazebo so that we could run a virtual mock-up of the contest over and over again with different navigation parameters to determine which settings resulted in the best performance.  We stayed in contact via e-mail and were trying out new ideas right up to the last minute.

In the end, four of the team members (Greg Maxwell, Steve Okay, Ralph Gnauck and Girts Linde) flew to Chicago several days ahead of the event and fine tuned the robot’s behavior even further. They even had the good fortune to meet up with Mathieu Labbé who gave a paper on RTAB-map at the IROS conference. Of the six teams that entered the contest, only one other team made as many waypoints as our robot (3 out of 5 locations), but our robot did the course twice as fast so we came out on top. The fourth waypoint was at the end of a long featureless hallway that completely confused the vision-based localization methods used by RTAB-map and was not a situation that we anticipated or tested. At least we know better for next time!

Robohub will be bringing you a write up about RTAB by Mathieu Labbé in the coming days. You can also read the official press release from the SV-ROS user’s group on the ROS.org newsfeed.

Source: SV-ROS

In the 9th part of the ShanghAI Lecture series, we look at ontogenetic development as Rolf Pfeifer talks about the path from locomotion to cognition. This is followed by two guest lectures: The first one by Ning Lan (Shanghai Jiao Tong University, China) on cortico-muscular communication in the nervous system, the second by Roland Siegwart (ETH Zurich) on the design and navigation of robots with various moving abilities.

The ShanghAI Lectures are a videoconference-based lecture series on Embodied Intelligence run by Rolf Pfeifer and organized by me and partners around the world.

Ning Lan: Cortico-Muscular Communication of Movement Information by Central Regulation of Spindle Sensitivity

Ning Lan is Professor at the Shanghai Jiao Tong University in China. He tells us about cortico-muscular communication in the nervous system.

Roland Siegwart: Design and Navigation of Wheeled, Running, Swimming and Flying Robots

Roland Siegwart is professor at ETH Zurich and the director of the Autonomous Systems Laboratory.

Robots are rapidly evolving from factory work-horses, which are physically bound to their work-cells, to increasingly complex machines capable of performing challenging tasks as search and rescuing, surveillance and inspections, planetary exploration or autonomous transportation of goods. This requires robots to operate in unstructured and unpredictable environments and various terrains. This talk will focus on design and navigation aspects of wheeled, legged, swimming and aerial robots operating in complex environments.

Siegwart presents wheeled inspection robots designed to crawl into machines and take various measurement, quadruped walkers that exploit natural dynamics and serial elastic actuation, swimming robots and autonomous micro-helicopters used to inspect cluttered, GPS denied cities or narrow indoor environments. He also presents a small fixed-wing airplane capable of staying in the air indefinitely due to its solar powered generator.

http://www.youtube.com/watch?v=1jZtOlIrxjY

Related links:

• Slides (17.4 MB)
• ShanghAI Lectures website
• Book: How the Body Shapes the Way We Think

Last week, SenseFly and Pix4D announced deals with drone maker Parrot, in which Parrot will invest in both companies, 5 million Swiss francs in SenseFly and 2.4 million in Pix4D. Both spinoffs of EPFL, SenseFly and Pix4D have a history of cooperation, with SenseFly providing the camera-equipped UAVs for which they have also developed navigational software that allows them to fly complete missions autonomously, and Pix4D providing the software that transforms the thousands of images produced by the drones into unified geographical information. (Kudos to Engadget for their prompt reportage.)

Möller et al. look at this research question, tagged “Local Visual Homing” in an intuitive manner inspired from social insects returning to their nest. The idea is that a robot, when somewhere important, takes a snapshot of the surrounding visual information. To return to that location later on (homing), it compares its current view of the world with the stored snapshot.

A technique called “image warping” is used to guide the robot to the snapshot location. Simply put, the robot imagines all possible movements it can do and simulates their effect on its current view of the world. It then selects the action that would bring its view closest to the stored snapshot. The outcome of this method is a homing vector that the robot should follow and a measure of how much its orientation has changed.

Using three different implementations of image warping, Möller et al. show how a robot equipped with a panoramic camera could effectively home with reasonable computational effort. Experiments were conducted on a database of real-world images taken by a robot (see example images below).

In the future, robots could use visual homing to go from snapshot to snapshot, thereby navigating through large environments.

Finally, don’t miss the author’s website for an extensive overview of visual navigation techniques.

Instead, Hornung et al. propose to use reinforcement learning to determine the optimal policy which allows the robots to go at speeds appropriate for navigation while ensuring that they get to destination as fast as possible. The actual implementation uses an augmented Markov decision process (MDP) to model the navigation task.

The learned policy is then compressed using a clustering technique to avoid being memory-sassy, which would be a major limitation for robots with low storage capacity.

Experiments were successfully conducted on two different robots in indoor and outdoor scenarios (see video) and the robots were faster than if they had navigated at constant speed. In the future, Hornung et al. hope to implement their system on fast moving robots, such as unmanned aerial vehicles!