Place recognition and localization with omnidirectional vision

Let’s say you just purchased a new service robot and you want it to be able to know its way in your apartment. The obvious thing to do would be to show it around, going from room to room saying “this is the living room” and “this is the kitchen”. The robot, equipped with an omnidirectional camera, could then take pictures along the way while recording its location. This will build-up its visual memory of the apartment. The challenge for the robot next time around is to figure out in what room it is (place recognition) and where it is in this room (localization) based on its current view of the world.

This requires finding a good way to compare new images to the robot’s visual memory. The comparison needs to be robust to robot motion, objects changing place and transformations required to use omnidirectional images. As a solution, Labbani-Igbida et al. propose to compute signatures for each omnidirectional image based on invariant Haar integrals. Signatures are numbers that capture distinctive features in the image (color, shape, texture, interest points…). By comparing signatures between images (similarity), the robot is able to determine in what room it is and at what location much faster than having to process the raw images.

Experiments were conducted using a Koala robot equipped with a paracatadioptric omnidirectional sensor. The robot was first placed in different rooms of an office environment where it took images to build a visual memory. The robot was then set loose to explore the office including places in the environment that had not been previously visited during the memory building phase.

Results show that the robot is able to do space recognition and localization in ways that outperform or perform similarly to state-of-the-art algorithms while being very time and memory efficient. In the future, authors would like to limit the number of images needed for the robot to build its visual memory.

Sabine Hauert is President of Robohub and Associate Professor at the Bristol Robotics Laboratory

Sabine Hauert is President of Robohub and Associate Professor at the Bristol Robotics Laboratory