By Ilia Baranov

In this new Robots 101 series, we will be taking a look at how robots work, what makes designing them challenging, and how engineers at Clearpath Robotics tackle these problems. To successfully operate in the real world, robots need to be able to see obstacles around them, and locate themselves. Humans do this mostly through sight, whereas robots can use any number of sensors. Today we will be looking at lasers, and how they contribute to robotic systems.

Overview

When you think of robots and lasers, the first image that comes to mind might come from science fiction; robots using laser weapons. However, almost all robots today use lasers for remote sensing. This means that the robot is able to tell, from a distance, some characteristics of an object. This can include size, reflective, color, etc. When the laser is used to measure distance in an arc around the robot, it is called LIDAR. LIDAR is a portmanteau of Light and Radar, essentially think of a sweeping radar beam shown in the films, using light.

Function and Concepts

By Mike1024, via Wikimedia Commons

All LIDAR units operate using this basic set of steps:

1. Laser light is emitted from the unit (usually infrared)
2. Laser light hits an object and is scattered
3. Some of the light makes it back to the emitter
4. The emitter measures the distance (more on how later)
5. Emitter turns, and process begins again

A great visual deception of that process to the right.

Types of LIDAR sensing

How exactly the laser sensor measures the distance to an object depends on how accurate the data needs to be. Three different methods commonly found on LIDAR sensors are:

Time of flight
The laser is emitted, and then received. The time difference is measured, and distance is simply (speed of light) x (time). This approach is very accurate, but also expensive due to the extremely high precision clocks needed on the sensor. Thus, it is usually only used on larger systems, and at longer ranges. Rare to use on robots

Phase difference
In this method, the emitted laser beam is modulated to a specific frequency. By comparing the phase shift of the returned signal on a few different frequencies, the distance is calculated. This is the most common way laser measurement is done. However, it tends to have limited range, and is less accurate than time of flight. Almost all of our robots use this.

Angle of incidence
The cheapest and easiest way to do laser range-finding is by angle of incidence. Essentially, by knowing what angle the reflected laser light hits the sensor, we can estimate the distance. However, this method tends to be of low quality. The Neato line of robotic vacuum cleaners use this technology.

Another thing to note is that so far, we have only discussed 2D LIDAR sensors. This means they are able to see a planar slice of the world around them, but not above or below. This has limitations, as the robot is then unable to see obstacles lower or higher than that 2d plane. To fix this issue, either multiple laser sensors can be used, or the laser sensor can be rotated or “nodded” up and down to take multiple scans. The first solution is what the Velodyne line of 3D LIDAR sensors employ, while the second solution tends to be a hack done by roboticists. The issue with nodding a LIDAR unit is a drastic reduction of refresh rate, from tens of Hz, down to 1 Hz or less.

Manufacturers and Uses

Some of the manufacturers we tend to use are SICK, Hokuyo, and Velodyne.
This is by no means an exhaustive list, just the ones we use most often.

SICK
Most of our LIDARs are made by SICK. Good combo of price, size, rugged build, and software support. Based in Germany

Hokuyo
Generally considered cheaper than SICK, in larger quantities. Some issues with fragility, and software support is not great. Based in Japan

Velodyne
Most expensive, used only when robot MUST see everything in the environment. Based in US.

Once the data is collected, it can be used for a variety of reasons. For robots, these tend to be:

• Mapping
  • Use the laser data to find out dimensions and locations of obstacles and rooms. This allows the robot to find its position (localisation) and also report dimensions of objects. The title image of this article shows the Clearpath Robotics Jackal mapping a room.
• Survey
  • Similar to mapping, however this tends to be outside. The robot collects long range data on geological formations, lakes, etc. This data can then be used to create accurate maps, or plan out mining operations.
• Obstacle Avoidance
  • Once an area is mapped, the robot can navigate autonomously around it. However, obstacles that were not mapped (for example, squishy, movable humans) need to be avoided.
• Safety Sensors
  • In cases where the robot is very fast or heavy, the sensors can be configured to automatically cut out motor power if the robot gets too close to people. Usually, this is a completely hardware-based feature.

Selection Criteria

A lot of criteria is used to select the best sensor for a given application. The saying goes that an engineer is someone who can do for $1 what any fool can do for $2. Selecting the right sensor for the job not only reduces cost, but also ensures that the robot has the most flexible and useful data collection system.

• Range
  • How far can laser sensor see? This impacts how fast a robot is able to move, and how well it is able to plan.
• Light Sensitivity
  • Can the laser work properly outdoors in full sunlight? Can it work with other lasers shining at it?
• Angular Resolution
  • What is the resolution of the sensor? More angular resolution means more chances of seeing small objects.
• Field of view
  • What is the field of view? A greater field of view provides more data.
• Refresh rate
  • How long does it take the sensor to return to the same point? The faster the sensor is, the faster the robot can safely move.
• Accuracy
  • How much noise is in the readings? How much does it change due to different materials?
• Size
  • Physical dimensions, but also mounting hardware, connectors, etc
• Cost
  • What fits the budget?
• Communication
  • USB? Ethernet? Proprietary communication?
• Power
  • What voltage and current is needed to make it work?
• Mechanical (strength, IP rating)
  • Where is the sensor going to work? Outdoors? In a machine shop?
• Safety
  • E-stops
  • regulation requirements
  • software vs. hardware safety

For example, here is a collected spec sheet of the SICK TIM551.

• ROS 101: Intro to the Robot Operating System
• ROS 101: A practical example
• Arduino for Makers #1: Setting up a development station
• Arduino for Makers #2: Deconstructing Arduino programs
• Robots Podcast: Soft Robotics Toolkit, with Donal Holland

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

IDS Imaging Development Systems – USB3 Vision industrial cameras

Choosing a camera can be confusing. It is important to determine your requirements, and then figure out how these can be achieved between the camera and the optics.

The first thing that most people talk about is resolution. The classic resolution of a camera is based on pixels (e.g.600 pixels x 400 pixels). The other type of resolution we must talk about is the spatial resolution. The spatial resolution is how close the pixels are to each other; or how many pixels-per-inch (ppi) is on the sensor. In practice it is the spatial resolution that really controls how an image will look. Sometimes the spatial resolution is measured in micrometers. When we use a lens on the camera, this effective resolution can change.

After determining the resolution requirements you should look at the focal length. The focal length you choose depends on what zoom level you need. A larger focal length (such as 200) will be zoomed in, a focal length of 30-50 is about what we see with our eyes, and a smaller focal length (such as 10) will be zoomed out and you will see a larger picture (lenses with small focal lengths are often called a wide-angle lenses). An extreme example of a small focal length is a fish-eye lens that can see around 180° (with a fairly distorted image). If the focal length is specified as a range, it is probably an adjustable zoom lens.

The next thing to look at is the Maximum Aperture or f number. The f number is often specified as f/2.8 or F2.8. The larger the number, the less light that can enter into the aperture. If you need to take images in a low light condition you will want a small f number to avoid needing external lighting of the scene. The smaller the f number, the shallower the depth of field will be. The depth of field is the part of the image that appears in focus (and sharp).

F-numbers. Each step is allows half the light as the preceding step.

Arguably this should have been mentioned first, but you will also need to look at the Field of View or FOV. The FOV is the angular measurement window that can be seen by the lens. The FOV is usually specified with two numbers, such as 60° X 90°. In this case 60° is the horizontal FOV and 90° is the vertical FOV. Sometimes instead of giving 2 numbers people will just specify 1 number based on the diagonal. FOV can be related to the focal length of the lens.

We can compute the FOV by knowing the size of the camera imaging array (ie CCD) from the datasheet and the focal length of the lens.

So for a 36 X 24mm sensor size and a focal length of 50mm, we can get the following camera FOV’s:

To calculate the image (or camera) resolution required based on a specific lens FOV to detect a certain size obstacle there is a simple equation.

So for example if your lens has a FOV of 60°x90°, the working distance from the sensor (ie where the camera hits the target) is 2meters away, and the smallest obstacle that you need to be able to detect (with a minimum of 2 pixels on it) is 0.01meters:

So we now know that when we choose a camera it must have a minimum resolution of 460 X 800 to meet the requirement of seeing a 0.01m item from 2m away.

We can also look at the working distance and how that affects other camera/lens parameters.

In many cases the focal length and the sensor size are fixed. So we can then look at how changing the viewing area or changing the working distance affects each other. On other cameras the focal length is adjustable and we can also look at how changing that affects the other parameters.

Note the vignetting at the corners of the image.

With all lenses, and particularly with cheaper (<$150) lenses, distortion and vignetting can be a big problem. Common distortion modes are where the X or Y axis appears to no longer be straight and perpendicular with each other (rectilinear). Often you will see an image look wavy or bulging. With vignetting the edges and particularly the corners become darkened. You can often reverse the effects of warping by using a homography for image rectification.

With any lens and camera system you need to check the mount types. There are many different styles for mounting a lens to a camera. Wikipedia has a long list of mounts that can be found here. The most common mount that I have seen for computer vision cameras (ie. no viewfinder and designed for machine vision applications) is the C mount. Choose a lens that you can lock into place. Not only should you be able to lock the lens into place you should be able to lock any rotating knobs into place (such as for zoom and focus). This is usually accomplished with set screws. I am not that familiar with them but I will note that there are many different filters that can be attached to your lens for things such as polarization and blocking certain parts of the spectrum (such as IR). Your iPhone camera typically has a built in IR filter in the lens to remove IR components from your images.

Another semi-obvious thing to look into is whether you need a black and white (often expressed as B&W, BW, or B/W) or color camera. In many cases B/W cameras will be easier for your algorithms to process. However having color can give you more channels for humans to look at and for algorithms to learn from. In addition to black and white, there are hyper spectral cameras that let you see various frequency bands. Some of the common spectral ranges that you might want are ultra-violet (UV), near-infrared (NIR), visible near-infrared (VisNIR), and infrared (IR). You will also want to pay attention to the Dynamic Range of the camera. The larger the dynamic range, the greater the light difference that can be handled within an image. If the robot is operating outdoors in sun and shade you might want to consider a high dynamic range (HDR) camera.

You will also want to consider the interface you have to the camera – both the electrical and software interface. On the electrical side, the common camera protocols are camera-link, USB2.0, USB3, firewire (IEEE1394), and gigabit ethernet (GigE). GigE cameras are nice since they just plug into an ethernet port making wiring easy. Many of them can use power-over-ethernet (PoE) so the power goes over the ethernet cable and you only have one cable to run. The downside of GigE is the latency and non deterministic nature of ethernet. The other interfaces might be better if latency is a problem. Also, you generally can only send full video over GigE with a wired connection, and not from a wireless connection. This is because you need to set the ethernet port to use jumbo packets, which can not be done on a standard wireless connection. Probably the most common reason I see GigE cameras not working properly is that the ethernet port on the computer is not configured for jumbo packets. For software, you want to make sure that your camera has an SDK that will be easy for you to use. If you are using ROS or OpenCV, you will want to verify that the drivers are good. You will also want to check what format the image is supplied back to you from the camera: will it be a raw image format? or will it be a file format such as png of jpg?

If you are doing stereo vision, you want to make sure the cameras have a triggering method, so that when you issue an image capture command you get images from both cameras simultaneously. The triggering can be in software or hardware, but hardware is typically better and faster. If you are having problems with your stereo algorithm this can be the source of the problem, especially if you are moving fast.

Finally, if you are looking for video, you also need to look out for the Frame Rate typically specified in Frames per Second (fps). This refers to how many images the camera can take in a second. We can then call that sequence of images a video.

Hope this helps. If you have camera experiences you would like to share – especially related to good/bad SDKs and what works/doesn’t-work with ROS – please share in the comments below.

While writing this post I saw Wikipedia said “MV is related to, though distinct from, computer vision.” This surprised me as I always viewed machine vision and computer vision to be similar fields, just different terms used in different industries. So I followed the references and found an interesting article from Bruce Batchelor here. After reading his article I am still not sure if I agree. I think there are people who are theorists that only care about the vision aspects, there are integrator’s who only care about the system and don’t know much about the vision aspects. However there is also a large segment in the middle, such as roboticists, that care about both sides of the problem. Working in robotics you need to care about the real world applications while also understanding the vision problems and develop new vision methods and techniques.

As roboticists are we doing both? are we only doing computer vision? only machine vision? What do you think? Leave your comments below.

For optics and some good technical notes you can visit Edmund Optics. They have a large supply of everything optics related.

Sources:

• http://www.gizmag.com/camera-lens-buying-guide/29141/
• http://vitaleartconservation.com/PDF/affects_of_lens_quality_on_image_resolution_v3a_draft.pdf
• http://www.gizmag.com/camera-lens-buying-guide/29141/
• http://digital.ni.com/public.nsf/allkb/1948AE3264ECF42E86257D00007305D5
• http://www.nikonusa.com/en/Learn-And-Explore/Article/g3cu6o2o/understanding-focal-length.html#!
• http://en.wikipedia.org/wiki/Angle_of_view

If you liked this article, you may also be interested in:

• Things to think about when transporting your robot
• Controlling a brushless DC motor with no encoders
• UMich team works on perception and localization using cameras
• Multi-viewpoint robotic camera system creates real ‘bullet time’ slow motion replays

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

New report by Research and Markets, Global Machine Vision Market 2012-2016, a $2,500 report, forcasts global machine vision market to grow at a CAGR of 9.4% to 2016.

Image processing has been playing a more and more important role turning “blind” machines into “seeing” ones. Even at high speeds and with stringent requirements for accuracy it allows 100% product inspection and is an ideal tool to strengthen competitiveness.

Increasing demand is seen outside of the industrial sector and/or for materials handling, inspection and QC.

Key vendors mentioned in the report include Basler AG, Cognex, Teledyne Technologies, ISRA Vision and Matrox Electronic Systems.

Researchers at Wakayama University are developing a system that manipulates the color, tone and contrast of printed materials, and could be used as an aid for users with vision impairments such as low vision or color blindness.

“In this research, we use a projector and camera to freely manipulate the tone and contrast of printed materials like this, by shining light onto them.”

“Right now, white light from the projector is hitting the material. In this device, there’s a camera, which photographs the scene. By controlling the projected light based on the scene, we can make the colors more vivid, for example, like this. The way the system works is, to put it very simply: Blue light is projected onto blue parts, and red light onto red parts, making the colors more vivid.”

“By changing the image processing, we can change color tones as well, like this. We can also make the picture monochrome, make the brightness uniform, or increase the contrast.”

“The point about this technology is, this kind of processing can be done dynamically. In other words, even if the printed material is moved, the system can track it and emphasize the colors.”

“When the light is projected, the actual color tone changes. So, the camera captures the altered color tone. The problem with that is, you can no longer tell how many real colors there are, or how many colors should be projected next. In this research, the system calculates the physical reflectivity of the captured image and the projected image. That estimation is a major feature of this technology.”

“One possible application for this is augmenting human vision. For example, if a person is colorblind, this system could adjust color tones so colors can be distinguished, even if there are combinations of confusing colors. Or a person with cataracts could be helped to see other people’s expressions, by emphasizing outlines.”

Also, by arranging the camera and projector into the same position optically, this image processing technique can be applied to 3D objects as well.

“We’ve added a texture adjustment algorithm, to make opaque objects look semi-transparent, or emphasize their luster, to make them look metallic.”

“We’re currently looking for businesses that might have applications for this technology.”

This camera system can track very fast moving objects, keeping them in the center of the screen at all times. Currently under development by the Ishikawa Oku Lab. at the University of Tokyo, this latest version captures Full HD video and can be used outdoors.

“Ordinarily, to change the direction a camera faces, you move the camera mechanically. But in this system, it’s not the camera that moves, it’s the mirrors. This makes it possible to change where you’re looking really quickly. In this demonstration, we’re tracking a table tennis ball. The ball moves extremely fast, but this system can keep compensating for the ball’s motion, so the ball stays in the middle of the image.”

This device consists of two mirrors for pan and tilt, and a group of lenses. The Saccade mirrors can be controlled at high speed, on the order of milliseconds. The mirrors move independently, so this system doesn’t lose its high-speed response even if it’s connected to a large, heavy camera.

Also, by connecting a projector instead of a recording device, images can be projected onto a fast-moving object. This could also be used in AR applications, showing interactive content on moving objects.

“Using a rotating mirror is a common method, but usually, the mirror is in front of the camera, so a very large mirror is needed. But a feature of this system is, it can even capture wide-angle images with a small mirror. That’s because the system contains special optics called a pupil shift system.”

“Another important point is, this system does very fast image processing to recognize the subject. It captures and processes an image every 1/1000th of a second. In this way, it can track the subject stably and continuously, simply by feeding back the subject’s position, without particularly predicting its behavior.”

“For example, this system can record, in great detail, the instant a player hits a home run, including how the bat bends and the ball reacts, and the ball’s subsequent path. Or in soccer, it can record things like penalty kicks in amazing detail. We think this will make it possible to shoot sports in a really compelling way.”

“Right now, we’re actually taking this outdoors to where sports are played, to check how accurately it works. We hope it will be usable for actual broadcasting in about two years.”

Technos has introduced the Super5000K 7K Model neuro-visual sensor. This is the world’s highest precision visual inspection system, and can detect spots 0.05mm in size at a distance of 2m, with a 1.4m field of vision. This is 1,000 times the resolving power of a conventional 4,000 pixel line sensor CCD camera, and 4,000 times that of a full HD camera.

“For industrial applications, this product has recently been used in many automotive-related situations, but it is also used in the steel, electronics, and flat-panel display industries. Typical manufacturing processes end with a visual inspection; we can fully automate that. Our sensor has high speed and high precision. It operates on the same principles as the human eye, but with 100 times the precision. It can perform automatic inspections with 1,000 times the precision of conventional CCDs. In principle, it emulates the movement of cells in the human eye; your eye vibrates up and down 80 times per second, and we emulate that vibration using electronic circuits.”

Conventional CCD-based equipment has had difficulty detecting color variations. By applying the principles of human vision, Technos has achieved a sensor with 100 times the precision of human eyes, making it possible to detect color variation.

“Color variation is a problem in a variety of settings; this sensor is used in industrial applications, but is also used in maintenance applications such as inspecting highways or oil storage tanks. This technology for rapidly picking up small details will be even more widely used in the future, and we plan to develop those applications.”

Pricing for the minimum configuration starts at an equivalent of $240,000, and runs up to between $470,000 and $730,000, depending on the particular specifications. Technos estimates that about 240 companies listed on the first section of the Tokyo Stock Exchange will need equipment with this level of precision. The company aims to sell about 10 units per year to the automobile, steel, semiconductor, and liquid crystal industries. Technos has been awarded patents in 14 countries around the world. With inquiries coming from foreign countries, it is looking to expand overseas as well.

Source: Michael Sapienza, Miles Hansard and Radu Horaud (2012) Real-time visuomotor update of an active binocular head, Autonomous Robots.