Underwater Autonomous Vehicles face challenging environments where GPS Navigation is rarely possible. John McConnell discusses his research, presented at ICRA 2022, into fusing overhead imagery with traditional SLAM algorithms. This research results in a more robust localization and mapping, with reduced drift commonly seen in SLAM algorithms.

Satellite imagery can be obtained for free or low cost through Google or Mapbox, creating an easily deployable framework for companies in industry to implement.

Links

• Download mp3 (12.2 MB)
• Sonar SLAM opensource repo
• Research Paper
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

In this episode, Audrow Nash interviews Erik Schluntz, co-founder and CTO of Cobalt Robotics, which makes a security guard robot. Erik speaks about how their robot handles elevators, how they have humans-in-the-loop to help their robot make decisions, robot body language, and gives advice for entrepreneurs.

Episode Links

• Cobalt’s website
• Erik’s website
• Video introducing the Cobalt robot
• Video of the Cobalt robot using an elevator

Podcast info

• Podcast website
• Apple Podcasts
• Spotify
• RSS
• Full episodes playlist
• Clips playlist

Texas A&M’s Robin Murphy has deployed robots at 29 disasters, including three building collapses, two mine disasters and an earthquake as director of the Center for Robot-Assisted Search and Rescue. She has also served as a technical search specialist with the Hillsboro County (Florida) Fire and Rescue Department. The Conversation talked to Murphy to provide readers an understanding of the types of technologies that search and rescue crews at the Champlain Towers South disaster site in Surfside, Florida, have at their disposal, as well as some they don’t. The interview has been edited for length.

What types of technologies are rescuers using at the Surfside condo collapse site?

We don’t have reports about it from Miami-Dade Fire Rescue Department, but news coverage shows that they’re using drones.

A standard kit for a technical search specialist would be basically a backpack of tools for searching the interior of the rubble: listening devices and a camera-on-a-wand or borescope for looking into the rubble.

How are drones typically used to help searchers?

They’re used to get a view from above to map the disaster and help plan the search, answering questions like: What does the site look like? Where is everybody? Oh crap, there’s smoke. Where is it coming from? Can we figure out what that part of the rubble looks like?

In Surfside, I wouldn’t be surprised if they were also flying up to look at those balconies that are still intact and the parts that are hanging over. A structural specialist with binoculars generally can’t see accurately above three stories. So they don’t have a lot of ability to determine if a building’s safe for people to be near, to be working around or in, by looking from the ground.

Drones can take a series of photos to generate orthomosaics. Orthomosaics are like those maps of Mars where they use software to glue all the individual photos together and it’s a complete map of the planet. You can imagine how useful an orthomosaic can be for dividing up an area for a search and seeing the progress of the search and rescue effort.

Search and rescue teams can use that same data for a digital elevation map. That’s software that gets the topology of the rubble and you can start actually measuring how high the pile is, how thick that slab is, that this piece of rubble must have come from this part of the building, and those sorts of things.

How might ground robots be used in this type of disaster?

The current state of the practice for searching the interior of rubble is to use either a small tracked vehicle, such as an Inkutun VGTV Extreme, which is the most commonly used robot for such situations, or a snakelike robot, such as the Active Scope Camera developed in Japan.

Teledyne FLIR is sending a couple of tracked robots and operators to the site in Surfside, Florida.

Ground robots are typically used to go into places that searchers can’t fit into and go farther than search cameras can. Search cams typically max out at 18 feet, whereas ground robots have been able to go over 60 feet into rubble. They are also used to go into unsafe voids that a rescuer could fit in but that would be unsafe and thus would require teams to work for hours to shore up before anyone could enter it safely.

In theory, ground robots could also be used to allow medical personnel to see and talk with survivors trapped in rubble, and carry small packages of water and medicine to them. But so far no search and rescue teams anywhere have found anyone alive with a ground robot.

What are the challenges for using ground robots inside rubble?

The big problem is seeing inside the rubble. You’ve got basically a concrete, sheetrock, piping and furniture version of pickup sticks. If you can get a robot into the rubble, then the structural engineers can see the interior of that pile of pickup sticks and say “Oh, OK, we’re not going pull on that, that’s going to cause a secondary collapse. OK, we should start on this side, we’ll get through the debris quicker and safer.”

Going inside rubble piles is really hard. Scale is important. If the void spaces are on the order of the size of the robot, it’s tricky. If something goes wrong, it can’t turn around; it has to drive backward. Tortuosity – how many turns per meter – is also important. The more turns, the harder it is.

There’s also different surfaces. The robot may be on a concrete floor, next thing it’s on a patch of somebody’s shag carpeting. Then it’s got to go through a bunch of concrete that’s been pulverized into sand. There’s dust kicking up. The surroundings may be wet from all the sewage and all the water from sprinkler systems and the sand and dust start acting like mud. So it gets really hard really fast in terms of mobility.

What is your current research focus?

We look at human-robot interaction. We discovered that of all of the robots we could find in use, including ours – and we were the leading group in deploying robots in disasters – 51% of the failures during a disaster deployment were due to human error.

It’s challenging to work in these environments. I’ve never been in a disaster where there wasn’t some sort of surprise related to perception, something that you didn’t realize you needed to look for until you’re there.

What is your ideal search and rescue robot?

I’d like someone to develop a robot ferret. Ferrets are kind of snakey-looking mammals. But they have legs, small little legs. They can scoot around like a snake. They can claw with their little feet and climb up on uneven rocks. They can do a full meerkat, meaning they can stretch up really high and look around. They’re really good at balance, so they don’t fall over. They can be looking up and all of a sudden the ground starts to shift and they’re down and they’re gone – they’re fast.

How do you see the field of search and rescue robots going forward?

There’s no real funding for these types of ground robots. So there’s no economic incentive to develop robots for building collapses, which are very rare, thank goodness.

And the public safety agencies can’t afford them. They typically cost US\$50,000 to \$150,000 versus as little as \$1,000 for an aerial drone. So the cost-benefit doesn’t seem to be there.

I’m very frustrated with this. We’re still about the same level we were 20 years ago at the World Trade Center.

Robin R. Murphy volunteers with the Center for Robot-Assisted Search and Rescue. She receives funding from the National Science Foundation for her work in disaster robotics and with CRASAR. She is affiliated with Texas A&M.

Original post published in The Conversation.

Abate interviews Tessa Lau on her startup Dusty Robotics which is innovating in the field of construction.

At Dusty Robotics, they developed a robot to automate the laying of floor plans on the floors in construction sites. Typically, this is done manually using a tape measure and reading printed out plans. This difficult task can often take a team of two a week to complete. Time-consuming tasks like this are incredibly expensive on a construction site where multiple different teams are waiting on this task to complete. Any errors in this process are even more time-consuming to fix. By using a robot to automatically convert 3d models of building plans into markings on the floors, the amount of time and errors are dramatically reduced.

Dr. Tessa Lau

Dr. Tessa Lau is an experienced entrepreneur with expertise in AI, machine learning, and robotics. She is currently Founder/CEO at Dusty Robotics, a construction robotics company building robot-powered tools for the modern construction workforce. Prior to Dusty, she was CTO/co-founder at Savioke, where she orchestrated the deployment of 75+ delivery robots into hotels and high-rises. Previously, Dr. Lau was a Research Scientist at Willow Garage, where she developed simple interfaces for personal robots. She also spent 11 years at IBM Research working in business process automation and knowledge capture. More generally, Dr. Lau is interested in technology that gives people super-powers, and building businesses that bring that technology into people’s lives. Dr. Lau was recognized as one of the Top 5 Innovative Women to Watch in Robotics by Inc. in 2018 and one of Fast Company’s Most Creative People in 2015. Dr. Lau holds a PhD in Computer Science from the University of Washington.

Links

• Download mp3 (20.9 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

In this episode, Lauren Klein interviews Gennaro Notimista, a robotics PhD student in the Georgia Robotics and InTelligent Systems Laboratory at Georgia Tech. Gennaro discusses the SlothBot, a solar-powered robot that slowly traverses wires, like its animal namesake, to monitor the environment.

Gennaro Notomista

Photo from news.gatech.edu

Gennaro Notomista is a robotics PhD student in the Georgia Robotics and InTelligent Systems Laboratory at Georgia Tech. Gennaro studies control frameworks, with the goal of making robots robust against a changing environment so they can handle long-duration deployments. Toward this goal, he explores constraints-driven control and approaches to coverage control, or enabling robots to traverse closed environments.  In addition to the SlothBot, Gennaro has applied his research to areas such as autonomous driving and swarm robotics.

Links

• Download mp3 (12.6 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Abate interviews David Mindell, co-founder of Humatics. David discusses a system they developed that can detect the location of a special tracking device down to a centimeter level accuracy. They are currently developing a device to detect location down to a millimeter level accuracy. This solves a the core problem of localization for robots. David discusses the technology behind these products and their applications.

David Mindell

David co-founded Humatics with a mission to revolutionize how people and machines locate, navigate and collaborate. He is a professor of Aeronautics and Astronautics at MIT, as well as the Dibner Professor of the History of Engineering and Manufacturing, and Chair of the MIT Task Force on the Work of the Future. He has participated in more than 25 oceanographic expeditions and is an inventor on 8 patents in autonomous helicopters, AI-assisted piloting, and RF navigation. He is the author of five books, including Our Robots, Ourselves:Robotics and the Myths of Autonomy (2015) and Digital Apollo: Human and Machine in Spaceflight (2008). David has undergraduate degrees from Yale and a Ph.D. from MIT.

Links

• Download mp3 (15.9 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

You can learn more about the urban pest control project here:

This type of prevention is designed to help technicians and companies have better efficiency and control and a faster response, when it comes to controlling rodent pests.

“The project uses a mobile autonomous robotic platform with a robot arm to introduce a camera into the trap. It captures an image that is uploaded to the cloud”.

“The project is in collaboration with Robotnik, which is responsible for the assembly of the robot; and Hispavista, which is in charge of the cloud part”, Marco Lorenzo adds.

Aritz Zabaleta, a Systems Technician at Hispavista Labs explains that the application consists of two components:

“The first manages the entire fleet of robots that communicate with the server in the cloud and it processes the information collected by them. The second component is the one that allows customers to access processed images”.

If you want to watch more fascinating robotics videos, you can explore ROBOTT-NET’s pilot projects on our YouTube-channel.

In this interview, Audrow Nash interviews Marco Hutter, Assistant Professor for Robotic Systems at ETH Zürich, about a quadrupedal robot designed for autonomous operation in challenging environments, called ANYmal. Hutter discusses ANYmal’s design, the ARGOS oil and gas rig inspection challenge, and the advantages and complexities of quadrupedal locomotion. 

Here is a video showing some of the highlights of ANYmal at the ARGOS Challenge.

Here is a video that shows some of the motions ANYmal is capable of.

Marco Hutter

Marco Hutter is assistant professor for Robotic Systems at ETH Zürich since 2015 and Branco Weiss Fellow since 2014. Before this, he was deputy director and group leader in the field of legged robotics at the Autonomous Systems Lab at ETH Zürich. After studying mechanical engineering, he conducted his doctoral degree in robotics at ETH with focus on design, actuation, and control of dynamic legged robotic systems. Beside his commitment within the National Centre of Competence in Research (NCCR) Digital Fabrication since October 2015 Hutter is part of the NCCR robotics and coordinator of several research projects, industrial collaborations, and international competitions (e.g. ARGOS challenge) that target the application of high-mobile autonomous vehicles in challenging environments such as for search and rescue, industrial inspection, or construction operation. His research interests lie in the development of novel machines and actuation concepts together with the underlying control, planning, and optimization algorithms for locomotion and manipulation.

Links

• Download mp3 (11.1 MB)
• Robotic Systems Lab
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

A new system can automatically retouch images in the style of a professional photographer. It can run on a cellphone and display retouched images in real-time. Courtesy of the researchers (edited by MIT News)

The data captured by today’s digital cameras is often treated as the raw material of a final image. Before uploading pictures to social networking sites, even casual cellphone photographers might spend a minute or two balancing color and tuning contrast, with one of the many popular image-processing programs now available.

This week at Siggraph, the premier digital graphics conference, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory and Google are presenting a new system that can automatically retouch images in the style of a professional photographer. It’s so energy-efficient, however, that it can run on a cellphone, and it’s so fast that it can display retouched images in real-time, so that the photographer can see the final version of the image while still framing the shot.

The same system can also speed up existing image-processing algorithms. In tests involving a new Google algorithm for producing high-dynamic-range images, which capture subtleties of color lost in standard digital images, the new system produced results that were visually indistinguishable from those of the algorithm in about one-tenth the time — again, fast enough for real-time display.

The system is a machine-learning system, meaning that it learns to perform tasks by analyzing training data; in this case, for each new task it learned, it was trained on thousands of pairs of images, raw and retouched.

The work builds on an earlier project from the MIT researchers, in which a cellphone would send a low-resolution version of an image to a web server. The server would send back a “transform recipe” that could be used to retouch the high-resolution version of the image on the phone, reducing bandwidth consumption.

“Google heard about the work I’d done on the transform recipe,” says Michaël Gharbi, an MIT graduate student in electrical engineering and computer science and first author on both papers. “They themselves did a follow-up on that, so we met and merged the two approaches. The idea was to do everything we were doing before but, instead of having to process everything on the cloud, to learn it. And the first goal of learning it was to speed it up.”

Short cuts

In the new work, the bulk of the image processing is performed on a low-resolution image, which drastically reduces time and energy consumption. But this introduces a new difficulty, because the color values of the individual pixels in the high-res image have to be inferred from the much coarser output of the machine-learning system.

In the past, researchers have attempted to use machine learning to learn how to “upsample” a low-res image, or increase its resolution by guessing the values of the omitted pixels. During training, the input to the system is a low-res image, and the output is a high-res image. But this doesn’t work well in practice; the low-res image just leaves out too much data.

Gharbi and his colleagues — MIT professor of electrical engineering and computer science Frédo Durand and Jiawen Chen, Jon Barron, and Sam Hasinoff of Google — address this problem with two clever tricks. The first is that the output of their machine-learning system is not an image; rather, it’s a set of simple formulae for modifying the colors of image pixels. During training, the performance of the system is judged according to how well the output formulae, when applied to the original image, approximate the retouched version.

Taking bearings

The second trick is a technique for determining how to apply those formulae to individual pixels in the high-res image. The output of the researchers’ system is a three-dimensional grid, 16 by 16 by 8. The 16-by-16 faces of the grid correspond to pixel locations in the source image; the eight layers stacked on top of them correspond to different pixel intensities. Each cell of the grid contains formulae that determine modifications of the color values of the source images.

That means that each cell of one of the grid’s 16-by-16 faces has to stand in for thousands of pixels in the high-res image. But suppose that each set of formulae corresponds to a single location at the center of its cell. Then any given high-res pixel falls within a square defined by four sets of formulae.

Roughly speaking, the modification of that pixel’s color value is a combination of the formulae at the square’s corners, weighted according to distance. A similar weighting occurs in the third dimension of the grid, the one corresponding to pixel intensity.

The researchers trained their system on a data set created by Durand’s group and Adobe Systems, the creators of Photoshop. The data set includes 5,000 images, each retouched by five different photographers. They also trained their system on thousands of pairs of images produced by the application of particular image-processing algorithms, such as the one for creating high-dynamic-range (HDR) images. The software for performing each modification takes up about as much space in memory as a single digital photo, so in principle, a cellphone could be equipped to process images in a range of styles.

Finally, the researchers compared their system’s performance to that of a machine-learning system that processed images at full resolution rather than low resolution. During processing, the full-res version needed about 12 gigabytes of memory to execute its operations; the researchers’ version needed about 100 megabytes, or one-hundredth as much. The full-resolution version of the HDR system took about 10 times as long to produce an image as the original algorithm, or 100 times as long as the researchers’ system.

“This technology has the potential to be very useful for real-time image enhancement on mobile platforms,” says Barron. “Using machine learning for computational photography is an exciting prospect but is limited by the severe computational and power constraints of mobile phones. This paper may provide us with a way to sidestep these issues and produce new, compelling, real-time photographic experiences without draining your battery or giving you a laggy viewfinder experience.”

Emily – short for Emergency Integrated Lifesaving Lanyard – is a remote-controlled rescue boat used by lifeguards to save people’s life at sea (Photo: Hydrolanix – EMILY robot)

This article was first published on the IEC e-tech website.

Rapid advances in technology are revolutionizing the roles of aerial, terrestrial and maritime robotic systems in disaster relief, search and rescue (SAR) and salvage operations. Robots and drones can be deployed quickly in areas deemed too unsafe for humans and are used to guide rescuers, collect data, deliver essential supplies or provide communication services.

Well-established use

The first reported use of SAR robots was to explore the wreckage beneath the collapsed twin towers of the World Trade Center in New York after the September 2001 terrorist attacks. Drones and robots have been used to survey damage after disasters such as the Fukushima Daiichi nuclear power plant accident in Japan in 2011 and the earthquakes in Haiti (2010) and Nepal (2015). Up to now, more than 50 deployments of disaster robots have been documented throughout the world, according to the Texas-based Center for Robot‑Assisted Search & Rescue (CRASAR).

Robin Murphy, head of CRASAR and author of the book Disaster Robotics, says:

The impact of earthquakes, hurricanes, flooding […] is increasing, so the need for robots for all phases of a disaster, from prevention to response and recovery, will increase as well.

Eyes in the sky

Drones, also known as unmanned aerial vehicles (UAVs), can be used to detect and enter damaged buildings, assisting rescue robots and responders on the ground by speeding up the search for survivors through prioritizing which areas to search first. The more quickly SAR teams respond, the higher the survival rate is likely to be. Rescue drones create real-time maps by taking aerial surveys and send back photos, videos and sensor data to support damage assessments.

Drones used for SAR and disaster relief are most commonly powered by rechargeable batteries and are operated autonomously through onboard computers or by remote control. Their equipment typically comprises radar and laser scanners, multiple sensors and video and optical cameras as well as infrared cameras that are used to identify heat signatures of human bodies and other objects. This helps rescuers to locate survivors at night and in large, open environments and to identify hot spots from fires. Listening devices can pick up hard-to-hear audio, while Wi-Fi antennas and other attachments detect signals given off by mobile phones and plot a map that outlines the locations of victims.

New technologies in use or development for rescue drones and robots include ways of increasing survivor detection. Sensors scan areas for heartbeats and breathing, multisensor probes respond to odours or sounds and chemical sensors signal the presence of gases.

Standards put safety first

Much of the technology used in drones comes from commodity electronics developed for consumer essentials like mobile phones. Drones also require global positioning satellite (GPS) units, wireless transmitters, signal processors and microelectromechanical systems (MEMS). The flight controller also collects data from barometric pressure and airspeed sensors.

IEC International Standards produced by a range of IEC Technical Committees (TCs) and Subcommittees (SCs) cover the components of drones such as batteries, MEMS and other sensors, with an emphasis on safety and interoperability.

IEC TC 47: Semiconductor devices, and its SC 47F: Micro electromechanical systems, are responsible for compiling a wide range of International Standards for the semiconductor devices used in sensors and the MEMS essential to the safe operation of drone flights. These include accelerometers, altimeters, magnetometers (compasses), gyroscopes and pressure sensors. IEC TC 56: Dependability, covers the reliability of electronic components and equipment.

IEC TC 2: Rotating machinery, prepares International Standards covering specifications for rotating electrical machines, while IEC TC 91: Electronics assembly technology, is responsible for standards on electronic assembly technologies including components.

IEC SC 21A: Secondary cells and batteries containing alkaline or other non-acid electrolytes, compiles International Standards for batteries used in mobile applications, as well as for large-capacity lithium cells and batteries.

Ideal for isolated and remote hard-to-access areas

Using drones is useful not only when natural disasters make access by air, land, sea or road difficult, but also in isolated regions that lack accessible infrastructure. Recently, drones have started delivering medical supplies in areas where finding emergency healthcare is extremely difficult. In 2014, Médecins Sans Frontières piloted the use of drones to deliver vaccines and medicine in Papua New Guinea. In 2016, the US robotics company Ziplinelaunched a drone delivery service, in partnership with the government of Rwanda, to supply blood and medical supplies throughout the mountainous East African country. Zipline says its battery-powered drones can fly 120 km on a single charge to deliver medicine speedily, without the need for refrigeration or insulation.

Rwanda launches world’s first drone service to deliver blood to patients in remote areas of the country (Photo: Zipline)

A project by a company in the Netherlands to help refugees who get into difficulty in the Mediterranean Sea offers another example of drones being used for humanitarian purposes. Its search and rescue (SAR) drone is intended to fly over long distances, detect boats and drop life jackets, life buoys, food and medicine if necessary.

Currently only about a quarter of the world’s countries regulate the use of drones. Their deployment in disaster relief operations poses challenges involving regulatory issues, particularly when decisions are made on an ad hoc basis by local and national authorities. Humanitarian relief agencies also warn of the risks of relief drones being mistaken for military aircraft.

Two arms good, four arms better

Japan and the US lead the world in the development of rescue and disaster relief robots. Teams from both countries collaborated in recovery efforts after an earthquake and tsunami hit Japan in March 2011, causing a meltdown at the Fukushima nuclear power plant. While a Japanese team operated an eight-meter long snake-like robot fitted with a camera, the US contribution included two remote-controlled robots. The first was a lightweight 22 kg model previously used for bomb disposal and other military tasks before being reconfigured for disaster relief operations. The larger US model, capable of lifting up to 90 kg, was adapted from a device originally used for firefighting and clearing rubble.

Endeavor Robotics 510 Packbots are deployed in emergency situations when direct human intervention is dangerous or impossible, such as after industrial or nuclear accidents. These robots were used at the Fukushima nuclear power plant (Photo: Endeavor Robotics)

In 2017, Japanese researchers unveiled a prototype drone-robot combination for use in disaster relief work. It consisted of a vision-guided robot equipped with sensitive measuring systems, including force sensors, and a drone tethered to the robot. Four fish‑eye cameras mounted on the drone capture video of overhead views, allowing the robot’s operator to assess damage in the surrounding area.

Another Japanese rescue robot unveiled in 2017 is a multi-limbed robot that is 1,7 m tall, with four arms capable of independent operation and four caterpillar treads for mobility. Called The Octopus, it is capable of lifting 200 kg with each arm, crossing uneven terrain and lifting itself over obstacles with two arms while the other two clear debris.

Octopus, developed by Japan’s Waseda University a robot designed to clear rubble in disaster areas including nuclear facilities (Photo: Waseda University)

In the US, researchers are exploring ways in which a small and light collapsible, origami‑inspired robot, first produced by the National Aeronautics and Space Administration (NASA), could be adapted for use as a rescue robot. The device, known as PUFFER (Pop‑Up Flat Folding Explorer Robot) is designed to pack nearly flat for transport, and then re‑expand on site to explore tight nooks and crannies which are inaccessible to larger robots.

Over and underwater too

Given that 80% of the world’s population lives near water, maritime robotic vehicles can also play an important role in disaster relief by inspecting critical underwater infrastructure, mapping damage and identifying sources of pollution to harbours and fishing areas. Maritime robots helped to reopen ports and shipping channels in both Japan and Haiti after major earthquakes in 2011 and 2010 respectively.

In the Mediterranean, a battery‑powered robotic device first developed for use by lifeguards to rescue swimmers has been adapted to help rescue refugees crossing the Aegean Sea from Turkey. This maritime robot has a maximum cruising speed of 35 km/h and can function as a flotation device for 4 people.

Easy to find components and technology
Rescue robots use components and technology found in most other robots used for commercial purposes. Actuators and other electric motors, accelerometers, gyroscopes and dozens of sensors and cameras providing 360⁰ views enable these robots to maintain balance while moving over uneven ground covered with rubble or debris, and to get a sense of the environment around them.

A robot operating in a hazardous environment needs independent power and sensors for specific environments. It may be cut off from its human operator when communication signals are patchy. When remote operation guided by sensor data becomes impossible, a rescue robot needs the ability to make decisions on its own, using machine learning or other artificial intelligence (AI) algorithms.

Several IEC TCs and SCs cooperate on the development of International Standards for the broad range of electrotechnical systems, equipment and applications used in rescue robots. In addition to IEC TC 47: Semiconductor devices and IEC SC 47F: Microelectromechanical systems, mentioned above, other IEC TCs involved in standardization work for specific areas affecting rescue and disaster relief robots include IEC TC 44: Safety of machinery – Electrotechnical aspects; IEC TC 2: Rotating machinery; IEC TC 17: Switchgear and controlgear; and IEC TC 22: Power electronic systems and equipment.

Where humans fear to tread
The number of disasters recorded globally has doubled since the 1980s, with damage and losses estimated at an average of USD 100 billion a year since the start of the new millennium, according to the Overseas Development Institute (ODI), a UK think tank. This trend is likely to lead to increased demand for unmanned robotic devices that can assist disaster relief operations on land, in the air and at sea.

Robots of all kinds play a growing role in supporting SAR teams. Increasing autonomy will create more capable ground robots, while a combination of rapid technological advances and regulation should see the market for disaster relief drones soar over the next five years.

MarketsandMarkets, a research company, estimated in October 2016 that the total global market for drones, comprising commercial and military sales, would grow at a compound annual growth rate (CAGR) of nearly 20% between 2016 and 2022 to exceed USD 21 billion. Drones designed for humanitarian and disaster relief operations will account for 10% of the future drone market, according to the US‑based Association of Unmanned Vehicle Systems International (AUVSI).

Last August, Singapore became the first country to permit autonomous taxis on its roads. Boston-based startup nuTonomy moved its operations to the Far East, which enabled the company to launch public trials weeks before Uber’s test in Pittsburgh. The secret to the company’s speed to market was to skip America’s 19,492 municipal government licensing departments to pilot its technology years before any of the other competing technologies, with the exception of Google’s Waymo. In addition to less regulatory hoops, the Singapore Land Transport Authority has partnered with nuTonomy on its rollout.

Pang Kin Keong, Singapore’s Permanent Secretary for Transport and the Chairman of its Committee on autonomous driving, said “We face constraints in land and manpower. We want to take advantage of self-driving technology to overcome such constraints, and in particular to introduce new mobility concepts which could bring about transformational improvements to public transport in Singapore.”

The company is on track to offer its driverless taxis throughout the country by 2018. Doug Parker’s, nuTonomy’s COO, estimates that autonomous taxis could ultimately reduce the number of cars on Singapore’s roads from 900,000 to 300,000. Park explains, “When you are able to take that many cars off the road, it creates a lot of possibilities. You can create smaller roads, you can create much smaller car parks. I think it will change how people interact with the city going forward.” Park’s partnership with the city-state is made possible because Singapore is not straddled with the costs of aging infrastructure like many US and European cities.

Since first announcing its test in 2016, nuTonomy is on pace to expand globally with its recent partnership with ride-sharing company Lyft in a pilot in Boston. Karl Iagnemma, nuTonomy’s CEO, said: “By combining forces with Lyft in the U.S., we’ll be positioned to build the best passenger experience for self-driving cars. Both companies care immensely about solving urban transportation issues and the future of our cities, and we look forward to working with Lyft as we continue to improve our autonomous vehicle software system.”

Besides autonomous vehicles, drones have been widely embraced by Singapore’s infamously strict police department. Singaporean startup H3 Dynamics became the first company last year to launch a drone in the box solution that offers storage and charging stations in the field. H3’s “DRONEBOX” is a unique solar-based charging station that enables longer autonomous missions in areas that are typically hostile for humans. Since showcasing its technology above the streets of Singapore, H3 faces increased competition from formidable upstarts, including: Airobotics, EasyAriel, and HiveUAV.

According to its original press release, “DRONEBOX is an all-inclusive, self-powered system that can be deployed anywhere, including in remote areas where industrial assets, borders, or sensitive installations require constant monitoring. Designed as an evolution over today’s many unattended sensors and CCTV cameras installed in cities, borders, or large industrial estates, DRONEBOX innovates by giving sensors freedom of movement using drones as their vehicles. End-users can now deploy flying sensor systems at different locations, and measure just about anything, anywhere, anytime. They offer 24/7 reactivity, providing critical information to operators – even to those located thousands of miles away.”

In June, Dynamic H3 announced the opening of new drone operation centers to include Europe, America and the Middle East. Additionaly, H3 is now marketing its next generation of battery technology for extended high-value asset missions. H3’s HES Energy Systems is the product a decade-long research initiative between the company and the Singaporean government. Unlike typical drone lithium batteries that have a flight-time of 20-40 minutes, HES Energy’s developed its ground-breaking 6 hour battery (above) with a first of its kind “solid-hydrogen on demand powered system.” The combination of longer flights, self-charging stations and autonomous missions is a powerful value proposition for this Singaporean offering in differentiating itself in an already crowded unmanned flight market.

This past week, Dubai announced its plans to roll out a fleet of mini autonomous police cars for surveillance and crime prevention. This effort is part of the Middle Eastern city’s program to automate 25% of its police force over the next decade. The Gulf State’s ambitious plans were a perfect fit for Singaporean OTSAW Digital, a division of ActiV-a global tech powerhouse. Similar to nuTonomy and H3, OTSAW’s O-R3 grew out of the innovation friendly environment of the Asian republic.

OR-3 is smaller than a golf cart, and not meant actually to capture nefarious actors, but to identify suspicious activity while it is happening. Using facial recognition technology and a built-in ariel drone, the robot will begin patrolling the Dubai beat by the end of the year. The autonomous car/drone combo is almost a hybrid of nuTonomy and H3, with an array of sensors and machine intelligence technologies to survey the area via thermal imaging, license plate readers and cloud-based computing.

According to Abdullah Khalifa Al Marri, the head of the Dubai police department, the OR-3 isn’t meant to replace officers but rather to augment their skills with more efficient resources. “We seek to augment operations with the help of technology such as robots. Essentially, we aim for streets to be safe and peaceful without heavy police patrol,” said Al Marri. Last month, Dubai even deployed a humanoid-looking robot to monitor tourist attractions, dubbed Robocop, that speaks English and Arabic. According to Dubai, it plans to deploy larger humanoids that stand over 10 feet tall and go over 50 mph, while the human controller operates the device inside the robots.

Brigadier-General Khalid Nasser Al Razzouqi, Director-General of Smart Services at Dubai Police’s department, boasted, “The launch of the world’s first operational Robocop is a significant milestone for the Emirate and a step towards realizing Dubai’s vision to be a global leader in smart cities technology adoption.” In 2015, The World Economic Forum ranked the United Arab Emirates as the second most tech savvy government in the world, just behind Singapore.

As I write this article I find myself at another conference, “The State of New York: Smart Cities.” Hoping for insights about how my city will compete with the likes of these tech savvy counterparts, I was met by a group of bureaucrats touting App Competitions and Free WiFi. One speaker even suggested that the Metropolitan Transportation Authority (MTA) is run by the best executive team, even though New York’s Governor has politely called the organization dysfunctional due to multiple train derailments, signal problems, and overcrowded stations (see: The Summer of Hell). Science is not just about the possible, but the willing. Singapore’s ability to reinvent the very nature of how a city operates and partners with the private sector is proof positive that even a country founded 50 years ago can climb to the “top of the heap.”

The need for fast, accurate 3D mapping solutions has quickly become a reality for many industries wanting to adopt new technologies in AI and automation. New applications requiring these 3D mapping platforms include surveillance, mining, automated measurement & inspection, construction management & decommissioning, and photo-realistic rendering. Here at Clearpath Robotics, we decided to team up with Mandala Robotics to show how easily you can implement 3D mapping on a Clearpath robot.

3D Mapping Overview

3D mapping on a mobile robot requires Simultaneous Localization and Mapping (SLAM), for which there are many different solutions available. Localization can be achieved by fusing many different types of pose estimates. Pose estimation can be done using combinations of GPS measurements, wheel encoders, inertial measurement units, 2D or 3D scan registration, optical flow, visual feature tracking and others techniques. Mapping can be done simultaneously using the lidars and cameras that are used for scan registration and for visual position tracking, respectively. This allows a mobile robot to track its position while creating a map of the environment. Choosing which SLAM solution to use is highly dependent on the application and the environment to be mapped. Although many 3D SLAM software packages exist and cannot all be discussed here, there are few 3D mapping hardware platforms that offer full end-to-end 3D reconstruction on a mobile platform.

Existing 3D Mapping Platforms

We will briefly highlight some more popular alternatives of commercialized 3D mapping platforms that have one or many lidars, and in some cases optical cameras, for point cloud data collection. It is important to note that there are two ways to collect a 3D point cloud using lidars:

1. Use a 3D lidar which consists of one device with multiple stacked horizontally laser beams
2. Tilt or rotate a 2D lidar to get 3D coverage

Tilting of a 2D lidar typically refers to back-and-forth rotating of the lidar about its horizontal plane, while rotating usually refers to continuous 360 degree rotation of a vertically or horizontally mounted lidar.

Example 3D Mapping Platforms: 1. MultiSense SL (Left) by Carnegie Robotics, 2. 3DLS-K (Middle) by Fraunhofer IAIS Institute, 3. Cartographer Backpack (Right) by Google.

1. MultiSense SL

The MultiSense SL was developed by Carnegie Robotics and provides a compact and lightweight 3D data collection unit for researchers. The unit has a tilting Hokuyo 2D lidar, a stereo camera, LED lights, and is pre-calibrated for the user. This allows for the generation of coloured point clouds. This platform comes with a full software development kit (SDK), open source ROS software, and is the sensor of choice for the DARPA Robotics Challenge for humanoid robots.

2. 3DLS-K

The 3DLS-K is a dual-tilting unit made by Fraunhofer IAIS Institute with the option of using SICK LMS-200 or LMS-291 lidars. Fraunhofer IAIS also offers other configurations with continuously rotating 2D SICK or Hokuyo lidars. These systems allow for the collection of non-coloured point clouds. With the purchase of these units, a full application program interface (API) is available for configuring the system and collecting data.

3. Cartographer Backpack

The Cartographer Backpack is a mapping unit with two static Hokuyo lidars (one horizontal and one vertical) and an on-board computer. Google released cartographer software as an open source library for performing 3D mapping with multiple possible sensor configurations. The Cartographer Backpack is an example of a possible configuration to map with this software. Cartographer allows for integration of multiple 2D lidars, 3D lidars, IMU and cameras, and is also fully supported in ROS. Datasets are also publicly available for those who want to see mapping results in ROS.

Mandala Mapping – System Overview

Thanks to the team at Mandala Robotics, we got our hands on one of their 3D mapping units to try some mapping on our own. This unit consists of a mount for a rotating vertical lidar, a fixed horizontal lidar, as well as an onboard computer with an Nvidia GeForce GTX 1050 Ti GPU. The horizontal lidar allows for the implementation of 2D scan registration as well as 2D mapping and obstacle avoidance. The vertical rotating lidar is used for acquiring the 3D point cloud data. In our implementation, real-time SLAM was performed solely using 3D scan registration (more on this later) specifically programmed for full utilization of the onboard GPU. The software used to implement this mapping can be found on the mandala-mapping github repository.

Scan registration is the process of combining (or stitching) together two subsequent point clouds (either in 2D or 3D) to estimate the change in pose between the scans. This results in motion estimates to be used in SLAM and also allows a new point cloud to be added to an existing in order to build a map. This process is achieved by running iterative closest point (ICP) between the two subsequent scans. ICP performs a closest neighbour search to match all points from the reference scan to a point on the new scan. Subsequently, optimization is performed to find rotation and translation matrices that minimise the distance between the closest neighbours. By iterating this process, the result converges to the true rotation and translation that the robot underwent between the two scans. This is the process that was used for 3D mapping in the following demo.

Mandala Robotics has also released additional examples of GPU computing tasks useful for robotics and SLAM. These examples can be found here.

Mandala Mapping Results

The following video shows some of our results from mapping areas within the Clearpath office, lab and parking lot. The datasets collected for this video can be downloaded here.

The Mandala Mapping software was very easy to get up and running for someone with basic knowledge in ROS. There is one launch file which runs the Husky base software as well as the 3D mapping. Initiating each scan can be done by sending a simple scan request message to the mapping node, or by pressing one button on the joystick used to drive the Husky. Furthermore, with a little more ROS knowledge, it is easy to incorporate autonomy into the 3D mapping. Our forked repository shows how a short C++ script can be written to enable constant scan intervals while navigating in a straight line. Alternatively, one could easily incorporate 2D SLAM such as gmapping together with the move_base package in order to give specific scanning goals within a map.

Why use Mandala Mapping on your robot?

If you are looking for a quick and easy way to collect 3D point clouds, with the versatility to use multiple lidar types, then this system is a great choice. The hardware work involved with setting up the unit is minimal and well documented, and it is preconfigured to work with your Clearpath Husky. Therefore, you can be up and running with ROS in a few days! The mapping is done in real time, with only a little lag time as your point cloud size grows, and it allows you to visualize your map as you drive.

The downside to this system, compared to the MultiSense SL for example, is that you cannot yet get a coloured point cloud since no cameras have been integrated into this system. However, Mandala Robotics is currently in the beta testing stage for a similar system with an additional 360 degree camera. This system uses the Ladybug5 and will allow RGB colour to be mapped to each of the point cloud elements. Keep an eye out for a future Clearpath blogs in case we get our hands on one of these systems! All things considered, the Mandala Mapping kit offers a great alternative to the other units aforementioned and fills many of the gaps in functionality of these systems.

The post Rapid Outdoor/Indoor 3D Mapping with a Husky UGV appeared first on Clearpath Robotics.

Photo credit: Pierre-Yves Guernier

Technology-driven automation plays a critical role in the global economy, and its visibility in our lives is growing. As technology impacts more and more jobs, individuals and enterprises find themselves wondering what effect the current wave of automation will have on their future economic prospects.

We develop the fastest, smallest and lightest distance sensors for advanced robotics in challenging environments. These sensors are born from a fruitful collaboration with CERN while developing flying indoor inspection systems.

How we began started with a challenge: the European Centre for Nuclear Research (CERN) asked if we could use drones to perform fully autonomous inspections within the tunnel of the Large Hadron Collider. Now if you haven’t seen it, it’s a complex environment; perhaps one of the most unfriendly environments imaginable for fully autonomous drone flight. But we accepted the mission, rolled-up our sleeves, and got to work. As you can imagine, the mission was very challenging! 

Large Hadron Collider tunnel. Credit: CERN

One of the main issues we faced was finding suitable sensors to place on the drone for navigation and anti-collision. We got everything on the market that we could find and tried to make it work. Ultrasound was too slow and the range too short. Lasers tend to be too big, too heavy and consumed too much power. Monocular and stereo vision was highly complex and placed a huge computational burden on the system and even then was prone to failure. It became clear that what we really needed, simply didn’t exist! That’s how the concept for TeraRanger’s brand of sensors was born.

Having failed to make any of the available sensing technologies work at the performance levels required, we came to the conclusion that we would need to build the sensors from the ground up. It wasn’t easy (and still isn’t) but finally, we had something small enough, light enough (8g), with fast refresh rates and enough range to work well on the drone. Leading academics in robotics could see potential using the sensor and wanted some for themselves, then more people wanted them, and before too long we had a new business.

Millimetre precision wasn’t vital for the drone application, but the high refresh rates and range were. And by not using a laser emitter we were able to give the sensor a 3 degree field of view, which for many applications proved to be a real boon, giving a smoother flow of data when faced with uneven surfaces and complex and cluttered environments. It also enabled the sensor to be fully eye-safe and the supply current to remain low.

Plug and play multi-axis sensing

Knowing that we would often need to use multiple sensors at the same time, we also designed-in support for multi-sensor, multi-axis requirements. Using a ‘hub’ we can simultaneously connect up to eight sensors to provide a simple to use, plug and play approach to multi-sensor applications. By controlling the sequence in which sensors are fired (along with some other parameters) we are able to limit or eliminate, the potential for sensor cross-talk and then stream an array of calibrated distance values in millimetres, and at high speed. From a user’s’ perspective this is about as simple as it gets since the hub also centralises power management.

TeraRanger Tower

There’s no need to get in a spin

Using that same concept we continued to push the boundaries. A significant evolution has been our approach to LiDAR scanning – not just from a hardware point of view (although that is also different) but from a conceptual approach too. We’ve taken the same philosophy of small size, lightweight sensors with very high refresh rates (up to 1kHz) and applied that to create a new style of static LiDAR. Rather than rotating a sensor or using other mechanical methods to move a beam, TeraRanger Tower simultaneously monitors eight axis (or more if you stack multiple units together) and streams an array of data at up to 270Hz!

Challenging the point-cloud conundrum

With no motors or other moving parts, the hardware itself has many advantages, being silent, lightweight and robust, but there is also a secondary benefit from the data. Traditional thinking amongst the robotics community is that to perform navigation, Simultaneous Localisation and Mapping (SLAM) and collision avoidance you have to “see” everything around you. Just as we did at the start of our journey, people focus on complex solutions – like stereo vision – gathering millions of data points which then requires complex and resource-hungry processing. The complexity of the solution – and of the algorithms – has the potential to create many failure modes. Having discovered for ourselves that the complicated solution is not always necessary, our approach is different in that, we monitor fewer points. But,  we monitor them at very fast refresh rates to ensure that what we think we see, is really there. As a result, we build less intense point clouds, but with very reliable data. This then requires less complex algorithms and processing and can be done with lighter-weight computing. The result is a more robust, and potentially safer solution, especially when you can make some assumptions about your environment, or harness odometry data to augment the LiDAR data. Many times we were told you could never do SLAM monitoring on just eight points, but we proved that wrong.

Coming full circle: There are no big problems, just a lot of little problems

All of this leads back to our original mission. We’ve not solved it yet, but recently we mounted TeraRanger Tower to a drone and proved, for the first time we believe, that a static LiDAR can be used for drone anti-collision. The Proof of Concept was quickly put together to harness code developed for the open source APM 3.5 flight controller, with Terabee writing drivers to hook into the codebase. Anti-collision is just one step in the journey to fully autonomous drone flight and we are still on the wild-ride of technology, but definitely, we are taming the beast!

If you have expertise in drone collision avoidance and wish to help us overcome the remaining challenges, please contact us at teraranger@terabee.com. For more information about Terabee and our TeraRanger brand of sensors, please visit our website.

HARRIS

The Xponential 2017 national conference was held May 8-11 by the Association for Unmanned Vehicle Systems International (AUVSI) in the Kay Bailey Hutchison Convention Center in Dallas, Texas. The event took place in the largest exhibit hall ever dedicated to unmanned systems and robotics, with over 370,000 square feet. It featured over 650 robotics organizations – companies, research institutions, universities, consultants, nonprofits and more – from the U.S. and countries worldwide.

Here’s a sample of images from the world’s largest tradeshow for unmanned systems. All images by Lucien Miller, CEO of innov8tivedesigns.com.

AERYON Labs

AIRBUS DEFENCE

FIRSTEC CO

Infrared Cameras Inc

PRODRONE

QinetiQ North America

Quasonix

Robotis

STAMPEDE

TEI

TELEDYNE TECHNOLOGIES

Tianjin Aurora UAV Technology

Click here to view the full gallery.

I was recently asked about the differences between RADAR and LIDAR. I gave the generic answer about LIDAR having higher resolution and accuracy than RADAR. And RADAR having a longer range and performing better in dust and smokey conditions. When prompted for why RADAR is less accurate and lower resolution, I sort of mumbled through a response about the wavelength. However, I did not have a good response, so this post will be my better response.

LIDAR

LIDAR—which is short for Light Detection and Ranging—uses a laser that is emitted and then received back in the sensor. In most of the LIDAR sensors used for mapping (and self driving vehicles) the time between the emission and reception is computed to determine the time of flight (ToF). Knowing the speed of light and (1/2 of the) time for the wave to return (since the signal traveled out and back) we can compute how far away the object was that caused the light to bounce back. That value is the range information that is reported by the sensor. LIDAR’s generally use light in the near-infrared, visible (but not really visible), and UV spectrum’s.

There are some sensors that use triangulation to compute the position (instead of ToF). These are usually high accuracy, high resolution sensors. These sensors are great for verifying components on an assembly lines or inspecting thermal tile damage on the space shuttle. However that is not the focus of this post.

LIDAR data. The top shows the reflectivity data. The bottom shows the range data with brighter points being farther away.

The laser beam can also be focused to have a small spot size that does not expand much. This small spot size can help give a high resolution. If you have a spinning mirror (which is often the case) then you can shoot the laser every degree or so (based on the accuracy of the pointing mechanism) for improved resolution. It is not uncommon to find LIDAR’s operating at 0.25 degrees of angular resolution.

RADAR

RADAR—which is short for Radio Detection and Ranging—uses radio waves to compute velocity, and/or range to an object. Radio waves have less absorption (so less attenuation) than the light waves when contacting objects, so they can work over a longer distance. As you can see in the image below the RF waves have a larger wavelength than the LIDAR waves. The down side is that if an object is much smaller than the RF wave being used, the object might not reflect back enough energy to be detected. For that reason many RADAR’s in use for obstacle detection will be “high frequency” so that the wavelength is shorter (hence why we often use mm-wave in robotics) and can detect smaller objects. However, since LIDAR’s have the significantly smaller wavelength, they will still usually have a finer resolution.

Electromagnetic spectrum showing radio waves all the way on the left for RADAR and near-infrared/visible/ultra-violet waves towards the right for LIDAR usage.

Most of the RADAR’s that I have seen will have a narrow field of view (10’s of degrees) and then just return a single value (or they might have up to a few dozen channels, see multi-mode RADAR below) for the range of the detected object. There are tricks that some systems can do using multiple channels to also get the angle for the range measurement. The angle will not be as high resolution as most LIDAR’s. There are also some RADAR’s on the market that scan to get multiple measurements. The 2 approaches are a spinning antenna (such as what you see at airports or on ships) or electronically “spinning” which is a device using multiple internal antennas with no moving parts. More advanced (newer) RADAR’s can do things like track multiple objects. In many cases they will not actually return the points that outline the objects (like a LIDAR), but will return a range, bearing, and velocity to an estimated centroid of the detected item. If multiple objects are near each other, the sensor might confuse them as being one large object and return one centroid range [Here is the source and a good reference to read].

Using the Doppler frequency shift, the velocity of an object can also be easily determined with relatively little computational effort. If the RADAR sensor and the detected object are both moving than you get a relative velocity between the two objects.

With RADAR there are commonly two operating modes:

1. Time of Flight – This operates similar to the LIDAR sensor above, however it uses radio wave pulses for the Time of Flight calculations. Since the sensors are pulsed, it knows when the pulse was sent. So, computing the range can be easier than with the continuous wave sensors (described below). The resolution of the sensor can be adjusted by changing the pulse width and the length of time you listen for a response (a ping back). These sensors often have fixed antennas leading to a small operating field of views (compared to LIDAR).

There are some systems that will combine multiple ToF radio waves into one package with different pulse widths. These will allow for various ranges to be detected with higher accuracy. These are sometimes called multi-mode RADAR

2. Continuous Wave – This approach frequency modulates (FMCW) a wave and then compares the frequency of the reflected signal to the transmitted signal in order to determine a frequency shift. That frequency shift can be used to determine the range to the object that reflected it. The larger the shift the farther the object is from the sensor (within some bounds). Computing the frequency shift and the corresponding range is computationally easier than ToF, plus the electronics to do so are easier and cheaper. This makes continuous frequency modulated systems popular. Also, since separate transmit and receive antennas are often used, this approach can continuously be transmitting and receiving at the same time unlike the pulsed ToF approach, which needs to transmit then wait for a response. This feature and the simple electronics can make FMCW RADAR very fast at detecting objects.

There is another version of the Continuous Wave RADAR where the wave is not modulated. These systems are cheap and good for quickly detecting speed using the Doppler effect, however they can not determine range. They are often used by police to detect vehicle speed where the range is not important.

SONAR

Unrelated, I should note that SONAR or Sound Navigation and Ranging, can work in both of the modes as RADAR. The wavelength used is even larger than RADAR. It is farther to the left in the spectrum image shown before, and is off the chart.

I should point out that there are cool imaging SONAR sensors. The general idea is that you can make the sensing wave vertical so that the horizontal resolution is very fine (<1 degree) and the vertical resolution is larger (10+ degrees). You can then put many of these beams near each other in a sensor package. There are similar packages that do this with small wave length RADAR.

Cost

LIDAR Sensors tend to cost more than RADAR sensors for several reasons:

1. LIDAR using ToF needs high speed electronics that cost more.
2. LIDAR sensors needs CCD receivers, optics, motors, and lasers to generate and receive the waves used. RADAR only needs some stationary antennas.
3. LIDAR sensors have spinning parts for scanning. That requires motors and encoders. RADAR only needs some stationary antennas. (I know this is sort of similar to the line above).

Computationally Speaking

RADAR sensors tend to generate a lot less data as they just return a single point, or a few dozen points. When the sensor is multi-channel it is often just returning a range/velocity to a few centroid(ish) objects. The LIDAR sensors are sending lots of data about each individual laser point of range data. It is then up to the user to make that data useful. With the RADAR you have a simple number, but that number might not be the best. With LIDAR it depends on the roboticist to generate algorithms to detect the various objects and discern what is being viewed by the sensor.

If your goal is to detect a car in front of you (or driving towards you) and get its velocity, the RADAR can be great. If you are trying to determine the precise location of an item, generate a surface map, or find a small fence-post, a LIDAR might do better. Just remember that if you are in dust or rain the LIDAR might return a cloud of points right near the sensor (as it reads all of those particles/drops); while the RADAR might do much better.

Applications of JANUS are limitless. In the operational context, CMRE has already successfully tested the delivery of Automatic Identification System AIS data, which consists of identification, position, course and speed of ships at sea and Meteorological and Oceanographic (METOC) information to underwater assets using JANUS-standardized acoustic information. Credit: NATO

NATO Nations have agreed to use JANUS, a new underwater communications device, as the NATO Standard for digital underwater communications. This is the first time a digital underwater communication protocol has been acknowledged at an international level. Implementing JANUS paves the way for a standardised ‘Internet of Underwater Things’ with many exciting future applications.

Aerial and terrestrial communication methods do not work well underwater. In fact, none of the available communication protocols can be easily exported under the sea surface to work with and enhance capabilities of new robotics technologies. To address this limitation, the NATO STO Centre for Maritime Research and Experimentation (CMRE) developed a digital underwater coding standard aimed at providing a baseline common denominator for underwater acoustic communications called JANUS.

JANUS, named after the Roman God of openings and gateways, is now recognised as a NATO standard, called an STANAG, for Standardisation Agreement by all the NATO Nations. The protocol is a way of encoding information into sound simple enough that existing and new equipment can be brought into compliance with the standard relatively easily. CMRE is working to support effective underwater communication networks to allow undersea robots to work together and report back home (see the infographic below).

In practice, it works like a Lingua Franca, and it can make military and civilian, NATO and non-NATO, devices fully interoperable. As it happened with Wi-Fi protocol we use today, JANUS has the potential to leverage technological investments under the water making all compliant assets (and networks of those) able to cooperate with a robust signalling method.

Credit: NATO

Adopted globally, JANUS aims at stopping the current underwater “Tower of Babel” between modems and systems of different manufacturers and nations. For that reason the JANUS developing process has been as inclusive as possible from the start, involving academia, industry and governmental agencies. Testing at sea have been conducted in the last several years by CMRE onboard the NATO Research Vessel Alliance, and using the innovative CMRE Littoral Ocean Observatory Network (LOON), which facilitates experimentation of marine robots’ mission-base teams by creating a monitoring acoustic network with tripods of underwater communications equipment sitting on the seabed but accessible by users across the world via web.

About CMRE. The NATO STO CMRE (Science and Technology Organization – Centre for Maritime Research and Experimentation) is located in La Spezia, Italy. The Centre focuses on research, innovation and technology in areas such as defence of maritime forces and installations against terrorism and piracy, secure networks, development of the common operational picture, the maritime component of expeditionary operations, mine countermeasure systems, non-lethal protection for ports and harbours, anti-submarine warfare, modelling and simulation, and marine mammal risk mitigation.

CMRE operates two ships, NATO Research Vessel Alliance, a 93-meter 3,180-ton open-ocean research vessel, and Coastal Research Vessel Leonardo, a smaller ship designed for coastal operations. In addition to its laboratories, the Centre is equipped with a fleet of autonomous underwater and surface vehicles and a world-class inventory of seagoing sensors.

JANUS research was principally sponsored by NATO Allied Command Transformation (ACT), bringing interoperability to NATO forces.

Figure 15: Adding 40m overshot to both ends of each survey line

UgCS is easy-to-use software for planning and flying UAV drone-survey missions. It supports almost any UAV platform, providing convenient tools for areal and linear surveys and enabling direct drone control. What’s more, UgCS enables professional land survey mission planning using photogrammetry techniques.

How to plan photogrammetry mission with UgCS

Standard land surveying photogrammetry mission planning with UgCS can be divided into following steps :

1. Obtain input data
2. Plan mission
3. Deploy ground control points
4. Fly mission
5. Image geotagging
6. Data processing
7. Map import to UgCS (optional)

https://www.youtube.com/watch?v=SalCQoG6WUw&feature=youtu.be

Step one: Obtain input data

Firstly, to reach the desired result, input settings have to be defined:

• Required GSD (ground sampling distance – size of single pixel on ground),
• Survey area boundaries,
• Required forward and side overlap.

GSD and area boundaries are usually defined by the customer’s requirements for output material parameters, for example by scale and resolution of digital map. Overlap should be chosen according to specific conditions of surveying area and requirements of data processing software.

Each data processing software (e.g., Pix4D, Agisoft Photoscan, Dronedeploy, Acute 3d) has specific requirements for side and forward overlaps for different surfaces. To choose correct values, please refer to documentation of chosen software. In general, 75% forward and 60% side overlap will be a good choice. Overlapping should be increased for areas with small amount of visual cues, for example for deserts or forests.

Often, aerial photogrammetry beginners are excited about the option to produce digital maps with extremely high resolution (1-2cm/pixel), and to use very small GSD for mission planning. This is very bad practice. Small GSD will result in longer flight time, hundreds of photos for each acre, tens of hours of processing and heavy output files. GSD should be set according to the output requirements of the digital map.

Other limitations can occur. For example, GSD of 10cm/pixel is required, but designed to use a Sony A6000 camera. Based on mentioned GSD and camera’s parameters, the flight altitude would be set to 510 meters. In most countries, maximum allowed altitude of UAV’s (without special permission) is limited to 120m/400ft AGL (above ground). Taking into account the maximum allowed altitude, the maximum possible GSD in this case could be no more than 2.3cm.

Step two: Plan your mission

Mission planning consists of two stages:

• Initial planning,
• Route optimisation.

-Initial planning:

The first step is to set surveying area using UgCS Photogrammetry tool. Area can be set using visual cues on underlying map or using exact coordinates of edges. The result – survey area is marked with yellow boundaries (Figure 1).

Figure 1: Setting the survey area

The next step is to set GSD and overlapping for the camera in Photogrammetry tool’s settings window (Figure 2).

Figure 2: Setting camera’s Ground Sampling Distance and overlapping

To take photos in Photogrammetry tool’s setting window, define the control action of the camera (Figure 3). Set camera by distance triggering action with default values.

Figure 3: Setting camera’s control action

At this point, initial route planning is completed. UgCS will automatically calculate photogrammetry route (see Figure 4).

Figure 4: Calculated photogrammetry survey route before optimisation

-Route optimisation

To optimise the route, it’s calculated parameters should be known: altitude, estimated flight time, number of shots, etc.

Part of the route’s calculated information can be found in the elevation profile window. To access the elevation profile window (if it is not visible on screen) click the parameters icon on the route card (lower-right corner, see Figure 5), and from the drop-down menu select show elevation:

Figure 5: Accessing elevation window from Route cards Parameters settings

The elevation profile window will present an estimated route length, duration, waypoint count and min/max altitude data:

Figure 6: Route values in elevation profile window

To get other calculated values, open route log by clicking on route status indicator: the green check-mark (upper-right corner, see Figure 7) of the route card:

Figure 7: Route card and status indicator, Route log

Using route parameters, it can be optimised to be more efficient and safe.

-Survey line direction

By default, UgCS will trace survey lines from south to north. But, in most cases, it will be more optimal to fly parallel to the longest boundary line of the survey area. To change survey line direction, edit direction angle field in the photogrammetry tool. In the example, by changing angle to 135 degrees, the number of passes is reduced from five (Figure 4) to four (Figure 8) and route length is 1km instead of 1.3km.

Figure 8: Changed survey line angle to be parallel to longest boundary

-Altitude type

UgCS Photogrammetry tool has the option to define how to trace the route according to altitude, with constant altitude above ground (AGL) or above mean sea level (AMSL). Please refer to your data processing software requirements as to which altitude tracking method it recommend.

In the UgCS team’s experience, the choice of altitude type depends on desired result. For orthophotomap (standard aerial land survey output format) it is better to choose AGL to ensure constant GSD for the entire map. If the aim is to produce DEM or 3D reconstruction, use AMSL so the data processing software has more data to correctly determine ground elevation by photos in order to provide more qualitative output.

Figure 9: Elevation profile with constant altitude above mean sea level (AMSL)

In this case, UgCS will calculate flight altitude based on the lowest point of the survey area.

If AGL is selected in photogrammetry tool’s settings, UgCS will calculate the altitude for each waypoint. But in this case, terrain following will be rough if no “additional waypoints” are added (see Figure 10).

Figure 10: Elevation profile with AGL without additional waypoints

Therefore, if AGL is used, add some “additional waypoints” flags and UgCS will calculate a flight plan with elevation profile accordingly (see Figure 11).

Figure 11: Elevation profile with AGL with additional waypoints

-Speed

In general, if flight speed is increased it will minimise flight time. But high speed in combination with large camera exposure can result in blurred images. In most cases 10m/s is the best choice.

-Camera control method

UgCS supports 3 camera control methods (actions):

1. Make a shot (trigger camera) in waypoint,
2. Make shot every N seconds,
3. Make shot every N meters.

Not all autopilots support all 3 camera control options. For example (quite old) DJI A2 does support all three options, but newer (starting from Phantom 3 and up to M600) cameras support only triggering in waypoints and by time. DJI promised to implement triggering by distance, but it’s not available yet.

Here are some benefits and drawbacks for all three methods:

Table 1: Benefits and Drawback for camera triggering methods

In conclusion:

• Trigger in waypoints should be preferred when possible
• Trigger by time should be used only if no other method is possible
• Trigger by distance should be used when triggering in waypoints is not possible to use

To select triggering method in UgCS Photogrammetry tool accordingly, use one of three available icons:

• Set camera mode
• Set camera by time
• Set camera by distance

-Glibal control

Drones, e.g., DJI Phantom 3, Phantom 4, Inspire, M100 or M600 with integrated gimbal, have the option to control camera position as part of an automatic route plan.

It is advisable to set camera to nadir position in the first waypoint, and in horizontal position before landing to prevent lenses from potential damage.

To set camera position, select the waypoint preceding the photogrammetry area and click set camera attitude/zoom (Figure 12) and enter “90” in the “Tilt” field (Figure 13).

Figure 12: Setting camera attitude

Figure 13: Setting camera position

As described previously, this waypoint should be a Stop&Turn type, otherwise the drone could skip this action.

To set camera to horizontal position, select last waypoint of survey route and click set camera attitude/zoom and enter “0” in the “Tilt” field.

-Turn types

Most autopilots or multirotor drones support different turn types in waypoints. Most popular DJI drones have three turn-types:

• Stop and Turn: drone flies to the fixed point accurately, stays at that fixed point and then flies to next fixed point.
• Bank Turn: the drone would fly with constant speed from one point to another without stopping.
• Adaptive Bank Turn: It is almost the same performance like Bank Turn mode (Figure 13), but the real flight routine will be more accurately than Bank Turn.

It is advisable not to use Bank Turn for photogrammetry missions. Drone interprets Bank Turns as “recommendation destination waypoint”. The drone will fly towards this direction but will almost never pass through the waypoint. Because drone will not pass the waypoint, no action will be executed, meaning the camera will not be triggered, etc.

Adaptive Bank Turn should be used with caution because a drone can miss waypoints and, again, no camera triggering will be initiated.

Figure 14: Illustration of typical DJI drone trajectories for Bank Turn and Adaptive Bank Turn types

Sometimes, adaptive bank turn type has to be used in order to have shorter flight time compared to stop and turn. When using adaptive bank turns, it is recommended to use overshot (see below) for the photogrammetry area.

-Overshot

Initially overshot was implemented for fixed wing (airplane) drones in order to have enough space for manoeuvring a U-turn.

Overshot can be set in photogrammetry tool to add an extra segment to both ends of each survey line.

Figure 15: Adding 40m overshot to both ends of each survey line

In the example (Figure 15) can be seen that UgCS added 40m additional segments to both ends of each survey line (comparing to Figure 8).

Adding overshot is useful for copter-UAVs in two situations:

1. When Adaptive Bank Turns are used (or similar method for non-DJI drones), adding overshot will increase the chance that drone will precisely enter survey line and camera control action will be triggered. UgCS Team recommends to specify overshot that is approximately equal to distance between the parallel survey lines.
2. When Stop and Turn type is in use in combination with action to trigger camera in waypoints, there is a possibility that before making the shot, drone will start rotation to next waypoint – it can result in having photos with wrong orientation or blurred. To avoid that, shorter overshot has to be set, for example 5m. Don’t specify too short value (< 3m) because some drones could ignore waypoints, that are too close.

Figure 16: Example of blurred image taken by drone in rotation to next waypoint

-Takeoff point

It is important to check the takeoff area at site before flying any mission! To better explain best practice on how to set takeoff point, first discuss an example of how it should not be done. Supposing that the takeoff point in our example mission (Figure 17) would be from the point marked with the airplane-icon, and drone pilot would upload the route on the ground with set automatic mission for automatic take-off.

Figure 17: Take-off point example

Most drones in automatic takeoff mode would climb to low altitude about 3-10meters and then fly straight towards the first waypoint. Other drones would fly towards first waypoint straight from ground. Looking closely at the example map (Figure 17), some trees between the takeoff point and the first waypoint can be noticed. In this example, the drone more likely will not reach a safe altitude and will hit the trees.

Not only the surroundings can affect takeoff planning. Drone manufacturers can change drones elevation behavior in drone firmware, therefore after firmware updates it is recommended that you check drones automatic takeoff mode.

Also, a very important consideration is that most small UAVs use relative altitude for mission planing. Altitude counted relatively according to first waypoint is a second reason why an actual takeoff point should be near the first waypoint, and on the same terrain level.

UgCS Team recommends placing the first waypoint as close as possible to actual takeoff point and specifying a safe takeoff altitude (≈30m in most situations will be above any trees, see Figure 18). This is the only method that warrants safe takeoff for any mission. It also protects from any weird drone behaviour and unpredictable firmware updates, etc.

Figure 18: Route with safe take-off

-Entry point to the survey grid

In the previous example, (see Figure 18), it can be noticed, that after adding the takeoff point, the route’s survey grid entry point was changed. This is because if additional waypoint is added subsequently to the photogrammetry area, UgCS will plan to fly the survey grid starting from nearest corner to the previous waypoint.

To change the entry point to survey grid, set additional waypoint close to the desired starting corner (see Figure 19).

Figure 19: Changing survey grid entry point by adding additional waypoint

-Landing point

If no landing point will be added outside the photogrammetry area after the survey mission, the drone will fly and hover in the last waypoint. There are two options for landing:

1. Take manual control over the drone and fly to landing point manually,
2. Activate the Return Home command in UgCS or from Remote Controller (RC).

In situations when the radio link with the drone is lost, for example if the survey area is large or there are problems with the remote controller, depending on the drone and it’s settings, one of these actions can occur:

• Drone will return to home location automatically if lost radio link with ground station,
• Drone will fly to last waypoint of survey area and hover as long as battery capacity will enable that, then: drone will perform emergency landing, or it will try to fly to home location.

The recommendation is to add an explicit landing points to the route in order to avoid relying on unpredictable drone behavior or settings.

If the drone doesn’t support automatic landing, or the pilot prefers to land manually, place the route’s last waypoint over the planned landing point with an altitude for comfortable manual drone descending and landing above any obstacles in the surrounding area. In general 30m is best choice.

-Action execution

Photogrammetry tool has a magic parameter “Action Execution” with three possible values:

• Every point
• At start
• Forward passes

This parameter defines how and where camera actions specified for photogrammetry tool will be executed.

The most useful option for photogrammetry/survey missions is to set forward passes, the drone will make photos only on survey lines, but will not make excess photos on perpendicular lines.

-Complex survey areas

UgCS enables photogrammetry/survey mission planning for irregular areas, having functionality to combine any number of photogrammetry areas in one route, avoiding splitting the area in separate routes.

For example, if a mission has to be planned for two fields connected in a T-shape, and if these two fields are marked as one photogrammetry area, the whole route will not be optimal, regardless any direction of survey lines.

Figure 20: Complex survey area before optimisation

If the survey area is marked as two photogrammetry areas within one route, survey lines for each area can be optimised individually (see Figure 21).

Figure 21: Optimised survey flight passes for each part of a complex photogrammetry area

Step three: deploy ground control points

Ground control points are mandatory if the survey output map has to be precisely aligned to coordinates on Earth.

There are lots of discussions about the necessity of ground control points in cases when a drone is equipped with Real Time Kinematics (RTK) GPS receivers with centimeter-level accuracy. This is useful, but the drone coordinates are not in themselves sufficient because, for precise map aligning, image center coordinates are necessary.

Data processing softwares like Agisoft Photoscan, Dronedeplay, Pix4d, Icarus OneButton and others will produce very accurate maps using geotagged images, but the real precision of the map will not be known without ground control points.

Conclusion: ground control points have to be used to create survey-grade result. For a map with approximate precision, it is sufficient to rely just on RTK GPS and the capabilities of data processing software.

Step four: fly your mission

For carefully planned missions, flying it is the most straightforward step. Mission execution differs according to the type of UAV and equipment used, therefore it will not be described in detail in this article (please refer to equipment’s and UgCS documentation).

Important issues before flying:

• In most countries there are strict regulations for UAV usage. Always comply with the regulations! Usually these rules can be found on web-site of local aviation authority.
• In some countries special permission for any kind of aerial photo/video shooting is needed. Please check local regulations.
• In most cases missions are planned before arriving to flying location (e.g., in office, at home) using satellite imaginary from Google maps, Bing, etc. Before flying always check actual circumstances at the location. There could be a need to adjust take-off/landing points, for example, to avoid tall obstacles (e.g., trees, masts, power lines) in your survey area.

Step five: image geotagging

Image geotagging is optional if ground control points were used, but almost any data processing software will require less time to process geotagged images.

Some of the latest and professional drones with integrated cameras can geotag images automatically during flight. In other cases images can be geotagged in UgCS after flight.

Very important: UgCS uses the telemetry log from drone, that is received via radio channel, to extract the drone’s altitude for any given moment (when pictures were taken). To geotag pictures using UgCS, assure robust telemetry reception during flight.

For detailed information how to geotag images using UgCS refer to UgCS User Manual.

Step six: data processing

For data processing, use third party software or services available on the market.

From UgCS Team experience, the most powerful and flexible software is Agisoft Photoscan (http://www.agisoft.com/), but sometimes too much user input is required to get necessary results. The most uncomplicated solution for users is online service Dronedeploy (https://www.dronedeploy.com/). All other software packages and services will fit somewhere between these two in terms of complexity and power.

Step seven (optional): import created map to UgCS

Should the need arise for the mission to be repeated in the future, UgCS enables importing the GeoTiff file as a map layer and using it for mission planning. More detailed instructions can be found in UgCS User Manual. See the result of an imported map created using UgCS photogrammetry tool imported as GeoTiff file in Figure 22.

Figure 22: Imported GeoTiff map as layer. The map is output of a Photogrammetry survey mission panned with UgCS

Visit the UgCS homepage

Download this tutorial as a PDF

If you liked this tutorial, you may also enjoy these:

• Programming for robotics: Introduction to ROS
• Python programming your NAO robot
• Robotics, maths, python: A fledgling computer scientist’s guide to inverse kinematics
• Adjust how you learn and quickly pick up robotics programming
• 3-D printing hydraulically-powered robots, no assembly required

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

We have just taken another momentous step in the journey to unveil the hidden wonders of our own planet!

Since the launch of the Shell Ocean Discovery XPRIZE at the American Geophysical Union Fall Meeting in San Francisco in December 2015, individuals from around the world have been racing to form Teams and develop a range of groundbreaking technologies to access the deep-sea. Registration closed at the end of September 2016 with 32 bold Teams stepping forward to take on the challenge of mapping and imaging our ocean as never before.

Today, we announce the 21 semi-finalists Teams advancing in the Ocean Discovery XPRIZE. These innovative semi-finalist Teams, consisting of almost 350 individuals from 25 countries, represent a broad, impressive diversity of backgrounds and expertise, including middle and high school students, university students, maker-movement enthusiasts, and water and ocean industry professionals.

This diversity is also reflected in the array of technologies and techniques the semi-finalist Teams are developing. Some are proposing to use drones as a mechanism to drop subsea instruments into the water, while others are proposing to use drones that not only go through air, but then dive into the watery depths. Non-aerial Entries include autonomous surface vehicles carrying subsea robots that will return to the ‘mothership’ when their work is done, as well as vehicles and robots that will remain beneath the sea surface from the moment they leave the shore.

The 21 semi-finalist Teams will now have to prove their technology in the field in Round 1 testing, which will take place later this year. As with every XPRIZE, the Shell Ocean Discovery XPRIZE sets audacious goals. In Round 1, the Entries that these Teams are developing will have only 16 hours to map the sea-floor at depths of 2000m and produce a high-resolution map (at least 5m horizontal resolution, at least 0.5m vertical resolution) of at least 20% of the 500km2 Competition Area. Additionally, Teams will have to bring back 5 images of an archeological, biological, or geological feature, as well as an image of an object that we will specify. As if this wasn’t enough, they have an additional hurdle to overcome – they will have to deploy from the shore with no humans allowed in the competition area.

To put this challenge into perspective, it can take days to map 500 km2 of the ocean using current state-of-the-art technologies. If you want 5m accuracy, mapping can take over a week and requires going out on a ship, which can easily cost over $60,000 per day, making it a very expensive endeavor. The cutting-edge technology that will come out of this XPRIZE will truly revolutionize our ability to access the ocean.

In addition to the main Prize of $6 million, a dozen Teams have also opted to compete for the National Oceanic and Atmospheric Administration Bonus Prize of $1 million. These Teams are developing pioneering technology to detect a chemical or biological signal underwater and then autonomously track that signal back to its source.

Partnering with us to truly make this competition a global success are industry titans, Fugro and Esri. Fugro, the industry leader in ocean mapping and survey, will be provide the baseline maps against which the judging panel will assess the Entries. Esri, the global leader in GIS, is providing software to the Teams to produce their maps.

We are looking forward to an exciting competition with exponentially evolving technologies, including smart underwater robots, robot swarms, drones, and artificial intelligence to perform the tasks that have been laid out. These Teams will really take us where no one has gone before! As their journeys of discovery set sail, we invite you to follow the competion and this phenomenal mission to map our beautiful planet.

If you liked this article, you may also be interested in:

• XPRIZE calling all innovators to help map our ocean
• Unmanned and underneath: Stay in the know about underwater drones
• Into the deep: Underwater machines keep an eye on climate change
• What you need to know about underwater drones
• CoCoRo: Tracking the development of the world’s largest autonomous underwater swarm
• Ocean-going robot fleet successfully completes fish-tracking mission

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The Zwentendorf Nuclear Power Plant in Austria was completely built but never put into operation. Reactor supervision and decommissioning is one important operational area for robots in the RN domain.

There is significant potential for the use of unmanned vehicles in scenarios involving radiological and nuclear (RN) threats. The European Reference Network for Critical Infrastructure Protection (ERNCIP) has therefore established a thematic group on the protection of critical infrastructure from Radiological and Nuclear Threats (RNTs).

The group looks at issues such as standardisation of deployment protocols, response procedures and the use of unmanned systems. Possible threats involve measurement and sampling scenarios that are too risky for humans to carry out, also in the event of criminal or unauthorised acts involving nuclear or other radioactive material. Situations envisaged for the use of unmanned ground vehicles (UGV) are:

• Reactor supervision and related accidents, such as Chernobyl and Fukushima;
• Illicit release of radioactive material (radiological dispersion devices and dirty bombs before or after an explosion);
• Search of sources out of regulatory control;
• Long-term measurements.

Lessons learned from incidents such as Fukushima and Chernobyl, as well as from decommissioning of old nuclear power plants, show that robots have certain advantages. Robots can operate in areas with high radiation, danger of explosives, for example Boiling Liquid Expanding Vapour Explosions (BLEVE), collapsing structures, Improvised Explosive Devices (IED), booby traps, aggressive chemicals (e.g. Chlortrifluorid) and extreme heat. Additionally, they can manipulate the environment and take samples. Robots can also be used for long-time surveillance of contaminated areas and for monitoring the movements of a threat using real-time data from mobile sensors.

Robotics in the RN domain – applications and standards

The ERNCIP RNTs group focusses on possible applications. The identified scenarios can be separated in two categories. First, there are prevention scenarios where unmanned systems can be used to prevent incidents involving radioactive material and deterrence. Second, there are response scenarios where unmanned systems gather information after incidents with radioactive material occurred. We further identified several tasks for unmanned systems:

• spatial mapping of RN sensor data (exploration, change detection);
• searching for RN sources (active sensing, isocurves, hotspots);
• sampling (air, sweep or material sampling);
• 3D infrastructure mapping (situation awareness & assessment).

Possible applications for robots in the RN domain: taking samples of potentially hazardous materials (left) or measuring and searching for radiological and nuclear sources (right).

Another important finding was that no standards, best practices or norms for sampling or taking measurements with unmanned systems have been systematically developed thus far. Therefore, a first set of potential standards for unmanned systems in RN measurement scenarios was compiled. A widely accepted standard collection of frameworks for robot software development is the Robot Operating System (ROS). Further important standards concerning communication with robots and control of unmanned systems are the Battle Management Language (BML), InterOperability Profile (IOP) and Joint Architecture for Unmanned Systems (JAUS). Furthermore, there are efforts for standardisation in the International Electrotechnical Commission (IEC) regarding international standards for RN measurements with unmanned systems.

Trials and competitions – great way to raise interest in RN related robotics

The results of the ENRCIP RNTs group pointed to an inherent need for a practical element shaping and enforcing the standardisation process in the robotics community. Unfortunately, it was also obvious that there is currently little to none interest of the academic robotics community in the RN field. There is, however, an effective means to address large robotic communities: the field of robotic competitions. Since robots usable in RN scenarios most probably have to be outdoor/off-road systems, the RNT group, thus, decided to set up an RN scenario at a popular outdoor robotics competition.

The group also identified a huge gap between what the R&D community is able to deliver, the existing industry state of the art and the user requirements (if properly determined). This includes the complete absence, non-observance or non-compliance of standards, best practices and norms. Naturally, the R&D community does not have a focus on standardisation, manageability, sustainability, robustness or reliability. Again, robotic competitions can be used as method to bring together users, academia and industry. The users can visualise their needs in the form of real world scenarios to be tackled by the R&D community and industry. At the same time, the industry is able to mingle with the R&D community. Thus, a constructive way to push forward unmanned systems in RN scenarios might be the utilisation of trials and challenges.

The European Land Robot Trial 2016 took place in Eggendorf, Austria. For the first time, a RN task with real radiation sources was included and attracted robot teams from all over the world.

In 2016, the European Land Robot Trial (ELROB) was hosted by the Austrian Army in the Tritolwerk in Eggendorf. The Tritolwerk is an old ordnance factory, now used for CBRNE training and training for emergency services. These premises gave the perfect setting for a very special scenario: Reconnoitring of urban structures with a focus on radiological and nuclear measuring and mapping. This was the first real world live scenario with strong radiation sources (Co60, 2,8GBq each) on a robotics competition ever! The task involved detecting an unknown number of hidden radiation sources. The robotic systems had to measure the radiation, display measurements to the operator, acquire imagery and mark the sources in an online-built map representation. Ten teams participated in the scenario. Since this was the first try of such a scenario, the expectations were curbed. However, the teams performed reasonably well, and most presented RN readings. It turned out that many teams did not have enough background knowledge on radiation detection to produce suitable results.

European Robotics Hackathon – fully focused on RN tasks

Based on these circumstances a new robotic trial, the first European Robotics Hackathon (EnRicH) was created and will take place in June 2017. The scenario at EnRicH is inspired by the radiological reconnaissance task at ELROB 2016 and has a similar setting, indoor search for radiation sources and 3D mapping, but with an additional manipulation task. The event will be accompanied by a R&D workshop which is planned and conducted by the ERNCIP RNTs task group. The workshop has a focus on standards for unmanned systems in the RN field of application. The EnRicH hackathon will take place in the inoperative nuclear power plant (NPP) in Zwentendorf, Austria. The boiling water reactor of NPP Zwentendorf has been fully completed but never put into operation. In Zwentendorf areas are easily accessible which in other NPPs can only be visited under severe difficulties. In an active nuclear power plant extensive safety precautions are needed for human personnel due to the high level of radioactivity. Instead, in the NPP Zwentendorf engineers have transformed the plant and turbine halls into a training centre. Repair and dismantling measures but also critical incidents and disaster scenarios can be trained under realistic conditions.

Naturally, a nuclear power plant contains areas which cannot be safely entered by human personnel on one hand, due to inherent radiation like in the reactor building, and on the other hand due to technical problems or accidents leading to an unpredictable and uncontrolled radioactive contamination of parts of the plant. In any case, a robotic system doing an exploration of the scene would be helpful. This is the basic setting for the newly created European Robotics Hackathon.

In EnRicH, the exploration task consists of three sub-tasks: first, a digital 3D map of the area of interest has to be built; second, radiation and its sources must be detected, measured and marked inside a digital map; and, third, if a system is equipped with a manipulator device, some radioactive material has to be handled. All collected information has to be passed on to a reach back team for further evaluation.

The interior of the Zwentendorf Nuclear Power Plant provides the setting for the first European Robotics Hackathon (EnRicH). Robots will have to build a conventional 3D map of the scene, measure and map radiation sources, and manipulate hazardous material.

During the trial, major challenges are waiting for the participating systems and teams: First, severe mobility problems are caused by the typical interior of a power plant with elements like low or no light, closed doors, dead ends and blockings, sharp turns or steep stairs. Second, major difficulties for any kind of communication approach can be expected due to massive concrete walls and metal interiors. However, as a fall-back solution the organizers provide a standard wireless communication infrastructure for the teams. And, finally, due to the active radiation sources inside the scenario nobody will be allowed to accompany the robots, making robustness an essential prerequisite for the systems. However, the scene will be equipped with cameras, allowing some basic supervision for the operators from outside.

Hence, the problems of robots in hazardous operations and emergency response, especially when involving RN components, are diverse. Trials like ELROB and EnRicH reveal many remaining technical challenges, such as communication, sensors, situational awareness, mobility/locomotion and robustness, which often accrue from the fact that these systems have not been used and tested enough. There are still a lot of lessons to be learned.

If you liked this article, you may also be interested in:

• Russia looks inward and outward at Moscow’s 2013 Open Innovations Forum
• Mining and nuclear decommissioning: Robots in dangerous and dirty areas
• American nuclear materials storage site guarded by robots
• Disaster response robot will inspect the damaged Fukushima nuclear reactor building

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Jussi Aaltonen explaining some robotics-specific topics.

The UNEXMIN (Underwater Explorer for Flooded Mines) project is almost one year old. After a busy first year of work, UNEXMIN is on-schedule to deliver the first mechanical UX-1 prototype. Jussi Aaltonen, from TUT (Tampere University of Technology), leaders of WP1 – Robotic Functions Validations, talks about what has been done over the past year concerning his team’s work in UNEXMIN’s development scene.

Luís Lopes, LPRC: What is WP1 (Robotic Functions Validations) and what has been developed from it?

Jussi Aaltonen: The purpose of WP1 is to develop and validate a robotic platform, i.e. all mechanical and mechatronic structures, systems and components of UX-1 (the name of the robot). Also, low level (directional) control principles are developed and validated in this work package. WP1 has successfully established a robotic platform which fulfils all necessary requirements.

LL: How important is the work developed in WP1 for the future UX-1 robots

JA: Work in WP1 is fundamental for the successful completion of the project. All later development stages rely on this work.

LL: The systems/subsystems that the robot will employ were defined in this WP. What can you tell about them? How crucial are they for the robot’s overall functionality?

JA: All systems are needed for the robot to achieve full functionality. However, survivability, i.e. the robot’s capability to return from its missions, is more heavily affected by some systems than others. It is, for example, possible to return with a broken multi-spectral camera, but losing a sonar may cause the robot to be lost.

LL: What kinds of tests were developed to test the hardware/software systems? How about the results obtained, were they positive?

JA: Tests are done as so-called hardware-in-the-loop (HWIL) tests in laboratory. In HWIL testing, computer simulations and real hardware components interact, making it possible to do laboratory testing more closely to real operating conditions. Components and systems have also been tested in the pool by using a plastic mock-up of UX-1. Tests have been successful, which, unfortunately, does not always mean positive results. But weaknesses were found and lessons learned.

LL: In what conditions was the prototype tested?

JA: Tests have been done in shallow depths (below 5 m) and moderate temperatures. Pressure testing at 60 bars (equivalent of 600 m depth) will start soon.

LL: The robot is designed considering stakeholders’ wants. How hard was it to define the requirements and specifications for the UX-1 robots?

JA: I have to say that requirements and specifications are the easy part. The hardest is to design a robot complying them all.

LL: Being leader of such an important WP of the UNEXMIN project must be challenging. Has it worked out as you imagined? Do you feel that everything has been accomplished?

JA: Over this past year of cooperation, UNEXMIN teams from the various partners have gotten to know each other and each other’s ways of working. The possibilities to meet the others in person and to really collaborate in real life conditions, such as in workshops, have been a great benefit and a very important thing, bonding people from all around Europe together and working towards  a common goal. Of course there are always some challenges. The biggest challenge has been finding a common and efficient enough platform for information exchange.

LL: TUT will organise a UNEXMIN workshop in the beginning of February (2017) in Tampere, Finland. What can be expected to come out of this?

JA: This event will conclude the work of the first year in the UNEXMIN project. Most of the work packages have already started or about to start (exception goes to WP7: Demonstration/Pilots), and thus we will hear a lot about results and developments in this meeting. We will also hear about the plans for a couple of essential work packages starting right after the meeting. I expect that we will have a good and fruitful meeting with the same good spirit and warm atmosphere that we have always had. There will probably be some heated discussions, but that is only a sign of people being passionate about their work.

LL: You are a specialist in the robotics field. What possible impact do you think the UNEXMIN project will have on the robotics field?

JA: UX-1 [UNEXMIN robot design-prototypes] will have a very different structure and layout than what autonomous underwater vehicles usually have. This will most probably have some effects on the field of robotics and will open new doors to further miniaturised underwater robots – the future starts here. Also, the environment where UX-1 will operate is probably the most challenging after Mars and asteroid landers and that will also give tiger leaps in the field of robotics, developing new and better robotic systems.

LL: After one year, how do you feel the UNEXMIN project has progressed?

JA: We are roughly in the place where we should be according to our plans. Work has been harder than what was expected. On the other hand, we learned a lot more, while still managing to keep pace.

This post was originally published on UNEXMIN.

If you enjoyed this article, you might also be interested in:

• Surveying flooded mines with UNEXMIN: Interview with Norbert Zajzon
• When robots become art: Interview with Yi-Wei Keng
• A European perspective on robot law: Interview with Mady Delavaux
• Art meets robotics in Patrick Tresset’s OPLINE Prize entry
• Robots Podcast #171: Robotics in theatre, film and television, with Grant Imahara and Richard McKenna

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Simulation results for a unmanned drone flying over downtown Los Angeles showing the true trajectory (red line), from GPS only (yellow line), and GPS aided with cellular signals (blue line). Image: ASPIN Laboratory at UC Riverside

A team of researchers at the University of California, Riverside has developed a highly reliable and accurate navigation system that exploits existing environmental signals such as cellular and Wi-Fi, rather than the Global Positioning System (GPS). The technology can be used as a standalone alternative to GPS, or complement current GPS-based systems to enable highly reliable, consistent, and tamper-proof navigation. The technology could be used to develop navigation systems that meet the stringent requirements of fully autonomous vehicles, such as driverless cars and unmanned drones.

Led by Zak Kassas, assistant professor of electrical and computer engineering in UCR’s Bourns College of Engineering, the team presented its research at the 2016 Institute of Navigation Global Navigation Satellite System Conference (ION GNSS+), in Portland, Ore., in September. The two studies, “Signals of Opportunity Aided Inertial Navigation” and “Performance Characterization of Positioning in LTE Systems,” both won best paper presentation awards.

Most navigation systems in cars and portable electronics use the space-based Global Navigation Satellite System (GNSS), which includes the U.S. system GPS, Russian system GLONASS, European system Galileo, and Chinese system Beidou. For precision technologies, such as aerospace and missiles, navigation systems typically combine GPS with a high-quality on-board Inertial Navigation System (INS), which delivers a high level of short-term accuracy but eventually drifts when it loses touch with external signals.

Despite advances in this technology, current GPS/INS systems will not meet the demands of future autonomous vehicles for several reasons: First, GPS signals alone are extremely weak and unusable in certain environments like deep canyons; second, GPS signals are susceptible to intentional and unintentional jamming and interference; and third, civilian GPS signals are unencrypted, unauthenticated, and specified in publicly available documents, making them spoofable (i.e., hackable). Current trends in autonomous vehicle navigation systems therefore rely not only on GPS/INS, but a suite of other sensor-based technologies such as cameras, lasers, and sonar.

“By adding more and more sensors, researchers are throwing ‘everything but the kitchen sink’ to prepare autonomous vehicle navigation systems for the inevitable scenario that GPS signals become unavailable. We took a different approach, which is to exploit signals that are already out there in the environment,” Kassas said.

A schematic showing how Zak Kassas, assistant professor of electrical and computer engineering at UC Riverside, and his team are using existing communications signals to complement satellite-based navigation systems like GPS for the control of driverless cars and unmanned aerial vehicles (UAVs). Image: ASPIN Laboratory at UC Riverside

Instead of adding more internal sensors, Kassas and his team in UCR’s Autonomous Systems Perception, Intelligence, and Navigation (ASPIN) Laboratory have been developing autonomous vehicles that could tap into the hundreds of signals around us at any point in time, like cellular, radio, television, Wi-Fi, and other satellite signals.

In the research presented at the ION GNSS+ Conference, Kassas’ team showcased ongoing research that exploits these existing communications signals, called “signals of opportunity (SOP)” for navigation. The system can be used by itself, or, more likely, to supplement INS data in the event that GPS fails. The team’s end-to-end research approach includes theoretical analysis of SOPs in the environment, building specialized software-defined radios (SDRs) that will extract relevant timing and positioning information from SOPs, developing practical navigation algorithms, and finally testing the system on ground vehicles and unmanned drones.

“Autonomous vehicles will inevitably result in a socio-cultural revolution. My team is addressing the challenges associated with realizing practical, cost-effective, and trustworthy autonomous vehicles. Our overarching goal is to get these vehicles to operate with no human-in-the loop for prolonged periods of time, performing missions such as search, rescue, surveillance, mapping, farming, firefighting, package delivery, and transportation,” Kassas said.

In addition to Kassas, graduate students Joshua Morales, Joe Khalife, Kimia Shamaei, Jesse Garcia, and Sonya Ragothaman and undergraduate student Souradeep (Gogol) Bhattacharya contributed to this research, which was funded with support from the Office of Naval Research (ONR). Additional videos of the team’s research can be viewed on the ASPIN Laboratory’s YouTube page.

The UCR Office of Technology Commercialization has filed patents for the inventions above.

This article was originally published by UCR Today.

If you liked this article, you may also enjoy reading:

• Swarms of precision agriculture robots could help put food on the table
• Quadrotor automatically recovers from failure or aggressive launch, without GPS
• Robots Podcast #214: Project Ngulia: from Phone to Drone, with Fredrik Gustafsson
• Oxbotica’s autonomous vehicle software learns its environment and how it changes over time

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Swarms of drones will help farmers map weeds in their fields and improve crop yields. This is the promise of an ECHORD++ funded research project called ‘SAGA: Swarm Robotics for Agricultural Applications’. The project will deliver a swarm of drones programmed to monitor a field and, via on-board machine vision, precisely map the presence of weeds among crops.

Additionally, the drones attract each another at weed-infested areas, allowing them to inspect only those areas accurately, similar to how swarms of bees forage the most profitable flower patches. In this way, the planning of weed control activities can be limited to high-priority areas, generating savings at the same time as increasing productivity.

“The application of swarm robotics to precision agriculture represents a paradigm shift with a tremendous potential impact” says Dr. Vito Trianni, SAGA project coordinator and researcher at the Institute of Cognitive Sciences and Technologies of the Italian National Research Council (ISTC-CNR). “As the price of robotics hardware lowers, and the miniaturization and abilities of robots increase, we will soon be able to automate solutions at the individual plant level,” says Dr. Trianni. “This needs to be accompanied by the ability to work in large groups, so as to efficiently cover big fields and work in synergy. Swarm robotics offers solutions to such a problem.” Miniature machines avoid soil compaction and can act only where needed; robots can adopt mechanical, as opposed to chemical, solutions suitable for organic farming; and robot swarms can be exactly scaled to fit different farm sizes. The Saga project proposes a recipe for precision farming consisting of novel hardware mixed with precise individual control and collective intelligence.

In this particular case, innovative hardware solutions are provided by Avular B.V., a Dutch firm specializing in industrial level drones for monitoring and inspection. Individual control and machine vision are deployed thanks to the expertise of the Farm Technology Group at Wageningen University & Research, The Netherlands. Swarm intelligence is designed at the already mentioned ISTC-CNR, leveraging their expertise to design and analyse collective behaviours in artificial systems. For the next year, these organisations will team up to produce and field-test the first prototype for weed control based on swarm robotics research.

About SAGA

SAGA is funded by ECHORD++, a European project that wants to bring the excellence of robotics research “from lab to market”, through focused experiments in specific application domains, among which is precision agriculture. SAGA is a collaborative research project that involves: the Institute of Cognitive Sciences and Technologies (ISTC-CNR) of the Italian National Research Council (CNR), which provides expertise in swarm robotics applications and acts as the coordinator for SAGA’s activities; Wageningen University & Research (WUR), which provides expertise in the agricultural robotics and precision farming domains; and Avular B.V., a company specialised in drone solutions for industrial and agricultural applications.

Click here for more information

Raheeb Muzaffar, an information technology specialist, has developed an application-layer framework that improves the transmission of videos between moving drones and mobile devices located at ground level. His work within the Interactive and Cognitive Environments (ICE) doctoral programme will be completed soon. Raheeb explains what makes this technology innovative and talks about his plans for the future in a conversation with Romy Mueller.

Raheeb Muzaffar was already aware of what to expect when he arrived in Klagenfurt as a PhD student, travelling from Islamabad, the capital of Pakistan, December 2012: He’s travelled well before, had experience working with various universities and professional organizations, and was acquainted both with the European culture and the research environment. The doctoral programme “Interactive and Cognitive Environments (ICE)” suited him perfectly, allowing him to conduct research in an international environment. Doctoral students accepted into the programme are funded for a period of three years. During this time, they work at two universities. The research teams at the Alpen-Adria-Universität and the Queen Mary University in London offered Muzaffar, who wanted to work on the communication between drones, an environment that allowed him insight into various laboratories and research approaches. By combining the key research areas he experienced in Austria and Great Britain, he ultimately arrived at his very own topic: video transmission in drone networks.

He spotted a problematic issue in this context, which he has studied very closely over the past few years. “In a situation where several drones are airborne and are transmitting videos to several units located at ground level, we cannot currently guarantee that the footage will be reliably transmitted.” The problem is that the receiving devices do not have a feedback mechanism for the video packets being transmitted from the drones. The missing feedback mechanism causes unreliability rendering a distorted video reception. Also, when the transmitting devices and the receiving devices are in motion, the wireless transmission conditions are subject to continuous change, introducing an additional factor of uncertainty.

To address this problem, Raheeb Muzaffar has developed a so-called “video multicast streaming framework”, which allows feedback from multiple ground devices to the drones to add reliability allowing a smooth video reception. However, the framework is even more capable: “The drone does not only learn whether the video has arrived, but thanks to the feedback mechanism, the drone can adjust the transmission rate and the video quality to match the prevailing wireless conditions. Thus, the transmitting drones can re-send lost video packets following the transmission and video rate adaption.” Video communication between drones and receiving devices is useful for various applications such as search and rescue, surveillance, and disaster management.

When we asked him, if he can make profitable use of his technology, Raheeb Muzaffar laughed. At various conferences, he talked to the developers from the industry, who confirmed to him that modules endowed with similar functionality are expensive and that the WLAN protocol requires modification. However, his application is freely available. Anyone wishing to commercialise the technology as a product, is able to do so. In any case, over the course of numerous simulations and experiments, he has demonstrated that the streaming framework works and uses the conventional WLAN protocol without modifications.

Muzaffar’s research in Klagenfurt is coming to an end. By November, he will complete his doctoral thesis, and will pull up stakes in Klagenfurt, following his hopes and dreams to work in the research-based industry. The researcher was accompanied by his wife when he came to Klagenfurt. In the meantime, they celebrated the birth of their daughter. The Muzaffar family intends to travel more: “I am not tied to one location. I would enjoy working anywhere in Europe, but we also like the idea of moving to New Zealand.”

A few words with Raheeb Muzaffar

What career would you have chosen, if you had not become a scientist?

If I would not have become a scientist, I would have become a social worker. Before turning to research, I worked with Non-governmental organizations including iMMAP and UNICEF thinking that my technical expertise could help expedite the social work that the organizations are involved in. Besides the work I do plan to involve myself in some sort of social work may it be technical, educational, financial or emotional.

Do your parents understand what it is you are working on?

Yes, especially my father, he is quite interested in technology. He may not have the technical details but he is aware of research and the work I am involved in.

What is the first thing you do when you arrive at the office in the morning?

First thing in the morning is to check and answer my emails. Thereafter, I work by the plan. I always have a plan for the week/month regarding the tasks I need to do. Although sometimes things don’t go by the plan and I have to readjust the plan to meet deadlines.

Do you have proper holidays? Without thinking about your work?

Yes, at least once a year.

Why are so many people afraid of technical sciences?

In my opinion, every individual has a different aptitude to his/her work domain and one should do what one thinks he/she is interested in and can perform best. Having said that, working in the domain of technical science is quite challenging. The field has advanced and it is getting increasingly complex to even gain the basic understanding of different domains. To work in this field, one has to stay up to date. Developing something new involves a minimal margin of error. I think, people are afraid due to the complex nature of the studies that involve an extra bit of hard work, understanding, and patience.

Written by: Romy Müller, University of Klagenfurt

Originally published in German language in AAU News of the University of Klagenfurt, Austria. Also published at Medium.com.

Two UGVs and two UAVs in front of the San Francesco church. Credit: TRADR project

After the devastating earthquake in Amatrice, Italy, on August 24th, the Vigili del Fuoco (Italian fire brigade) requested assistance from the TRADR (Long-Term Human-Robot Teaming for Robot-Assisted Disaster Response) project (EU FP7 framework, grant No. 60963). TRADR deployed two Unmanned Ground Vehicles (UGVs) and three Unmanned Aerial Vehicles (UAVs) to assist the post-earthquake response in Amatrice.

A TRADR team under the command of the VVFF officer Emanuele Gissi, coordinated by Ivana Kruijff-Korbayova (DFKI, Germany) and composed of TRADR project collaborators from the Sapienza University, Rome, Italy; Czech Technical University, Prague, Czech Republic; Fraunhofer Institute for Intelligent Analysis and Information Systems, Sankt Augustin, Germany and Ascending Technologies, Munich, Germany was assembled on short notice and arrived at the earthquake disaster site within 48 hours after receiving the request for help.

The team involved in the TRADR deployment. (left-to-right): Erik Zimmermann; Wolfgang Rottner; Hartmut Surmann; Ivana Kruijff-Korbayová; Emanuele Gissi;VdF fire fighter; policeman Luigi Ugliano; Mario Gianni; Kneeling: Krešimir Dilic; Václav Hlaváč; Luigi Freda; Valsamis Ntouskos; Vladimír Kubelka. Credit: TRADR project

The task was to use robots to provide 3D textured models of two churches, San Francesco and Sant’ Agostino, national heritage monuments from the XIVth century. Both were in a state of partial collapse and in need of shoring to prevent potential further destruction. The models should serve to plan the shoring operations and to assess the state of various objects of cultural value inside the churches, such as valuable frescos.

UAV image inside the Sant’Agostino church. Credit: TRADR

The UGVs successfully entered the San Francesco church, (operated by Vlada Kubelka and Mario Gianni, who were supported by Erik Zimmermann and Luigi Freda, respectively), teleoperated entirely out of line of sight and partially in collaboration. For part of the mission, one UGV provided a view of the other one to enable maneuvering in very constrained space with low connection bandwidth. One of the UGVs operated in the church continuously for four hours. A UAV was also present for a short time in parallel and provided additional views of the UGVs.

UAV image inside the San Francesco church. Credit: TRADR

Several flights were carried out on the outside, and one flight inside of each church. Entering with a UAV was a tough challenge, which we could only manage thanks to a collaboration between three UAVs operated in parallel. While one UAV (piloted by Hartmut Surmann who was supported by Erik Zimmermann) entered through a hole in the roof the other two (piloted by Kresimir Dilic and Wolfgang Rottner) provided simultaneous video feed from different angles, that the pilot used as an additional source for orientation. An assistant watching the video then gave the pilot verbal instructions on how to fly. This demanding operation presented a teamwork masterpiece.

UAV entering the Sant’Agostino church with one of the UAVs observing it (the other observing UAV is taking the picture). Credit: TRADR

To our knowledge, this was the first time several different kinds robots have been used in collaboration in a real disaster response deployment. The mission fulfilled its goal to collect data for the construction of high-quality textured 3D models, which have been provided both to the VVFF and to the Italian Ministery of Culture. Additional great success was the fact that the TRADR team was able to arrive and deploy at the location on such short notice. The VVFF has expressed high appreciation for the success of the TRADR project mission and smooth operation, and expressed their intention to continue and further deepen collaboration efforts.

A late breaking report about the TRADR deployment in Amatrice will be presented at the 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), October 23-27, Lausanne, Switzerland.

Flooded shaft in Ecton. Source: UNEXMIN

The UNEXMIN project (Underwater Explorer for Flooded Mines) started February 2016. It is an EU-funded project, from the Horizon 2020 research and innovation programme. Luís Lopes, from the La Palma Research Centre, interviews project coordinator Norbert Zajzon to discuss the past, present and future for UNEXMIN.

Luís Lopes, LPRC: First things first – what is UNEXMIN?

Norbert Zajzon: UNEXMIN is the abbreviation of the “Underwater Explorer for Flooded Mines” H2020 project, based on the topic “New sustainable exploration technologies and geomodels”.

LL: Where did the idea for UNEXMIN come from? It seems to be a crazy one.

NZ: It is a quite ambitious idea! Basically, it is to support the raw materials’ needs of the EU, to gather valuable information from flooded mines without risking human lives, but it also can be used in different fields like archaeology, rescue or water pipeline monitoring and so on. If I am correct, the first basic idea did come from Balázs Bodó (LPRC) a few years ago, which was further evolved and matured for the final submission of the project. In the beginning, the idea seemed to be taken from a sci-fi movie!

UNEXMIN team learning about the geology in Ecton Mine.

LL: The main point of this project is to develop a multi-robotic system that can autonomously map flooded mines. How difficult is it to achieve this?

NZ: It is a very difficult task. Developing a non-contact robot, which is fully autonomous, in a very complex setting with lots of obstacles and mapping the environment and simultaneously gathering geochemical and mineralogical information underwater is very hard. I feel it would be easier to perform the same task on the surface of the Moon or a planet like Mars.

Big shaft in Deep Ecton

LL: How important is it to map the flooded mines in Europe?

NZ: In Europe that are about 30.000 closed mine sites, many of which are now flooded. It is very important to map and know what is in our abandoned mines under the surface, mainly from the raw materials’ point of view, but also to document and safeguard cultural heritage or as a hazard study for buildings on the surface in the case of mine collapse.

LL: UNEXMIN has a total of 13 partners within its consortium. What areas of expertise are present in this project?

NZ: This is a truly multidisciplinary consortium with experts in geology-geochemistry, mineral exploration, oceanic research, mine engineering, mechanical engineering, automatization, robotic development, autonomous system control, 3D geological software development and visualization, experts in dissemination and networking, experts in founding and running enterprises. And even with this extensive list I feel I might have missed some important areas present in our consortium.

LL: What are the big challenges that the UNEXMIN consortium is facing/is going to face during the robotic platform development?

NZ: The three main challenges I personally feel are:

1. Energy consumption of the robot, as it will have to run on batteries and cannot use any outside energy sources.
2. The small size of the robot, as it will have to fit into medieval mine tunnels, which can be very small, and still carry all the necessary instrumentation.
3. Survive the pressure and the – sometimes – very aggressive mine waters and still perform the geological and mineralogical measurements in that environment.

LL: What impacts can be expected from UNEXMIN?

NZ: UNEXMIN is a project that could have a lot of impacts in the future, mainly at European level, but also internationally. The three most important impacts I foresee are:

1. Pushing the EU to the forefront in sustainable minerals surveying and exploration technologies.
2. Increase Europe’s capacity to evaluate its abandoned mines for their mineral potential.
3. Help to document and safeguard  Europe’s unique mining heritage.

Besides the main scope of the above impacts, UNEXMIN could also lead to the deploy of robotics technologies in new application domains, acquiring valuable geological information and allow technology transfer between robotic solutions and mineral industries, that will lead to further innovation in those areas.

LL: What are the next steps to be taken for the project development?

NZ: The next important steps are to further develop, simulate and test different conceptual models of the robot to be able to carefully choose the basic robot design, and also specify what kind of scientific instruments can be incorporated and with what kinds of performance constraints.

LL: Who are the interested parties in this project? And who can benefit from the service that is going to be available in the future?

NZ: There are a lot of interested parties, the so-called stakeholders, in this project. Overall, I can say that all individuals from the geology, minerals exploration and mining sector, the ICT/robotics community and even the general public interested in the thematics of this project are our main targets. I believe that the technology developed in UNEXMIN will benefit a whole range of future consumers: geology-related companies (mining, geological surveys or geoheritage sites), national authorities, universities and other companies in areas such as environment or industrial diving and even for tourist sites (e.g. caving).

Norbert Zajzon, UNEXMIN project coordinator, presenting the poster in Brussels. Source: UNEXMIN

LL: You were present in Brussels in an international event about raw materials. How did it go?

NZ: It went very well, actually! Many people were interested in our project not only from Europe but also from overseas, like Canada or Mexico. And UNEXMIN was invited to participate in further discussions and conferences in similar fields of research in the next year. We are starting to make a name for ourselves.

LL: When should we expect to see the first looks of UX-1?

NZ: The basic concept and the main conceptual sub-system plans should be ready in the beginning of 2017, so the first reliable look of UX-1 can be seen on next spring.

LL: And will it be a good looking robot or an ugly scrap of metal and other things?

NZ: The most important point of the design is functionality and survivability. As it has to move underwater in an environment with a lot of obstacles like ropes and spikes the basic shape is already decided, which is spherical with as a smooth surface as possible. Of course, it has to carry enormous amounts of light sources, lasers, cameras, different sensors and many thrusters on its surface, so the final look will be very interesting. The recent conceptual drawings are very exciting, so I think it will be a good looking robot, moreover, when it will run with all the different lights and lasers on for its sensors – the look will be astonishing!

LL: Final question: Is it difficult to be the project coordinator of such a big EU project?

NZ: Yes, it is a difficult and tiring job, but at the same time it is a very good and joyful experience to be in the centre of a very interesting technology development with a great variety of scientist experts in different fields from different parts of Europe, from Finland to Portugal thru Spain, England, Slovenia and Hungary.

In this episode, Audrow Nash interviews Fredrik Gustafsson, Professor in Sensor Informatics at Department of Electrical Engineering in Linköping University, about an initiative to reduce poaching in a rhino sanctuary in Ngulia, Kenya. Gustafsson discusses how he first became involved in this project, how he has worked with the rangers to develop solutions, and the future of this work.

Fredrik Gustafsson

Fredrik Gustafsson is professor in Sensor Informatics at the Department of Electrical Engineering, Linköping University, since 2005. He received the M.Sc. degree in electrical engineering 1988 and the Ph.D. degree in Automatic Control, 1992, both from Linköping University. His research interests are in stochastic signal processing, adaptive filtering and change detection, with applications to communication, vehicular, airborne, and audio systems. His work in the sensor fusion area involves design and implementation of nonlinear filtering algorithms for localization, navigation and tracking of all kind of platforms, including cars, aircraft, spacecraft, UAV’s, surface and underwater vessels, cell phones and film cameras for augmented reality. He is a co-founder of the companies NIRA Dynamics (automotive safety, including tire pressure monitoring systems found in more than 20 million, here), Softube (plug-ins for music studios and software solutions for Marshall and Fender), and Senionlab (leading manufacturer of indoor navigation solutions for sm, here).

Links:

• Download mp3 (16.9 MB)
• Project Ngulia website
• A PDF overview of Project Ngulia
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

UPDATE: Only 7 days left to back Sweep! They are now almost 80% funded.

Sweep is a small, lightweight and low-cost laser scanner. It spins a laser range finder to form a 360 degree view of the world, up to 10 times per second. What makes Sweep different is who it’s designed for. Until now, sensors like Sweep were too expensive to be used in cost sensitive products, relegating them to research, industrial, or military applications. With Sweep, we aim to bring the awesome accomplishments of the last two decades of LiDAR research out of the lab and into consumer products!

Sweep is the culmination of years of development work, and like so many inventions, it came as a bi-product of wanting to make something else. We started off developing outdoor consumer robots for the home. We wanted to make a smart product that could navigate through typical yard environments, avoiding obstacles and keeping track of its charging station location. We didn’t like the hassle of other solutions, like buried guide wires, and knew a better way would be to have the robot form a map of the environment. We quickly realized to do any kind of mapping we would need a scanning LiDAR (or something similar) that works outdoors. It was then we discovered that there were no affordable LiDAR sensors on the market that met the requirements.

So, we set out to make one ourselves!

We designed Sweep to be very small and light! It fits right in the palm of your hand and weighs a tad less than most smart phones today. Sweep is also really quiet, something you may have never thought about until you’ve used one that makes annoying sounds as it spins. You won’t be hearing this scanner.

After more than a dozen iterations, we arrived at our current production ready design. We’ve discovered that the key innovation with Sweep will be its ability to perform much of the required data filtering internally. Most LiDAR sensors output a tremendous amount of data, which is great for applications where power and cost are not an issue, but cumbersome for consumer applications. For example, aerial vehicles may only want to know where the closest obstacles are. Sweep is equipped with several sensing modes designed to make interfacing with lower power controllers possible.

Testing with beta units has been successful!

For more than a month we’ve had beta units in the hands of testers. We’ve received great feedback on how people want to use Sweep.

Companies like Dispatch Robotics want a sensor to help guide their delivery robot through city streets. Others want to use it to navigate through grocery isles and office spaces. They are all excited about how Sweep can cut down the parts cost of their products by thousands of dollars.

 At Dispatch, we are very excited by Sweep. LiDAR is a very powerful sensing modality and low-cost LiDAR will have significant impact as we develop and grow our autonomous vehicle fleet.”

Uriah Baalke, Dispatch Robotics

The Sweep is a great fit for high school robotics. It is easy to setup and use, robust, and is at an incredible price point. Students can quickly explore mapping and navigation problems with little prior experience. Best yet, the Sweep is virtually student-proof – it can survive a high school lab environment.”

Charles Williams Mentor, Cupertino High School Robotics

How does Sweep compare to other sensors used for obstacle avoidance?

Traditionally low cost robotic vehicles have used sonar, structured light cameras, or stereo vision to sense their environment, but they all have a limited range. When following user controls or GPS directions they may only have milliseconds to react when obstacles come into their field of view. In contrast, Sweep has a range of 40 meters, giving much more time to react to approaching obstacles. It also works in all lighting conditions, making it more reliable in the real world.

We’re targeting the initial release of Sweep toward early adopters and developers looking to try out the technology. As we receive more insight from our customers about their specific applications, we will refine future versions of Sweep to meet the demand. Likely areas of improvement include additional internal data filtering modes, more compact packaging, faster update rates, longer ranging capability and more environmental hardening.

The ultimate goal of Sweep is to empower new and existing applications with a smart and affordable scanning LiDAR solution!

Contact

More information can be found at their website: www.scanse.io

Thier kickstarter can be found here: https://www.kickstarter.com/projects/scanse/sweep-scanning-lidar

You can contact them directly at: Team@scanse.io

There is a very interesting planet near us. It has almost 130 million square miles of land and the longest continuous mountain chain in the solar system, spanning over 40,000 miles. This planet receives very little sunlight, so temperatures hover around 40C (39oF) regardless of the time of year, and atmospheric pressure is, on average, a crushing 400 times greater than our own. Our advanced probes tell us that there is very little oxygen and the environment is corrosive, so it is difficult to imagine anything living under these conditions. And yet we know with 100% certainty that life exists on this planet. Is this place real or science fiction? It’s real—and it’s only 4km away!

This is the planet at the bottom of our ocean.

Although it is so close to us, the deep ocean is a place we know very little about. We have better maps of Mars, hundreds of millions of miles away, than we do of our own sea floor. We don’t know how many volcanoes or mountains are hidden in the ocean, let alone what strange life forms exist down there. Less than 5% of the ocean has been explored because it is opaque and a physically challenging environment. It is also expensive to access; ships may cost over $60,000 per day to operate and vehicles that can go down into the deep sea can cost over $1 million. We need new breakthrough technologies that can help us to overcome the costs and can operate quickly and easily under great pressure in these cold and corrosive watery environments.

The Shell Ocean Discovery XPRIZE is a $7 million competition challenging teams from around the world to build advanced deep-sea underwater robots that will provide safe access to this tough environment, paving the way for autonomous, fast, and high-resolution ocean exploration. The success of this prize will result in technologies with which we can fully explore and map the ocean floor, uncovering our planet’s greatest wonder and allowing us to sustain and protect our deep-sea resources. We are also asking teams to advance our ability to see what is down there by producing high-resolution images of biological, geological, and archeological features. Embedded in this competition is a $1 million bonus prize from the National Oceanic and Atmospheric Administration (NOAA) to incentivize the development of smart technologies that can detect underwater chemical or biological signals and trace them to their source.

These breakthrough technologies will assist with search and rescue operations, giving us the ability to quickly and easily find planes and vessels that are lost at sea. The small fraction of the ocean we have explored has already provided us with potential treatments for cancer, Alzheimer’s, and AIDS. Imagine what other cures are down there. Underwater archeologists have uncovered long lost treasures from ancient shipwrecks such as the San Jose galleon off the coast of Columbia, shining a light on human history (it is estimated that there are 3 million shipwrecks at the bottom of the ocean). Our limited underwater images have shown astonishing creatures with abilities to camouflage themselves, generate their own electricity, and glow in the dark. What else might we discover if we could explore more of this vast resource?

By increasing our access to the ocean, we will increase our understanding of this wonderful and mysterious world, and by doing so we will realize it’s true value. The ocean is responsible for 50% of the oxygen we breathe, is the primary driver of our weather, and is a source of food for over 2 billion people. As such, it is a precious resource—and we need to make sure it remains healthy. The advances sparked by the Shell Ocean Discovery XPRIZE will also provide access and transparency in the deep-sea for better management and conservation efforts.

Join us on this grand adventure. We are calling all innovators to help us discover this new planet. Form a team and sign up to compete, spread the word, become a resource for teams, or follow us and find out about the mysteries of the deep at OceanDiscovery.xprize.org. Together, we can usher in a new and unprecedented age of ocean exploration.

Transcript below.

In this episode, Ron Vanderkley speaks with Stefan Williams of the University of Sydney’s Australian Centre for Field Robotics, Marine Systems Group. They discuss the future of Autonomous Underwater Vehicles (AUVs), and a recent expedition where they used multi-session SLAM to map the famous Antikythera Shipwreck (circa 60-80 B.C.), one of the richest ancient wrecks ever discovered. It is located under 55m of water on the NE coast of the island of Antikythera. The site is famous for the first Analog Computer known as the Antikythera Mechanism, a geared device designed to calculate and display celestial information, including phases of the sun and a luni-solar calendar.

The ACFR leads Australia’s Integrated Marine Observing System (IMOS) AUV Facility. IMOS is a nationally coordinated program designed to establish and maintain the research infrastructure required to support Australia’s marine science research. The IMOS AUV facility generates physical and biological observations of benthic variables that cannot be cost-effectively obtained by other means.

Stefan Williams
Stefan Williams is part of the Faculty of Engineering & Information Technologies, School of Aerospace, Mechanical and Mechatronic Engineering, University of Sydney’s Australian Centre for Field Robotics Marine Systems Group.

Links:

• Download mp3 (13.3 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• ACFR Marine Group

Transcript

Ron Vanderkley: Stefan, can I get you to introduce yourself to our podcast listeners?

Stefan Williams: I’m a professor of Marine Robotics at the University of Sydney’s Australian Center for Field Robotics. We undertake both fundamental and applied research in the area of marine robotics and the development of underwater systems.

Ron Vanderkley: What is the main goal of the Marine Group?

Stefan Williams: Our focus over the last decade or so has been to develop tools and techniques for better understanding marine environments. We work in a variety of different application areas, from ecology to archeology and geoscience, and we work in the engineering research areas that underpin these applications – such as vehicle navigation, control planning, machine learning, and computer vision – in order to provide better understanding and information to our end users about the data we are collecting.

Ron Vanderkley: When you talk about autonomous marine surveying systems, what kind of capabilities are we talking about today? Is this a mix of manual and autonomous techniques? Where are the AUVs?

Stefan Williams: The main work that we do is on autonomous underwater vehicles. These are vehicles that are disconnected from the surface. We program them with a particular mission that we’d like them to execute, which might be a big long transect or a particular depth profile, or it might include very detailed studies over a particular feature of interest. Then a vehicle goes down and executes that mission. We typically stay in communication with the vehicle using acoustic modems; the vehicle tells us what it’s doing, it gives us feedback on the data that it’s collecting, and we have some supervisory control to tell it “no, you’re in the wrong spot, please come up” or “there’s some issue with the mission so abort” for example. We also have some ability to redirect the missions while they’re underway.

The data is collected onboard the vehicles. Once we recover them we download the data and build detailed 3D models of these marine habitats primarily using vision data, but we also collect multi-beam sonar, so we have acoustic data as well if we’re looking to map broader areas of the sea floor. That’s the core technology that has been developed around these autonomous platforms, but we’ve also built systems for both shallow water surveys using divers, and also very deep-water exploration using remotely operated vehicles. We also have platforms and sensor packages that we can use in other ways beyond just the autonomous platforms.

Ron Vanderkley: You spoke about the acoustic modems. What level of data can you get out of that? Is it to the level of telemetrics? Can you actually sift some data instead of just downloading it at the end of the mission?

Stefan Williams: Most of the data that we get back at the moment is supervisory in nature. The robot will tell us where it is, how deep it is, how far it is off the bottom, and the amount of data or number of images that it has collected. This lets us monitor the progress of the mission and make sure that things are on track. That modem is also piggybacked on positioning information, so it tells us where the vehicle is as an independent measurement from the ship. We then send both the ship position and the observations of the robot’s position relative to the ship down to the vehicle, and it updates its estimate of where it is using that information. We’re able to correct any drift in the navigation estimate using those external observations.

We’ve just taken on a new positioning and modem package in the past year or so that is capable of higher baud rates. In theory, we can get thumbnails of images. I’ve seen some work where they’re actually using this same sort of acoustic modem to do low-bandwidth video across an acoustic length, but that’s a capability we still have to develop.

Ron Vanderkley: I believe that your group is working on some novel enhancement to acoustic communication from the operator to the robot. Is that to increase the baud rate?

Stefan Williams: We’ve just been awarded an Australian Research Council Linkage Grant, which encourages the research community to engage with interested partners. This is a partnership with Tellus Australia – they do a variety of different developments in the marine space, particularly focused on defense applications. The project is really about managing the communication channels: it’s less about developing new acoustic communications equipment than it is about what information is best to send across the acoustic communication channel so that the operator can most effectively monitor the progress of the mission. In an information theoretic sense, we want to decide what information is best from the point of view of giving the operator an idea of what’s happening, and then allowing that operator to make decisions and query for further information from the vehicle. For instance, it might be used to refine and estimate where particular targets are in an environment, or to request more detailed high-resolution components of a particular area that might be of interest to the operator.

Ron Vanderkley: I also was aware of a multi-session SLAM-based AUV mapping system, and there was an example of a wreck that you worked on. What did that entail?

Stefan Williams: We’ve been involved over the past year in projects together with the Woods Hole Oceanographic Institute and the Hellenic Ministry of Culture and Sport (and a few other organizations) in mapping and excavating a 1st century BC shipwreck off the coast of the Greek island of Antikythera. This wreck was first discovered in 1900 and excavated around that time, yielding a whole trove of statues and other ancient artifacts – including something called the Antikythera Mechanism, which is one of the first examples of what people consider to be an analog computer. It was a geared mechanism that was used for predicting celestial motion over a fairly extended period of time to predict where the sun was and the phases of the moon, etc. It’s a really unique artifact that hasn’t been documented previously. It’s a pretty fascinating site to be exploring.

The objective of our work there was to provide the archeologists with a baseline map before they started further excavation on the site. We first visited the site in September 2014 and conducted surveys over the main target area – as well as looking at other targets of interest that had been identified using acoustic multi-beam data collected in the previous year.

We returned in June 2015 and mapped a very extensive part of the coastline around the wreck site because some of the diving operations they’d conducted in 2014 identified other areas of interest. We delivered a map of their target excavation sites and they were able to use those maps in situ. They actually had copies of the maps on iPads, and were able to annotate these while diving underwater in depths of 50-60 meters. They were able to log all of their finds and situate themselves on the wreck site relative to the maps that we generated. It provided them with a baseline of what the site looked like before they started with the excavation operations.

They’ve just finished their field research and an exhibit has opened in Basel, Switzerland showcasing some of the finds and describing the site and some of the archeological work that’s been going on there. It’s been a really exciting project to be part of and has attracted a lot of interest. Our part of that was using AUV systems to give them baseline high-resolution 3D maps of the wreck site.

Ron Vanderkley: It kind of sounds like “Jacques Cousteau meets robot”

Stefan Williams: In fact Jacques Cousteau visited the site in 1976 and they did some excavation that was part of their Calypso program. They did a lot of underwater exploration and recovered a few artifacts, but this is really the first time that the site has been systematically mapped.
Ron Vanderkley: What is the future of AUVs? Where are the challenges?

Stefan Williams: AUVs have a big role to play in expanding our understanding particularly in deepwater environments. Our main focus has been in coastal environments linked to ecological and biological studies. I think there is a role to play also in understanding how changes in oceanography (for example temperature, salinity, and current profiles) are affecting the marine habitat. We have a big program as part of Australia’s Integrated Marine Observing System Program, running AUVs to look at how key ecological environments around the country are changing over time. We revisit these areas, and work closely with biologists and ecologists to document changes.

Another big role for these autonomous platforms is in understanding deepwater environments – getting them out into the abyssal plains to find out more about what’s down in the deep parts of the oceans, of which we know very little about.

One thing that really excites me is looking at some of the advancements that were made in long-term autonomous deployment. A lot of these autonomous underwater vehicle systems are really starting to come into the norm. There’s some fantastic work going on around the world in designing and building these systems both on manned surface vessels that are capable of very long range operation, and on underwater vehicles that are capable of thousands of kilometers of deployment.

That really changes the dynamics of how ocean science can be conducted. It reduces the requirement of having people on very expensive ships out at sea for long periods of time if you can send these robotic systems off to work autonomously. We’ve seen the precursors in oceanography, with autonomous float systems being used to help oceanographers get a better understanding of ocean circulation and physical oceanography on a global scale. The next revolution will be autonomous platforms that can do mapping and excavation work over long periods of time in deep water across the world. I think there’s real potential in that area.

Ron Vanderkley: What is Australia’s role in that? Are we able to create our own submersibles? Are we concentrating on the programming in that arena? Where do you see us?
Stefan Williams: We have covered both areas natively in Australia. Some of these vehicles are being purpose-designed for particular tasks. We’re currently designing our next generation of hydrolution survey vehicle for supporting the Marine Robotics program, and there are programs in Queensland and Tasmania looking at requirements for working under ice or in complex reef environments.

There is work here in Australia in building these sorts of systems, though there is not a very large commercial scene yet (a lot of the commercial platforms are being developed in the US and Europe). But there’s certainly a lot of research effort focused on designing vehicles that are fit for purpose, and I think we’re particularly well known for the mapping and the navigation work that we do. We have links with all the major oceanographic groups around the world. I think Australia as a whole is very well regarded in robotics and autonomous systems – both within the marine space, and also in ground and air vehicles – and is having a big impact in this area.

Ron Vanderkley: Finally, I’d like to ask you about employment prospects in Australia. If you were a student going to university looking at the options, where would you point them?

Stefan Williams: We have a lot of students come both through undergraduate and postgraduate teaching and research schemes, and the majority of them do find work both domestically and internationally. There’s certainly a lot of activity currently in the robotics space. Some of the big companies like Google, Apple and Uber have made well documented forays into autonomous driving systems, and they’re certainly employing a lot of PhD students, post docs and research fellows who have been employed in various labs around the world, so there’s a lot of potential there. We have former students and staff embedded in the majority of those companies.

Here in Australia we have a domestic market with a number of small startups that are really doing well in developing robotics systems and sensor payloads. The experience students get – and the type of skills that they acquire through their study of mechatronics and robotics – have application across a wide variety of domains, from traditional engineering work, to software, electronics and mechanical design work … all the way through finance and business studies. There’s a lot of interest in data analytics and machine learning, and a lot of the tools and techniques that we apply in robotics can be transferred across a variety of different domains.

One of my most recent students is now heading up a small Machine Learning group at one of the major banks. The skills and underpinning tools required here in our lab to classify marine imagery have direct application into other areas. When students ask me about what jobs are out there for mechatronics engineers, I tell them that there are always good jobs for good students. One of the keys to that is really being passionate about the things that you’re studying, and I think robotics excites a lot of people.

I teach an experimental robotics course – it’s an elective in the 4th year mechatronics programs here at Sydney University. The things that the students do in the course of the 13-week curriculum are astounding. They put a lot of time, effort, and thought into building these robotic systems. I think they feel very rewarded at the end, and the feedback I’ve gotten has always been very positive.
Ron Vanderkley: Thank you, Stefan, for your time. I hope to speak to you again soon.

Stefan Williams: It’s a real pleasure, Ron. Thanks for helping to promote the activities of robotics here in Australia and overseas.

This audio interview was transcribed and edited for clarity. 

Monitoring the ocean for pollution, ecology and climate change effects is a costly and elaborate task, especially in a complex area like Venice, with its lagoon, its many channels, cable and pipe infrastructure, industrial areas, harbor, marshland, and mussel farms. An interdisciplinary team of European scientists is breaking new ground in underwater environmental monitoring with the EU-funded subCULTron project. With a budget of 4M Euros, they are developing the world’s largest intelligent underwater monitoring system that coordinates, communicates and collects data autonomously. 

subCULTron is coordinated by the Artificial Life Lab of the University of Graz, working together with scientists from Italy, Belgium, France, Germany and Croatia. The group is developing a swarm of 120 autonomous underwater robots that will collect environmental data to help provide insight on how the Venice waters are influenced by the complex interplay of industry, tourism, transport, and local inhabitants. One of the long term goals of the project is to find ways to prevent future damage in the region.

A smart monitoring-system based on bio-inspired autonomous robots

Scientists have chosen an autonomous swarm approach for this challenging task because such systems are known to be very robust, flexible and scalable, especially in unpredictable, changing environments. Bio-inspired algorithms derived from derived from social insects, fish or slime molds are used for the swarm communication, as these systems can work without any central unit of control, and the loss of individual robots doesn’t lead to the failure of the whole system.

Another advantage of developing a self-organizing robot society is that the single robots are inexpensive and small compared to most conventional underwater robots, which makes it possible to build a larger fleet (in this case 120 robots) to maximize the sampling area and to collect large amounts of very diverse data in parallel. 

How does the subCULTron swarm work?

The subCULTron swarm will consist of three different robot types that are designed and developed especially for the project and which represent cutting edge technology and software: 

• artificialMussels (aMussels) are novel autonomous robots that sit on the ground after being deployed in the water to collect various data over a long period of time, establishing a dedicated sensory network. When necessary, they will also be able to move as a group, exploiting given energy sources, e.g., turbulences or water currents. 
• artificialFish (aFish) are fast and agile underwater robots that are used to monitor the ground and to search for given targets. They also bridge communication from aMussels to artificialPads (aPads).
• aPads are robots that float on the water surface, inspired by lily pads on a pond. They complete the communication chain from aMussels on the seabed via aFish to aPads and finally to the scientists in the lab.

The goal is to have these three different robot types running efficiently by exploiting advanced energy harvesting techniques developed by the project team.

What novel knowledge is expected from the subCULTron swarm?  

Thanks to the large number of swarm-robots, the autonomous communication system and the novel robot types, an enormous amount of environmental data about the Venetian underwater world will be collected in parallel. The hope is that this data will yield new insights into the effects of human intervention on the natural water habitat in and around Venice, helping to preserve this world heritage site from the effects of climate change.

Biologists and ecologists should also benefit from insight gained into ecological systems, and it is hoped that the tech industry will be able to build on subCULTron to innovate technology for underwater applications.

subCULTron goes to EXPO 2015

From October 15-16, subCULTron 2015 scientists will demonstrate their first robot prototypes live in small pool experiments at the Venice EXPO 2015. 

• WHEN: 15^th-16^th of Oct. 2015
• WHERE:  AQUAE VENICE 2015 Pavillion
• ADRESS: Via Galileo Ferraris, 5 , Venezia Marghera
• LINK: (http://www.aquae2015.org/where/?lang=en)

This video shows how a robot team can work together to map and navigate toward a goal in an unknown terrain that may change over time. Using an onboard monocular camera, a flying robot first scouts the area, creating both a map of visual features for simultaneous localization and a dense elevation map of the environment. A legged ground robot then localizes itself against the global map, and uses the elevation map to plan a traversable path to a goal.

While following the planned path, the absolute pose corrections are fused with the legged robot’s state estimation and the elevation map is continuously updated with distance measurements from an onboard laser range sensor. This allows the legged robot to safely navigate towards its goal while taking into account any changes in the environment.

More info: http://leggedrobotics.ethz.ch

This work was published as:
P. Fankhauser, M. Bloesch, P. Krüsi, R. Diethelm, M. Wermelinger, T. Schneider, M. Dymczyk, M. Hutter, and R. Siegwart, “Collaborative Navigation for Flying and Walking Robots,” in IEEE International Conference on Intelligent Robots and Systems (IROS), 2016.

Torre del Sale building, Piombino, Italy. Image: eurathlon

Inspired by the 2011 Fukushima accident, euRathlon is a civilian outdoor robotics competition focused on realistic cooperative disaster response scenarios. euRathlon 2015 takes place in the area surrounding the building of Torre del Sale, on the site of the Enel-owned power plant, and 16 teams from ten different countries have travelled to Piombino, Italy, to tackle this unique challenge. In the first day of trials, aerial, ground and marine robots had to deal with the very real conditions of a demanding coastal environment. Watch the Day One recap video.

The Opening

The mayor of the city of Piombino, Massimo Giuliani, gave a warm welcome to all the attendees, followed by the Enel engineer Antongiulio Bertoncini, Dr Anne Bajart (the project’s European Commission officer), Professor Alan Winfield (euRathlon project Coordinator) and Dr Gabriele Ferri (Head of the euRathlon 2015 robotics competition).

The Trials

The euRathlon 2015 scenarios are arranged in increasing order of complexity, with the trials first, followed by the sub-challenges and finally the Grand Challenge.

Trials are single-domain scenarios designed to encourage the participation of teams that may not have the time, resources and/or experience to directly address the euRathlon Grand Challenge. But, at the same time, they serve as a warm up for the multi-domain (land, sea and air) teams. There are a total of six trials divided into two land trials (L1, L2), two sea trials (S1, S2) and two air trials (A1, A2).

Land Trial (L1): Reconnaissance in Urban Structure

For the first of the two land trials the competitors’ robots were required to approach, inspect and map a ruined building. The land robots had to navigate across a beach filled with obstacles such as wood debris, vegetation and sand dunes. Once inside the building, robots had to look for mock damage and the entrance to the machine room.

A schematic map of the building was given to the teams at the beginning of the trial. The map didn’t include detailed dimensions of entrances, corridors and rooms. As part of the staging, entrances are randomly blocked as a result of a hypothetical earthquake and tsunami.

The seven teams participating in this land trial were B.R.A.I.N. Robots, Cobham, Bebot, ICARUS, ENSTA Bretagne Team 1, ENSTA Bretagne Team 2 and SARRUS.

The beach natural environment turned out to be a real challenge for the robots and only two teams successfully entered the building.

Team ICARUS (ICARUS EU-FP7) preparing for the land trial. Image: euRathlon

Sea Trial (S1): Navigation and Environmental Survey

The sea trial took place in the Enel-owned plant harbour, where robots were tasked with surveying an area of 15×15 metres in the basin of the dock.

The AUVs (Autonomous Unmanned Vehicles) had to navigate from the launch point to a validation gate, pass through it and search for damage on the seabed, marked by submerged orange buoys.

The 11 teams participating in the sea trial competed in parallel, in two different areas. The teams were OUBOT, AUV Team TomKyle, ENSTA Bretagne Team 2, AVORA, UNIFI, ENSTA Bretagne Team 1, AUGA, Team Nessie, ICARUS, Robdos Team and Universitat de Girona.

Teams had to contend with poor water visibility worsened by the rough sea conditions of the previous days. Although acquiring live video and images of the buoys with the on-board camera was a challenge, most of the teams succeed in the navigation task.

Team OUBOT (Obuda University) after their participation in the sea trial.

Air Trial (A1): Aerial Detection and Mapping

In this first air trial, teams were called on to conduct an aerial inspection to find a safe path from the starting point, for a land robot, to an unblocked entrance of the building.

Specific take-off and landing areas were designated for the UAVs (Unmanned Aerial Vehicles).

The robots had to map the area surrounding the building and look for objects of potential interest (OPIs) representing blocked and unblocked entrances.

The six teams that participated in this trial were UNIFI, ICARUS, ENSTA Bretagne Team 1, ENSTA Bretagne Team 2 and ISEP/INESC TEC.

Team UNIFI ( University of Florence) getting ready for the aerial trial. Image: euRathlon

The teams faced variable wind conditions during the day, but that did not stop their robots from taking-off and flying around the building. Most of the teams succeed in surveying the area and finding a safe path for the land robot. 

Watch this space for Day Two: the second set of land, and sea trials!

The location of euRathlon 2015 competition at Torre del Sale, Piombino, Italy. Photo credits: EURATHLON

On March 11, 2011, a tsunami and earthquake struck Fukushima Dai-ichi energy plant causing a major disaster. Since then emergency services have deployed robots to inspect the area and collect data. Some robot surveys were successful while others failed, underlining the pressing need to advance the state-of-the-art in autonomous robots for disaster response. Further research is necessary, and so is testing in realistic environments. Shinji Kawatsuma, from the Japan Atomic Energy Authority (JAEA), urged in his outstanding talk in euRathlon 2013: “Don’t build robots, build robot systems”. And this is what euRathlon 2015 is all about: testing the capabilities of robot systems in realistic mock emergency-response scenarios.

euRathlon is an outdoor robotics competition funded by the European Union with a focus on realistic cooperative search and rescue response scenarios for land, sea and air robots. Inspired by the 2011 Fukushima accident, the euRathlon 2015 Grand Challenge can only be met when these three domains of robotics are working cooperatively. Previous single-domain euRathlon competitions focussed on land (2013), sea (2014). These, alongside team building workshops, and have been building up to the 2015 Grand Challenge, where the overall aim is to advance the state-of-the-art in heterogeneous teams of autonomous vehicles. Multi-disciplinary and multi-organisational teams of students and engineers will survey the scene, collect data, search for missing workers, identify critical hazards, and more, all in a race against the clock. Of the sixteen teams that have qualified for euRathlon 2015, ten will participate in the Grand Challenge (sea, land and air) creating a total of 6 multi-domain teams.

The scenarios

The euRathlon 2015 Challenge will be held in the area surrounding the Tor del Sale building, and in the Enel-owned thermal power plant sheltered harbour.

Although competing robots will face mock scenarios, the environmental conditions and difficulties are intended to be as realistic as reasonably possible, and the success criteria will reflect straightforward end-user priorities such as task completion and minimal intervention with the robots.

Three different types of scenarios have been defined:

• The Grand Challenge: a three-domain air, land and sea scenario, which comprises three missions.
• Sub-Challenge: two-domain scenarios; each is a combination of two trials.
• Trial: single-domain scenarios.

The scenarios will begin with the six Trials (2 land, 2 sea and 2 air), move on to the three Sub-Challenges (land + air, sea + air, land + sea), and finish with the Grand Challenge. The euRathlon Grand Challenge main goal is task fulfilment, but autonomy and cooperation between domains are also essential for achieving the mission objectives – to search for missing workers, conduct an environmental survey, and stem a leak – within the allocated time.

Drafts of the scenarios, competition rules and the scoring/benchmarking schema were published in advance for teams to comment. However, detailed descriptions of the scenario setups, for instance the location of blocked doors or missing workers, have not been given to teams because such details would not be known ahead of time in a real life scenario.

euRathlon 2015 Grand Challenge schematic. Credits: EURATHLON

The teams

As in previous euRathlon events, teams were required to undertake a qualifying process to prove that their vehicle(s) would be able to cope with the scenarios. The teams were asked to write a Scenario Application Paper (SAP) for each of the scenarios they were planning to enter. In the SAPs they describe their robot(s) and explain how they plan to address the technical challenges for each specific scenario. A technical advisory board reviewed the SAPs and checked that the proposed solutions were realistic.

The 16 teams participating in the euRathlon 2015 Challenge are:

1. AUGA – Team AUGA (or “water” in Galego) is from ACSM (Advanced Crew and Ship Management), a company that will participate in the sea trials with one of the SPARUS II AUVs that were loaned to less experienced teams by the euRathlon organisation.
2. AVORA – The AVORA team is from the University of Las Palmas de Gran Canaria, Spain, which will participate in the sea trials with their robot, Charle.
3. AUV Team Tom Kyle – AUV Team Tom Kyle is from the University of Applied Sciences of Kiel, Germany, which will participate in the sea trials with their robot Robbe 131 and will partner with Bebot team for the Sub-challenges and Grand Challenge.
4. Bebot Team – Bebot is a two-domain team from the University of Applied Sciences of Bern, Switzerland. They will participate in the competition with their robots Bebot (land) and Bee (Air), and will partner with AUV Team Tom Kyle to participate in the Grand Challenge.
5. B.R.A.I.N. Robots –  B.R.A.I.N. Robots e.V. from Esslingen, Germany, participated in euRathlon 2013. In 2015 they will participate in trials and sub-challenges with their robots Monique (land), Alice (land), Caro (sea) and Hannah (sea). They will team up with UNIFI Team for the Grand Challenge.
6. Cobham – Team Cobham is from the German-based company Cobham Mission Systems. Cobham was one of the winners of euRathlon 2013 with their robot Telemax. This year they will participate in land trials and will team up with ISEP/INESC TEC and Universitat de Girona for the Sub-Challenges and Grand Challenge.
7. ENSTA Bretagne Team 1 – one of the two multi-domain teams from the Institute of ENSTA Bretagne, France, that are going to participate in the trials, such-challenges and Grand Challenge. ENSTA is the only institution that participated in both euRathlon 2013 and 2014. They will participate with the robots SAUC’ISSE, SARDINE, USV, ETAS WHEEL, ETAS TRACK, buggy, and two quadrotors.
8. ENSTA Bretagne Team 2 – the second of the two multi-domain teams from the Institute of ENSTA Bretagne, France. Like Team 1, they are going to participate in trials, such-challenges and Grand Challenge. Their robots are CISCREA, EUREKA, TREX, a buggy and two quadrotors.
9. ISEP/INESC TEC Aerial Team – The team from ISEP/INESC TEC (Portugal) will participate in the aerial trials with their robot OTUS. They will team up with Universitat de Girona and Cobham for the Sub-Challenges and Grand Challenge.
10. ICARUS – This is the team of the ICARUS EU-FP7 Project, with partners from Belgium, Germany, Poland, Portugal and Spain. This multi-domain team will participate in the trials, sub-challenges and the Grand Challenge with their robots ASCAMM-LIFT-V and ASCAMM-LIFT-VI (air), UKL-SUGV and RMA-IMM-tEODor (land), INESC_ROAZ and INESC_MARES (sea).
11. Team Nessie – The team from the Ocean Systems Laboratory & Heriot Watt University (UK) will participate in sea trials with their robots NESSIE VII and E.M.I.L.Y.
12. OUBOT – The team from Obuda University (Hungary) will participate with one of the SPARUS II UAV loaned by euRathlon consortium in the sea trials.
13. Robdos Team Underwater Robotics – Robdos is formed by a group of students from the University Polytechnic of Madrid (UPM) and the Spanish-based company Robdos SRL. They will participate in the sea trials with one of the SPARUS II that euRathlon loaned to them and which they participated in euRathlon 2014.
14. SARRUS-Search And Rescue Robot of UPM & SENER – is the team of the Universidad Politecnica de Madrid (UPM), Spain, and the company SENER Ingenieria y Sistemas S.A. They will participate in land trials with their robot Summit XL.
15. UNIFI Team – UNIFI is the team of the University of Florence (Italy) that will participate in the sea and land trials with their robots FEELHIPPO (sea), MARTA (sea) and BATIGOL (air). They will team up with B.R.A.I.N Robots for the Sub-Challenges and Grand Challenge.
16. Universitat de Girona – The team of the University of Girona is formed by students and researchers of the Underwater Vision & Robotics Research Centre (CIRS). The team develops and operates the robot vehicle SPARUS II AUV that was one of the winners of euRathlon 2014 sea competition. This year they will participate in sea trials and will team up with Cobham and ISEP/INESC TEC for Sub-Challenges and the Grand Challenge.

Which teams will successfully address the Grand Challenge? Don’t miss the updates starting this week.

For pictures and more information about the teams visit: http://www.eurathlon.eu/index.php/compete2/teams2015/

Flying Fotokite close up

Zurich-based robotics company Perspective Robotics this week unveils Fotokite Phi, their first consumer targeted, affordable, accessible, tethered flying camera. The Fotokite Phi is available now for pre-order on Indiegogo for $349 USD.

https://www.youtube.com/watch?v=aglTWdI-4Bo

The story behind Fotokite

Inspired by drone operators during the Russian protests in 2011, Perspective Robotics’ CEO Sergei Lupashin set out to make a drone that is safe and easy to use. The result is the Fotokite, a tethered user-friendly quadcopter, with a unique take on safety for both the operator and the people nearby. The Fotokite Phi is based on the same technology that Lupashin demo’ed on the TED stage in 2014, and that is also currently being used by major news outlets, such as the BBC, for their broadcast coverage and special programming.

Key benefits include:

Instant launch, intuitive gesture control

Fotokite is the only drone on the market that doesn’t rely on remote piloting or GPS, making for a simpler, more intuitive, user experience. To launch the Fotokite Phi, the user simply points it in the direction they want it to go and gives the device a quick shake. The onboard GoPro is powered by the Phi and automatically turns on when flying. The Fotokite’s core algorithms – developed by Lupashin while he worked on his PhD at the Flying Machine Arena at ETH Zurich, Switzerland – are protected by a patent application and enable the device to fly itself, controlled by simple gestures, on the end of the tether, with no need for a remote control or GPS.

Foldable and portable

The Fotokite Phi easily fits into a backpack or suitcase. It weighs 12 ounces, including the GoPro 3/3+/4 camera and battery, and is the lightest GoPro-carrying quadcopter on the market. The Fotokite Phi folds into a compact carrying case (about the size of a two liter soda bottle) and can easily fit in a backpack or suitcase for people traveling with it. The Fotokite Phi comes with a 26 foot leash included and a removable battery that charges directly through a USB port.

Safer for users and the people around them

If the Fotokite Phi detaches from the tether, the device is programmed to detect a lack of tension and float slowly to the ground, instead of flying away. In addition, the Fotokite’s soft propellers are safe to the touch.

Fotokite Phi folded, half in its tube

Accountability

The tether makes it clear to bystanders who’s controlling the device, allowing for accountable aerial photography. To this end, Fotokite has secured special exemptions in Switzerland and France to be flown near crowds. Fotokite is currently working towards acquiring the same permission from the FAA in the US.

Fotokite at a museum

Flight Unit

Propellers : Soft Plastic 6,5 inch Fotokite Prop

Weight : 300g (excluding GoPro Camera), 350g (including GoPro camera)

Battery : Li Po Battery (1000 Ah, 11.1Wh), 3 cells

Flight time : We are aiming for 15 minutes

Camera : Go Pro 3 / 3+ / 4

Charger : USB 3C Charging cable

Dimensions : 90mm height, 350mm diameter, 170mm rotor diameter

CPU : ARM Cortex Processor

Sensors : IMU, no GPS

Motors : 1400 Kv Brushless

Smart-Leash

8m cord, self retracting

Dimensions : 40mm height, 90mm diameter

Sensors : IMU

Case

Dimensions : 90mm x 300mm

Weight : 150g

China is the world’s largest consumer market of industrial robots for the second consecutive year, according to statistics released by the China Robot Industry Alliance (CRIA). Sales in China increased by 54.6% in 2014 to around 57,000 units, 25% of the global total. Data shows that nearly 17,000 units were made in China with a value of $474m, an increase of 60% from 2013.

Various research reports are predicting that more than 250,000 industrial pick and place, painting and welding robots will be purchased and deployed in China by 2019. That figure represents more than the total global sales of all types of industrial robots in 2014! The Chinese government is encouraging the development of an in-country robotics industry because, currently, three-quarters of all robots purchased are from off-shore companies (albeit some with manufacturing and assembly facilities in China).

We have been able to identify 47 Chinese companies and five startups that call themselves industrial robot makers, and another 26 which offer to provide integration services and speciality equipment. 34 more provide ancillary services, sensors and components to the robot makers. Overall, 107 Chinese companies are involved in industrial robotics. Many of these new industrial robot makers are producing products that, because of quality, safety and design regulations, will only be acceptable to the Chinese market.

In addition to growth in Chinese industrial robotics, progress in the service sector is also healthy. A shining example of the 48 mostly new service robotic companies on the map is Shenzhen DJI Innovations, of quadcopter fame. DJI is projected to sell devices worth $1bn in 2015.

Two other notable growth areas in the service sector are materials handling and the production of unmanned air, sea and ground devices for China’s developing space, military and defense industries.

The consumer sector is also growing: robotic vacuums and personal/social robots are examples. Last “Singles Day,” a brand new shopping holiday in China, Alibaba sold over 70,000 Ecovacs robotic vacuum cleaners!

The drivers propelling this boom are many and varied:

• Government mandates and political encouragement to develop the industry domestically and, ultimately, to export products
• Low-cost loans and factory incentives
• Tax credits for robot users and also to robot makers that employ local workers
• Engineering talent spewing out from Chinese technical universities and research facilities
• Rising wages
• A diminishing factory-age work force
• A rapidly aging work force, hampered by the one-child policy of past regimes
• The high cost of training and housing for a short-term transient work force
• Ramping up of auto makers to satisfy domestic demand and to improve quality and produce Chinese cars for export — for China to have competitive cars in the international marketplace (not just price-competitive), they have to improve quality, consistency and technology — and robotics is enabling them to do so
• Serious investment in automation and robotics from companies, such as Foxconn and Apple, that manufacture their goods in China
• A growing middle and upper class with disposible income for high-tech devices

The Robot Report and the research team at Robo-STOX have been able to identify 194 companies that make or are directly involved in making robots in China. The CRIA and other sources, believe the number to be closer to 400.

The Robot Report’s Global Map is limited by our own research capabilities, language translation limitations, and scarcity of information about robotics companies and their websites and contact people. We could use your help in adding to the map. Please send companies that we have missed (or are new) to info@therobotreport.com.

You can explore the map online and filter it in a variety of ways. Use Google’s directional and +/- markers on the left of the map to navigate, enlarge, and hone in on an area of interest (or double click near where you want to enlarge). Click on one of the colored markers to get a pop-up window with the name, type, focus, location and a link to the company’s website.

[Note: the map shows a single entry for the company headquarters regardless how many branches, subsidiaries and factory locations that company might have.]

The Filter pull-down menu lets you choose any one of the seven major categories: (1) Industrial robot makers; (2) Service robots used by corporations and governments; (3) Service robots for personal and private use; (4) Integrators; (5) Robotic start-up companies; (6) Universities and research labs with special emphasis on robotics; and (7) Ancillary businesses providing engineering, software, components, sensors and other services to the industry.

As can be seen from the map at the top of the page, most all of the companies and educational facilities are located on the East Coast or in Central China; very few are located in the Northeast or in Western China.

The research firms forecasting dramatic growth for the domestic Chinese robotics industry are also predicting very low-cost devices. Their reports are contradicted by academics, roboticists and others who point out that there are so many new robot manufacturing companies in China that none will be able to manufacture thousands of robots per year and thus benefit from scale. Further, many of the components that comprise a robot are intricate and costly, e.g., reduction gears from Nabtesco, Harmonic Drive and Sumitomo (all Japanese companies). Although a few of the startups are attempting to make reduction gears and other similar devices, the lack of these component manufacturers in China may put a cap on how low costs can go and on how much can be done in-country for the time being. Another negative is that all but auto manufacturers have short product cycles requiring any robots on the line to be extremely flexible. Worker flexibility is still crucial in Chinese manufacturing.

In the chart below, the 194 Chinese companies are tabulated by their business type and area of focus.

Again, we request your help adding to the map and making it as accurate and up-to-date as possible. Please send robotics-related companies that we have missed (or new ones) to info@therobotreport.com.

FSR, the biannual single track meeting on field and service robotics, is on now in Toronto. With 42 papers on a range of applications and keynotes by Chris Urmson (Google[x]), Paul Newman (Oxford), Sanjiv Singh (CMU), and Ryan Gariepy (Clearpath Robotics), lots of ground will be covered. Robohub’s Hallie Siegel is on site at the event – check out the live tweets below.

#FSRToronto Tweets

Previously known for consumer products for smartphones, tablets and cars, Paris-based Parrot recently branched into both the consumer and commercial drone businesses.

Parrot created the AR.Drone quadcopter and revealed it at CES 2010 in Las Vegas. Since then they have sold over a million of them – 700M in 2014 alone! In 2014, drones generated 34% of Parrot’s total revenue. At CES 2014 they launched two new mini drones for the retail consumer market. All appear to have done quite well during the Christmas season.

But what is particularly interesting is Parrots growth In the commercial drone sector. Parrot is acquiring companies and developing products to provide drones, software and data solutions for the agriculture, mapping and surveillance industries. To jumpstart their movement toward commercial drones, two spinoffs from the Swiss Federal Institute of Technology (EPF), senseFly and Pix4D, were invested in in 2012.

• senseFly designs and develops a line of winged drones and is soon to release a quadcopter as well. All their drones are for commercial use. senseFly’s $25K eBee drone was rated one of the Top 7 Drones for Agriculture in 2014 by DroneLife.
• Pix4D provides image processing software for mapping and modeling geo-referenced 2D images into 3D models and for a wide range of GIS and CAD applications.

Two more acquisitions in B2B (business-to-business) drones happened in 2014: Parrot invested in MicaSense and Airinov.

• MicaSense is a Seattle-based software and systems company providing advanced data gathering and processing, coupled with professional analysis tools, to provide accurate and repeatable information on the status of a crop. Raw data from the MicaSense camera, a lightweight, multispectral camera, is transformed into vegetation index maps. Powerful analytics provide time-based trends and change maps. Analysis tools such as plant population counts enable optimized farm management. Parrot invested $2M in a Series A round of funding in November, 2014.
• Airinov is a French provider of UAV farming applications and sensors for use on senseFly and other UAS. Their software can process flight and sensor data and produce fertilization recommendations which are compatible with most brands of dispenser equipment. Parrot invested $2.2M for a 20.9% equity interest in Airinov.

Henri Seydoux, Parrot’s founder, chairman and CEO, said:

We are moving forward with our external growth policy initiated in 2011 and focusing on new products with high-potential, outstanding and complementary technological expertise, applications for commercial and retail customer segments, strong operational and financial synergies. Parrot intends to meet the needs of professionals moving into the civil drone age and firmly believes in the commercial potential of this market, (on which the Parrot AR.Drone has already enabled us to gain global recognition) and I am very pleased that we can also serve the commercial drone market as well.

Agriculture is one of the fastest-growing market segments for unmanned aircraft commercial applications. The sensors required to capture accurate data are a critical part of the solution, and [our recent acquisition of MicaSense and Airinov] brings this technology to the table.

As much fun as Parrot’s drones are, there is a growing market for professional service drones for mapping, surveying, protection, real estate photography and agricultural uses. Shenzen-based Dajiang Innovation Technology (DJI) has sold over 400M of their line of consumer and B2B drones; San Francisco startup Skycatch has partnered with one of the largest heavy machinery makers, Komatsu, to automate construction and mining job sites world wide using drones, mapping software and various sensors.

• There is also a steadily growing defense and first-responder market led by Lockheed Martin, Boeing, AeroVironment, Elbit and Northrup Grumman. The Global UAV Market 2015-2025 projects the global defense UAV market to grow at a 5.66% CAGR from 2014 to 2025 with North America and Europe the largest markets and the military drone segment (UCAV – unmanned combat aerial vehicle) to dominate the UAV market.
• But the real good news is Wintergreen Research’s report entitled “Commercial Drones: Highways in the Sky, UAS, Market Shares, Strategies and Forecasts, Worldwide, 2015 to 2021” (whew!) which projects commercial drone growth at 34.3% CAGR over the period from 2014 to 2021. That’s game-changing growth!

By Ryan Gariepy 

For our recent “hack week” we teamed up with one of the most innovative visual technology companies in the world, Christie, to make a 3D video game with robots. How did it all come together?

As seems to be the norm, let’s start with the awesome:

Inspiration

In late October, a video from MIT was making the rounds around at Clearpath. Since we usually have a lot of robots on hand and like doing interesting demos, we reached out to Shayegan to see if we could get some background information. He provided some insights into the challenges he faced when implementing this, and I was convinced that our team could pull a similar demo together so we could see how it worked in person.

Here’s what MIT’s setup looked like:

(Video: Melanie Gonick, MIT News. Additional footage and computer animations: Shayegan Omidshafiei)

At the most fundamental, this demo needs:

• A small fleet of robots
• A way for those robots to know where they are in the world
• A computer to bring all of their information together and render a scene
• A method of projecting the scene in a way which is aligned with the same world the robots use

In MIT’s case, they have a combination of iRobot Creates and quadrotors as their robot fleet, a VICON motion capture system for determining robot location, a Windows computer running ROS in a virtual machine as well as projector edge-blending software, and a set of 6 projectors for the lab.

There were three things we wanted to improve on for our demo. First, we wanted to run all-ROS, and use RViz for visualizations. That removes the performance hit from running RViz in a VM, and also means that any visualization plugins we came up with could be used anywhere Clearpath uses ROS. Second, we wanted to avoid using the VICON system. Though we have a VICON system on hand in our lab and are big fans, we were already using it for some long-term navigation characterization at the time, so it wasn’t available to us. Finally, we wanted to make the demo more interactive.

Improvement #1: All-ROS, all the time!

To get this taken care of, we needed a way to either run edge-blending software on Linux, or to use projectors that did the edge blending themselves. Fortunately, something that might be a little known fact to our regular audience is that Christie is about 10 minutes away from Clearpath HQ and they make some of the best digital projectors in the world that, yes, do edge blending and more. A few emails back and forth, and they were in!

For this project, Christie arrived with four Christie HD14K-M 14,000 lumens DLP® projectors and two cameras. The projectors use Christie AutoCal software and have Christie Twist software embedded right in. Christie rigged the four projectors in a 2 x 2 configuration on the ceiling of our warehouse. The cameras captured what was happening on the floor and sent that information on the Christie AutoCal software, which then automatically aligned and blended the four projectors into one giant seamless 30 foot projection mapped digital campus.

Christie sets up the 3D Projection system. (Photo courtesy of Christie.)

Improvement #2: No motion capture

Getting rid of the motion capture system was even easier. We already have localization and mapping software for our robots and the Jackals we had on hand already had LIDARs mounted. It was a relatively simple matter to map out the world we’d operate in and share the map between the two robots.

Now, I will make a slight aside here… Multi-robot operation in ROS is still not yet what one would call smooth. There are a few good solutions, but not one that is clearly “the one” to use. Since all of our work here had to fit into a week, we took the quick way out. We configured all of the robots to talk to a single ROS Master running on the computer connected to the computers, and used namespaces to ensure the data for each robot stayed tied to each robot. The resulting architecture was as follows:

All we had to do to sync the robot position and the projector position was to start training the map from a marked (0,0,0) point on the floor.

Improvement #3: More interactivity

This was the fun part. We had two robots and everyone loves video games, so we wrote a new package that uses Python (via rospy), GDAL, and Shapely to create a real-life PvP game with our Jackals. Each Jackal was controlled by a person and had the usual features we all expect from video games – weapons, recharging shields, hitpoints, and sound effects. All of the data was rendered and projected in real-time along with our robots’ understanding of their environment.

And, as a final bonus, we used our existing path planning code to create a entire “AI” for the robots. Since the robots already know where they are and how to plan paths, this part was done in literally minutes.

The real question

How do I get one for myself?

Robots: Obviously, we sell these. I’d personally like to see this redone with Grizzlies.

Projectors: I’m sure there are open-source options or other prosumer options similar to how MIT did it, but if you want it done really well, Christie will be happy to help.

Software: There is an experimental RViz branch here which enables four extra output windows from RViz.

The majority of the on-robot software is either standard with the Jackal or is slightly modified to accommodate the multi-robot situation (and can also be found at our Jackal github repository). We intend on contributing our RViz plugins back, but they too are a little messy. Fortunately, there’s a good general tutorial here on creating new plugins.

The game itself is very messy code, so we’re still keeping it hidden for now. Sorry (sad)
If you’re a large school or a research group, please get in touch directly and we’ll see how we can help.

Happy gaming!

Sky-Futures, a UK-based company offering drone inspection services to the global oil and gas industry, through their Houston office, was awarded an FAA exemption allowing their drones to fly in US air space.

The exemption allows Sky-Futures to perform infrastructure inspections of oil and gas facilities on and off shore. The drones collect HD video, stills and thermal imagery of live flare, structural and under deck platforms which are then analyzed by flare systems and structural engineers.

This type of flying visual inspection gathers digital data enabling oil and gas operators to make informed decisions with respect to asset conditions under deck, from exhausts, flare towers, and other hard to access structural areas.

In one example, the use of a drone team for this type of inspection saved over 90% of the costs of traditional rope access inspection work – work that would have required a shutdown during the inspection and the hiring of a support vessel, plus a very dangerous job for the human inspectors.

In this episode, Audrow Nash speaks with Professor John Leonard from MIT about his research on dense, object-based 3D Simultaneous Localization And Mapping (SLAM).

Leonard explains what SLAM is, as well as its practical applications. The explanations include what it means for SLAM to be object-based (versus feature-based) and to have dense (versus sparse) environmental mapping. The interview closes with advice for aspiring roboticists.

John Leonard
John J. Leonard is Professor of Mechanical and Ocean Engineering and Associate Department Head for Research in the MIT Department of Mechanical Engineering. He is also a member of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). His research addresses the problems of navigation and mapping for autonomous mobile robots. He holds the degrees of B.S.E.E. in Electrical Engineering and Science from the University of Pennsylvania (1987) and D.Phil. in Engineering Science from the University of Oxford (1994). He studied at Oxford under a Thouron Fellowship and Research Assistantship funded by the ESPRIT program of the European Community. Prof. Leonard joined the MIT faculty in 1996, after five years as a Post-Doctoral Fellow and Research Scientist in the MIT Sea Grant Autonomous Underwater Vehicle (AUV) Laboratory. He has served an associate editor of the IEEE Journal of Oceanic Engineering and of the IEEE Transactions on Robotics and Automation. He was team leader for MIT’s DARPA Urban Challenge team, which was one of eleven teams to qualify for the Urban Challenge final event and one of six teams to complete the race. He is the recipient of an NSF Career Award (1998), an E.T.S. Walton Visitor Award from Science Foundation Ireland (2004), the King-Sun Fu Memorial Best Transactions on Robotics Paper Award (2006), and he is an IEEE Fellow (2014).

Links:

• Download mp3 (16.3MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• John J. Leonard’s website

The OpenROV submersible is a low cost and open source kit designed for exploration and education. Originally launched on Kickstarter in 2012 it has now grown into a dedicated global community launching missions everywhere from Hawaii to the UK. The project raised $111,662 from 484 backers and has now reached version 2.7. Today the kit costs $899 with a fully assembled version coming in at $1,450 which you can purchase from the online store. Units weigh in at 2.6kg and normal battery life is expected to last between 2-3 hours depending on use.

Features include:

• Live HD video with wide-angle lens and tilt function
• LED lighting for low-light environments
• Easy-to-use web browser user interface
• Beaglebone Black AND Arduino MEGA microprocessors for a flexible and powerful developer platform with dozens of input/output channels and plenty of computing power for user-designed features and experiments
• Rechargeable lithium batteries (sold separately)
• 100-meter lightweight 2-wire tether
• Compact and lightweight–backpack or carry-on ready
• Payload area for additional hardware or equipment
• Join our open source, ocean-loving community–find all the software and hardware files online as well as a vibrant and enthusiastic community to share adventures, ask questions, and meet other people with passion for exploration

With this piece of kit and your laptop you really can start exploring the oceans, lakes and rivers around you without getting your hair wet. The maker community around 3d printers, drones and robotics is the step we usually see just before the commercial applications – so watch this space.

To find out more on what’s going on with the OpenROV take a look at the community website called Open Explorer.

Here is a great video of a trial run with Nasa’s Astronaut underwater training facility NEEMO-16 back in 2012.

Waveglider riding rough seas. Source: National Oceanography Centre (NOC).

The second phase of an ambitious project to gather valuable information on ocean processes and marine life using a fleet of innovative marine robots has just reached its conclusion. Co-ordinated by the National Oceanography Centre (NOC), the Exploring Ocean Fronts project took place off southwest England and saw the largest deployment of robotic vehicles ever attempted in UK waters . 

The marine robots, which are powered by a combination of wave, wind and solar power, are controlled by satellite communications and can cover hundreds of kilometres in a single mission.

Tagged Plaice. Source: National Oceanography Centre (NOC).

In the latest phase of the project, three unmanned surface vehicles were used to track fish carrying acoustic ‘pingers’ off the Devon coast. About 85 fish, including rays, sole and plaice, were tagged and released by scientists from the Marine Biological Association (MBA), with the aim of understanding how these fish use Marine Protected Areas. The roaming robotic vehicles carried acoustic receivers and worked alongside a series of fixed receivers on the seabed in order to track fish movements inside and outside of the protected sites.

Commenting on the fish-tracking trial, Professor David Sims of the MBA said: “The patrolling robots successfully located tagged fish, and also tracked the movements of individual fish over several days by re-locating them. This demonstrates the potential of ocean robots for monitoring dynamic changes in distributions of commercially important fish, which will underpin effective management and understanding of climate change impacts”.

The surface vehicles were also carrying camera systems that successfully imaged a variety of marine wildlife, including porpoises and gannets. It is the first time images of marine life have been captured by NOC’s robotic vehicles operating far from land and demonstrates huge potential for surveying parts of the ocean that are rarely visited by research ships.

Phase one of the project saw the robot fleet head up to 150 kilometres offshore from the Isles of Scilly over a two-week period. Despite having to cope with a series of autumn storms, the fleet continued to collect high-quality data that have provided valuable insights into the effect that stormy weather has on ocean processes and marine life. Head of NOC’s Marine Autonomous Robotics Systems group, Dr Maaten Furlong said: “These missions have demonstrated the ability of these new vehicles to travel hundreds of kilometres in very demanding conditions. Although some of the vehicles were impacted by the bad weather, it provided us with the opportunity to gain experience of piloting in harsh conditions”.

The pioneering trials were co-ordinated by the National Oceanography Centre, in partnership with 20 other organisations representing the marine robotics industry, research organisations, and marine data users such as the UK Met Office and the Royal Navy. Dr Russell Wynn of NOC, who was the scientific co-ordinator of the project, said: “Overall the project has been hugely successful and we have collected a huge amount of valuable scientific and engineering data. By working in partnership and sharing our resources and expertise we have developed a strong UK community in marine robotic operations, and we are already planning further joint missions in the near future”.

AutoNaut, C-Enduro and the SV3 Waveglider being launched as part of phase two of the Exploring Ocean Fronts project. Source: National Oceanography Centre (NOC).

If you liked this article, you may also be interested in:

• PacX Challenge winner shows large scale ocean mapping abilities
• Liquid Robotics continues its growth by partnering with Boeing
• Drones against illegal fishing
• Underwater ROV aids in investigation, searches for victims in South Korean ferry tragedy
• Liquid robotics gets $45 million to continue double-digit growth
• 3’600-strong underwater robot sensor network returns 1’000’000th measurement, has scientists craving for more
•  Swimming robots
• Wave Glider transmits real time weather data during Hurricane Sandy

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Khurana is a M.S. student in Computer Science from Lehigh University.

References:

www.gazebosim.org

Repositories:

https://bitbucket.org/osrf/gazebo

https://bitbucket.org/tkhurana/gazebo

https://bitbucket.org/tkhurana/sdformat

The Waveglider receives last minute checks before being deployed. Source: British National Oceanography Centre.

Collecting oceanographic data is usually accomplished by a combination of satellites, buoys and research ships. All those methods have their limitations (e.g. buoys are only gathering info from a fixed location) and can be expensive. That is especially the case with large, crewed research vessels. One solution to overcome these constraints is the use of marine drones, autonomous surface or underwater vehicles capable of traversing great distances and operating independently for prolonged periods of time. The latest project to take advantage of the benefits of marine drones – the Robotic Exploration of Ocean Fronts by the British National Oceanographic Centre (NOC) – is one of largest and most ambitious unmanned ocean exploration missions to date.

At the beginning of October, a fleet of seven marine drones were launched from the Isles of Scilly for a three-week journey across the ocean. The target area for exploration is an area of ocean marking the boundary between Atlantic waters and English Channel. These boundaries are called fronts, and they separate water masses of different properties. The targeted fronts off southwest UK provide ideal conditions for abundant plankton growth, which in turn can attract large numbers of fish, seabirds, dolphins and basking sharks.

The vehicles carry a range of instruments for observing these animals. As well as measuring the temperature of the water and the weather conditions at the ocean surface, they can detect the density of plankton in the water, listen for clicks and whistles of dolphins and porpoises, and image seabirds using surface cameras. This data will help scientists map the distribution of the fronts and their associated fauna.

There are four types of marine drones used in this research mission:

Autonaut

Autonaut is a solar- and methanol fuel cell-powered autonomous surface vessel. It was developed by MOST (Autonomous Vessels) Ltd, and is designed to gather oceanographic data over long periods of time.

C-Enduro

C-Enduro

C-Enduro is an innovative long-endurance autonomous surface vehicle built by Autonomous Surface Vehicles (ASV). The vehicle was officially launched in March 2014. Built to operate in all marine environments, C-Enduro uses solar panels, a wind generator, a lightweight diesel generator and a self-righting hull to enable it to remain at sea for long periods.

Wave Glider

Wave Glider

The Wave Glider is an autonomous, environmentally-friendly ocean-going vehicle, developed by Liquid Robotics. It is fitted with surface and sub-surface sensors to gather information such as water temperature and the atmospheric conditions above. It is roughly the size of a surf board and is equipped with computers for navigation and a satellite communication system. Three Wave Gliders have been deployed for this project.

Slocum Glider

Slocum Glider

The Slocum Glider, built by Teledyne Webb Research, is a high-endurance vehicle equipped with a variety of sensors. It can operate at depths of 1000 metres and can remain at sea for months at a time while transmitting data back to shore. Two gliders are used in the NOC project, and have been customized in order to cope with the demanding marine environment and to record different type of data.

The Robotic Exploration of Ocean Fronts project is divided into two phases. Phase one sees a fleet of seven marine drones being launched from a beach on the Isles of Scilly to travel up to 300 miles to the shelf edge and back. During this time, they will be taking measurements including chlorophyll, temperature and salinity, and also monitoring wildlife activity at the front. Phase two of the project sees further monitoring plus the tagging and release of 100 fish, which will be tracked using a new seabed and USV receiver.

Further information about the project available at the project’s official page.

Images and video courtesy of MOST Autonomous Vessels Ltd, Autonomous Surface Vehicles (ASV) and National Oceanographic Centre.

Source: Peter Corke

In this episode, Audrow Nash interviews Peter Corke from Queensland University of Technology, about computer vision – the subject of his plenary talk at IROS 2014 (link to slides below). He begins with a brief history of biological vision before discussing some early and more modern implementations of computer vision. Corke also talks about resources for those interested in learning computer vision, including his book, Robotic Vision & Control, and a massively open online course (MOOC) that he plans to release in 2015. 

Peter Corke
Peter Corke joined Queensland University of Technology at the start of 2010 as a Professor of Robotic Vision. Now he’s also director of the ARC funded Centre of Excellence for Robotic Vision. Peter is known for his research in vision-based robot control, field robotics and wireless sensor networks. He received a B.Eng and M.Eng.Sc. degrees, both in Electrical Engineering, and a PhD in Mechanical and Manufacturing Engineering, all from the University of Melbourne, Australia.

Links:

• Download mp3 (19.4MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Peter Corke’s Toolbox Website
• Peter Corke’s CyPhy Lab Wiki
• IROS 2014 Plenary Slides: Computer Vision (PDF; includes book recommendations)

SV-ROS team “Maxed-Out” won the IROS 2014 Microsoft Kinect Challenge

One week before the IROS 2014 conference, I received an email from the Maxed-Out team telling me that they were using my open source code RTAB-Map for the Kinect Robot Navigation Contest that would be held during the conference. I am always glad to know when someone is using what I have done, so I said I would answer whatever questions they had before the contest. Since I was already in Chicago to present a paper, I met them in person at the conference and helped them with the algorithm on site before the contest … and good surprise, they won the contest! See their official press release here.

This contest featured robots (equipped only with a Kinect) navigating autonomously across multiple waypoints in a typical office environment. The first part of the contest was to map the environment so that the robot could plan trajectories in it. The robot was driven through the environment to incrementally create a 2D map (like a top view map of an office), using only its sensors. Sensors are not perfect, so maps can become misaligned over time. These errors can be minimized when the robot detects when it comes back to a previously visited area, a process called loop closure detection. This mapping approach is known as Simultaneous Localization And Mapping (SLAM). SLAM is used when the robot doesn’t have a map already built of the environment, and when it doesn’t have access to external global localization sensors (like a GPS) to localize itself. The mapping solution chosen by the SV-ROS team was my SLAM approach called RTAB-Map (Real-Time Appearance-Based Mapping).

In this article I share some of the technical details of the RTAB-Map.

The robot

The robot was an Adept MobileRobots Pioneer 3 DX equipped with a Kinect sensor. The up-facing camera was only used by the scoring computer (detecting patterns on the ceiling to localize the robot). The RTAB-Map ROS setup was based on the “Kinect + Odometry + Fake 2D laser from Kinect” configuration. The “2D laser” was simulated using the Kinect and the depthimage_to_laserscan ros-pkg. In this configuration, the robot is constrained to a plane (x,y and yaw).

Source: IROS 2014

Contest environment

The challenge was held in the exhibit room in the Palmer House Hotel. Since RTAB-Map’s loop closure detection is based on visual appearance of the environment, the repetitive texture on the carpet made it very challenging. When there were only discriminative visual features on the floor (like a long hall), loop closures could be found with images from different locations, thus causing big errors in the map. To avoid this problem, just two minutes before the official mapping run, we set a parameter to ignore 30% of the bottom image for features extraction. We didn’t have the chance to test it before the official mapping run – all we could do was to cross our fingers and hope that no wrong loop closures would be found!

Contest environment

Filtering visual features of the floor

Results

After mapping, no wrong loop closures were found. However, the robot missed a loop closure on the bottom of the map (because the camera was not facing the same direction), causing a wall at the end of the hall. This was one case where, when exploring an unknown environment, it would be better to occasionally do a 360° turn. This would enable the system to see more images at different angles, and would result in more loop closures. However, by using rtabmap-databaseViewer and reprocessing data included in the database, we could detect a loop closure between locations before and after exploring the hall. The result was an aligned bottom corner. There was still a misalignment on the middle-top, but given the time we had, the map was good enough for the navigation part of the contest. Below are results with these remaining errors corrected. The loop closures are shown in red.

a) After first mapping; b) Used for navigation; c) Updated results (after the challenge)

During the autonomous navigation phase, RTAB-Map was set in localization mode with the pre-built map. By detecting “loop closures” with the previous map, the robot is relocalized so it can efficiently plan its trajectories through the environment.

Updated Results

In this section, I will show a parametrization of RTAB-Map that I did after the challenge to automatically find more loop closures. The video above can be reproduced using the data here:

• RTAB-Map database: IROS14-kinect-challenge.db
• RTAB-Map parameters: IROS14-kinect-challenge.ini

You can open RTAB-Map (standalone, not the ros-pkg) using the configuration file above, then set “Source from database” and select the database above. Set Source rate to 2 Hz and press start; this should reproduce the video above. Here the principal parameters modified from the defaults:

// -Filter 30% bottom of the image
// -Maximum depth of the features is 4 meters
Kp/RoiRatios=0 0 0 0.3
Kp/MaxDepth=4

// -Loop closure constraint, computed visually
LccBow/Force2D=true
LccBow/InlierDistance=0.05
LccBow/MaxDepth=4
LccBow/MinInliers=3

// -Refining visual loop closure constraint using 
//   2D icp on laser scans
LccIcp/Type=2
LccIcp2/CorrespondenceRatio=0.5
LccIcp2/VoxelSize=0

// -Disabled local loop closure detection based on time.
// -Optimize graph from the graph beginning, to 
//   have same referential when relocalizing to this 
//   map in localization mode.
RGBD/LocalLoopDetectionTime=false
RGBD/OptimizeFromGraphEnd=false

// -Extract more SURF features
SURF/HessianThreshold=60

// Run this for a description of the parameters:
$ rosrun rtabmap rtabmap --params

The following pictures show a comparison of the results with and without loop closures:

	Odometry Only	With Loop Closures
2D Map
3D Cloud
2D Map + 3D Cloud

3D Cloud (download here: cloud.ply)

3D Cloud + 2D Map (saved with map_server: map.pgm and map.yaml)

Conclusion

SV-ROS team has created a great solution for autonomous navigation and they won the IROS 2014 Kinect Navigation Contest. I’m glad that my algorithm RTAB-Map was part of it. It was really nice to meet the team – they were very motivated, enthusiastic and all their efforts were rewarded. This post has provided a technical description of the mapping approach used by the team and shows some promising results using visual features for loop closure detection on a robot equipped only with a Kinect.

Sunnyvale-based Liquid Robotics continues their growth-by-partnering formula by striking another collaboration, this time with Boeing.

Boeing Defense, Space & Security CEO Chris Chadwick said, “It’s a great opportunity to partner with Liquid Robotics to provide new and existing customers a unique portfolio of defense solutions and capabilities and allows the Boeing-Liquid Robotics team to solve maritime security and surveillance challenges in entirely new and highly effective ways.”

From an inquisitive mind trying to record the songs of the Humpback Whales off the coast of Hawaii, to a company employing more than 100 and with over 300 wave gliders involved in the activities described below, Licquid Robotics has grown from a single experimental wave glider to six different versions (distinguished by different features and sensors) of their third iteration of glider. Uses include:

• Defense (anti-submarine warfare, mine countermeasures, marine domain awareness)
• Private security (fixed asset protection, monitoring marine protected areas)
• Oil and gas (seismic surveying, hyperlocal METOC forecasting, directional drilling monitoring, hydrocarbon detection)
• Commercial (site assessment and monitoring, fisheries stock assessment, marine mammal monitoring)
• Science (environmental monitoring, global weather, tsunami early warning, CO2 monitoring)

Partnering has played an important role in Liquid Robotics’ growth. Oil and gas industry activities are handled by a joint venture with Schlumberger, a large provider of technology, project management and information solutions to the oil and gas industry. Defense applications will now be channeled through their new collaboration with Boeing. NOAA is their science and research partner and Ultra Electronics USSI and a couple of Teledyne subsidiaries are amongst their commercial partners.

Swarm communication and decision-making are next on the drawing board for this fast-growing company and inventor of the Wave Glider.

Aerial mapping can be quite time-consuming and expensive. Even if you are employing a UAV to do it, it might take hours for all the post-processing, time-stamp shifting and deleting blurred photos. And after all the time invested, it is quite possible that the map created is not up to expectations, and the whole process starts almost from scratch.

That is what Maps Made Easy, a Kickstarter project, is trying to improve. Maps Made Easy is a brainchild of a San Diego based Drones Made Easy, a DJI dealership with a twist, because behind it stands a group of engineers with aerial photography, photogrammetry and commercial drone mapping systems experience.

Aerial mapping process workflow with Maps Made Easy should work something like this:

1. Plan – Use the online planning tool or ground station software of your choice to select the area to be mapped
2. Collect – Program the aerial platform. Set the camera to take periodic images. Kick off the automated flight. Wait for the drone to land itself
3. Upload – Drag and drop the collected images into their web application. Upload completion automatically kicks off the processing
4. Interact – View the created maps on the Maps Made Easy site. Pan and zoom around using a familiar interface. Control public accessibility. Embed freely.

Image processing can tie up a personal computer for hours, if not days, unless you have a US$15.000,00 workstation that can spit it up in minutes. By offloading the job to the Maps Made Easy image-processing service, you can free your computer and just wait for the email with your hosted and stitched aerial image. Full geo-referencing can be done with just a few clicks.

If you have any doubt, just check what an experienced aerial mapper said about the service: “177 randomly named photos, without GPS data, without flight logs, were loaded onto alpha version of the server. I thought they don’t have a chance, but they nailed it. Just few hours later, an e-mail was waiting in inbox, with a URL for a finished map“

The Maps Made Easy service is the future of low-cost aerial mapping. It is able to provide up-to-date maps out of stitched imagery, that are accurate enough even for GIS professionals.

And if you are having a second thought on funding this Kickstarter, let dronologista give you some simple advice: back it!

Videos courtesy of Drones Made Easy.

A brilliant article, first hand experience, and an interview with CTO of Drones Made Easy can be found on DIYDrones.

Search and rescue operations are increasingly using drones, and the speed and ease with which UAVs can be launched is especially important when applying them as an emergency response tool. Jonathan Cheseaux, a masters student from Switzerland has developed a system to help drones home in on the location of people in need of rescue, and Dronologista had an opportunity to discuss the project with Jonathan via email. Here is what he says:

Dronologista: How did you come up with the idea?

Jonathan: The main goal of the project was to provide WiFi connection to people on the ground (in disaster areas, where there is no more connectivity), using swarms of flying robots. By analyzing the data collected by the WiFi antenna embedded in the plane, we thought about using this information to localize people and moving the plane around so that the connection is better.

Dronologista:  Have you been working on it alone or it was a team effort?

Jonathan:  A team effort. I was working with two people, Stefano Rosati, a postdoc, and Karol Kruzelecki, a graduate student. They helped me a lot to fulfill the project’s goals.

Dronologista:  How does this system work?

Jonathan:  The UAV sniff WiFi packets periodically broadcast by smartphone (if WiFi is switched on). The smartphone broadcast so-called “probe requests” to discover which WiFis are in range. We extract the signal power of these packets (RSSI) and use the GPS position of the plane to localize the users.

Dronologista:  Has it been tested, and if yes, how?

Jonathan: We flew the UAV around EPFL‘s campus and tried (and succeeded) to find the position of a smartphone randomly placed somewhere on a building with a satisfying precision.

Dronologista: What are potential problems?

Jonathan: If the WiFi of the smartphone is not switched on, we cannot localize the user. Also the signal can get scrambled by a lot of factors, such as buildings reflecting the signal, weather conditions, plane’s orientation, etc. In the case of an earthquake for example, if the victim is buried under several meters of concrete, the WiFi signal will not be detectable. Also Apple announced that they will randomize the MAC address while sending probe requests in iOS 8, and this will make the localization quite impossible because now we rely on the fact that a MAC address is unique for a user, and can thus fully determine which person we are localizing.

http://youtu.be/7ysg-m7kV14

Dronologista: Were you contacted by any organization yet?

Jonathan: I’ve been invited to the UAV Show 2014 (London), they want me to give a small speech about my project during the conference. I’ve also been contacted by several companies that want to commercialize search and rescue drones.

Dronologista: What are future steps for this system?

Jonathan: Enhance precision by conducting more real life testing, maybe use GSM data as well (IMSI catcher), but due to legal concern it could be difficult to implement. The real improvement will be to use avalanche transceivers to find avalanche victims. I really want to work on that after my internship.

Dronologista: What are other projects you are working on, that you want to share?

Jonathan: During my bachelor studies, I implemented a virtual reality project using Kinect and some physics engine. The goal was to project the user in a virtual environment and allow him to interact with [the environment] (shoot a virtual ball, smash a wall), as well as draw interactive shapes with the only use of hand gestures (drawn shapes we bound to the physics engine so that it also falls/bounces etc.). Last semester, I lead a project on Bitcoin trading. We’ve implemented sentiment analysis on tweets talking about bitcoin to probe the market’s mood and used time-series analysis to work directly on the bitcoin stock market. That was a really cool project. I’m going to start my master thesis soon, in which I will develop a smart search engine that will automatically classify documents, display augmented information, context, etc. I’m not sure yet what the project will be exactly, but it is definitely about machine learning.

***

Dronologista wishes Jonathan and his team great success with their projects, and expects to hear more good news about their endeavors soon. And so should you.

For further reading check EPFL news, SkyNews, and the Swarming Micro Air Vehicle Network (SMAVNET II) project page.

With industry losses worth US$23 billion per year, illegal fishing represents a major global problem. One out of three bluefin tunas are caught illegally, as are around 20% of all fish hauled around the world. Even the legal fishing industry, with its compliance to maritime wildlife protection standards, has the capacity to damage fragile maritime ecosystems. Illegal fishing can completely destroy them.

Efforts are being made on a global scale to tackle this problem, however the challenge has persisted due to the substantial number of time, people, boats and aircraft typically required to patrol large stretches of coastline. But unmanned aircraft patrols are proving to be a viable solution.

One of the countries hardest hit by illegal fishing is Belize, a small coastal country in Central America. It was so rampant that in March of this year the European Union suspended all seafood imports from Belize (as well as from Cambodia and Guinea, for the same reason).

Help comes from ConservationDrones.org, a non-profit organization dedicated to building and using UAVs for conservation-related applications, and sharing their knowledge with conservation workers and researchers worldwide, especially those in developing countries. Belize Fisheries Department officials are now being trained to use drones to monitor fishing areas, following a test phase that started in July 2013.

The fixed wing drones that are being used for this project can fly for over an hour, have a range of 50km and are capable of capturing high-definition photos and videos. They will be used to patrol difficult to reach areas, such as coastal mangrove forests, at a fraction of the cost of a conventional, manned aircraft. Once the illegal activity is located, authorities can dispatch a vessel and perform a seagoing search much more efficiently.

It will be interesting to see the effects of these new tactics against “pirate” fishing practice, and how soon Belize will begin to reap the benefits – specifically the lifting of the EU’s seafood import ban.

On a side note, drones provided by ConservationDrones.org should provide a much-needed, low-cost solution for the protection of one of the world’s most famous coral reefs, Glover’s Reef.

For further reading, see this post by the Founding Director of Conservation Drones Lian Pin Koh, check out his TED Talk, or see these articles on Gizmag, National Geographic and the New York Times.

Video courtesy of ConservationDrones.org

http://vimeo.com/97155951

But if a coach uses aerial filming platform to analyze his own team performance, what stops him from using a drone to analyze his opponents’ performance?

French Coach calls for investigation

Apparently, this exact scenario occurred last Tuesday during a closed training session for the French national team. French coach Didier Deschamps is reportedly calling for an investigation into a quadcopter drone that was spotted spying over his team’s closed training session, and local police are promising a quick resolution.

Closed training sessions are used to check the physical fitness of key players and to decide on gametime tactics and player selection. If rival managers acquire that information, it would be a distinctive tactical advantage during the actual game. Taking into consideration the amount of money that is involved in every aspect of the FIFA World Cup ($14 billion cost to Brazilian Government, ~$500 million is the prize money pot), it is no wonder that the French coach is upset about the drone with camera flying over his team.

Using drones for reconnaissance has a long military history, however, this is the first time to our knowledge that a drone has been used to acquire intel on a football field. Dronologista thinks that the Pandora’s box of drone-enabled sport espionage is officially open, and we can expect more events like this to happen.

Interesting article about the case you can find on Motherboard and Yahoo! sport. Image and video courtesy of 101 Great Goals and ESPN.

If you liked this article, you may also be interested in:

• Packbot: Serving the Military and World Cup Football
• While the cat’s away, the mice play: What the drone delivery biz is doing while FAA ruling is under appeal
• Table tennis champion Timo Boll to play against Kuka robot arm
• German robot learns ping-pong from people

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Photo Essay
Six months ago, Typhoon Haiyan devastated the Philippine’s South East coast. Now Drone Adventures is helping the reconstruction effort by providing much-needed aerial maps of the region.

When Typhoon Haiyan, known as Yolanda in the Philippines, struck the South East coast of the country six months ago, on November 8, 2013, it devastated entire communities – tearing apart thousands of homes and destroying livelihoods. Many of those worst hit were ‘informal’ unplanned homes, situated at or near the water’s edge.

One of the humanitarian organisations that stepped up to help was Swiss-based Medair, which is dedicated to relieving human suffering in some of the world’s most remote and badly affected places. However after arriving in the country just 48 hours after Yolanda struck and conducting their initial assessments, the Medair team soon discovered that accurate, up-to-date maps of the region didn’t exist, with communities often having to rely on either hand-drawn maps or outdated Google versions.

This data gap is why Medair asked Drone Adventures to get involved. Over the course of six days in March 2014, our team helped Medair to:

• Create detailed 2D base maps and 3D terrain models of Tacloban, Dulag and Julita municipalities
• Assess typhoon damage and plan shelter reconstruction

We then further expanded our activities, visiting different communities on or near the East coast of Leyte island and providing leaders with physical high-resolution maps they could use to assess the situations in their communities and plan reconstruction efforts based on current, accurate data.

Mapping Dulag

Known as ‘Liberation Town’, Dulag is a settlement of 50,000 residents, located 36 km (22 miles) south of the administrative capital of Leyte island, Tacloban. Typhoon Haiyan made landfall just south of Dulag.

Dulag was where Medair started its work, rolling out a project to provide the most vulnerable families in the municipality with what the organization calls ‘core homes’. These dwellings offer a family disaster-proof foundations, a strong wooden frame, and a roof.  But that is all. Each family must then – with support from Medair – take ownership of its future by completing the rest of the home itself by creating all the walls, floors and interior features they require. To date, Medair has built 600 shelters, providing 3,000 people with homes.

When Medair first contacted Drone Adventures, its team had already started to collect geographic data, marking down for example the GPS co-ordinates of homes in the area and recording family details for each. However what staff were missing was up-to-date, high-resolution base maps that they could use to assess villages, see where people are currently living, monitor reconstruction work and more.

The Medair team did have access to existing local maps, produced most likely through local ground assessments, however these were not geographically accurate, for example in terms of road locations and distances.

Medair staff were also keen to assess the local environmental damage caused by Yolanda, since many local livelihoods depend on natural resources like coconut trees. Many of these trees had either been decapitated by the typhoon or completely uprooted, with whole forests completely wiped out in some areas. Therefore we also mapped these regions, shown below, as part of our work in Dulag.

The Gift of Geography

In addition to working with Medair to assess storm damage and create base maps, our team also produced physical maps to gift to four different communities on the island: Barangay 105, Julita, Dulag and Cabacongan.

Our approach before arriving in a new area was to start by getting local buy-in. So we would first contact the local ‘barangay’ (barrio) captain – effectively the village mayor – to explain who we were and what kind of data we could offer.

The response was always positive.

Leaders were excited by the opportunity of having their own accurate maps. Captains and their teams might only be using hand-drawn maps at present – never mind having computers with internet access – but they all understand how much value up-to-date, highly accurate geographic information can bring and the transformative effect proper maps could have on their work.

Our typical workflow was to arrive on site in the morning, fly our drones, and then, in the evening, process the several hundred images we had acquired that day. We would then visit the nearest local print shop and print the resulting map on a giant tarpaulin sheet, presenting this to barangay leaders the next day.

It is difficult to overstate what a difference it makes to give a community a real, physical map. With a large high-resolution photo in their hands, people have something they can gather around and start using right away. And while many towns don’t have staff on hand who know how to work with geographic information systems (GIS) – and outputs like 3D digital surface models – everyone can appreciate and benefit from an accurate bird’s-eye view of their community.

Meeting the Mayor

Before we left the Philippines, we headed further north to Tacloban, the capital of the Eastern Visayas region, to meet Mayor Alfred Romualdez.

The city is home to 200,000 people and was all but destroyed by the typhoon and its resulting storm surges. Numerous families lost their homes, particularly those who had built in unsafe, unplanned locations – so called ‘informal’ developments – near the seafront.

Following the creation of a new planning law that bans the building of homes within 40 meters (131 ft) of the waterfront, the mayor had assigned a large unused plot of land, near Barangay 105, to re-house displaced families. This area is sub-divided into numerous plots, which are individually sponsored by a range of donor organizations from around the world, each responsible for building a section of this resilient new community.

Since the mayor’s existing map of the region was far from perfect, our goal role was to produce a high-definition image – hosted online – which both Mayor Romualdez’s team and donor organizations could use to make better decisions and more accurately direct rebuilding efforts.

Several drone flights later and the two images below show the difference in accuracy achieved between the mayor’s previous plan and his new drone-sourced image of the same area.

Project Statistics

During the course of our six-day trip, our two-man team, together with our Medair partners, flew 29 drone mapping flights in total with a combined flight time of 11.6 hours. We mapped 48.6 sq km (18.8 sq mi) of land and acquired 5,139 images.

These images were used to create four high-resolution maps – of Barangay 105, Cabacongan, Julita and Dulag – plus seven additional low-res versions of surrounding countryside. The typical ground resolution of these orthomosaics was 5 cm (1.96 in) per pixel for villages and 8-10 cm (3.1 – 3.9 in) for larger areas.

The value of our Philippines mission was perhaps best summed up by one of Medair’s own staff, Rob Fielding. In the organization’s report on its first drone experience – online here – he comments:

Drones do not have a good reputation. People associate them with the military and think of them as weapons, but they can also be used for good. We are using the images taken by the drones to carry out damage assessments, to identify land that could be safer to relocate families to and to see where recovery efforts are taking place. Relief organisations are increasingly using drones to provide services not otherwise possible, and we intend to use them in response to future emergencies.

Read Medair’s report on its work in Dulag.

The Drone Adventures team used senseFly eBee mapping drones and Pix4D software for data processing and orthomosaic generation. A special thanks to our friends at Mapbox for hosting all our data online.

For more photos of our Philippines mission, check out the full photo gallery.

This article was originally posted on Drone Adventures. If you liked this article, you may also be interested in:

• Fukushima photo essay: A drone’s eye view
• Drone Adventures in Haiti
• A discussion on deploying drones for international development
• Ten lessons for farm drones
• Robots Podcast: Drone Journalism
• TED Global 2013 speaker Lian Pin Koh on “Conservation Drones”

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The search and rescue team involved in the recent South Korean ferry tragedy has deployed a remotely-operated underwater robot to help in the investigation and hopeful search for possible survivors in what has been called the country’s worst maritime disaster in forty years. The Sewol ferry sunk off the coast of South Korea on April 14, 2014 carrying 476 passengers and crew on board; of these, 339 were high school students and teachers on a field trip. While 174 people have been rescued, divers’ efforts to locate the remaining victims have been hampered by strong currents, cold water and floating debris. An underwater ROV by Deep Trekker has been deployed to help rescuers assess the wreckage and the surrounding sea floor before sending divers into the ferry’s interior.

Hwang Dae-Sik, a diver at the scene, explained to Reuters the immense difficulty of his task: “We are trained for hostile environments, but it’s hard to be brave when we meet bodies in dark water … This is the most gruelling and heartbreaking job of my career.” According to the Reuters report, divers have been navigating the dark corridors of the ferry while carrying tethered oxygen tanks and communication lines that can snag on the sharp corners of the wreckage’s interior. Divers can only stay underwater for about an hour at a time.

“My heart goes out to the friends and families of these victims,” said President of Deep Trekker Sam Macdonald. “We hope our ROV can help keep divers from being put at further risk and that the families will get some closure.”

Websolus Co., Deep Trekker’s South Korean distributor, had a demo of the DTG2 ROV on hand and was able to quickly deploy the device and their expertise to the search and rescue team at the site of the accident. “We were fortunate that our dealers, who are also expert operators of the DTG2, were there to help,” said Amanda Coulas, Project Manager at Deep Trekker.

The DTG2 is a lightweight (8.5 kg) and portable remotely operated vehicle (ROV) designed for underwater exploration. It uses a patented internal pitching system that allows the ROV to move horizontally and vertically using only two thrusters. The independently mounted camera allows the operator to either set the camera to hold steady as the body rotates, or set the camera to move with the body as it pitches, making it possible to do detailed inspections. The DTG2’s power is contained within the submersible, and its low-drag tether has a diameter of just 4mm, allowing it to maneuver with minimal impact from the surface. Its die-cast aluminum casing is anodized and powder coated, making it suitable for both fresh and salt water use.

Until recently, most underwater ROVs have been too expensive and complicated to be easily deployed by rescue crew – many costing in the range of $25K – $50K and requiring skilled engineers to operate. But according to Macdonald, the search and rescue market is starting to realize that it can have an ROV solution within their budget. The Deep Trekker unit operating at the wreckage site in South Korea runs under $5000 USD and their DTG2 Worker model, which is equipped with a two-function grabber and allows operators to retrieve objects under water, is priced just under $8000 USD. Deep Trekker has ROVs working in 34 countries at present.

“In general, it only takes a few minutes to understand the controls, an hour or so to be good at them and maybe a weekend of practice to be an expert operator,” said Coulas. “Other ROVs require topside power – a generator – to send power down the tether to operate the ROV. This makes it much harder to transport the ROV in a zodiac or small rescue boat. Our ROV can operate 5-8 hours on one 3-hour charge.”

A Waterloo-based company, Deep Trekker got its start in the Great Lakes region of Ontario, Canada. “My partner and the engineering brains behind Deep Trekker Inc., Jeff Lotz, had developed an ROV for a college project about 14 years ago after he capsized his canoe and lost a lot of gear,” said Macdonald via email. “I had a similar situation in 2009 when I dropped a flashlight off of my boat, and began thinking about all the exploration that could be done in the Great Lakes if we had an ROV. Through a series of connections, the three partners in Deep Trekker Inc. got together and began exploring the possibility of an affordable, portable ROV that would be accessible to everyone. After meeting, conceptualizing and building for about 18 months, the first Deep Trekker voyage was made in Lake Huron.” The trio’s website was later discovered by an aquaculture equipment dealer in Norway, whose initial purchase of several of units helped to launched the company.

Search and rescue is a relatively new market for Deep Trekker, which in the past has concentrated on the aquaculture, commercial diving, recreation, and energy sectors. A Research and Markets report released in March estimates the global ROV market to be $1.2 billion in 2014, primarily driven by offshore drilling.

Ever since Apple bought Prime Sense, roboticists have been asking what’s going to fill the gap. Robots need to know where they are and what’s around them. The Carmine, and the unveiled but undelivered Capri from Prime Sense, were the smallest, cheapest and most effective ways of mapping the environment available. Microsoft’s first Kinect was based on Prime Sense technology, and it enabled a raft of robotics developments, as did the Asus Xteon. But with the core Prime Sense technology off the market now, there are questions about what will fill the gap, and what will some crowd funded projects use to fulfill their campaigns. There are some interesting projects that promise partial or future answers though, like the LIDAR-Lite from PulsedLight and Google’s Project Tango.

Prime Sense was acquired by Apple in November 2013 for around $350 million. As well as their newly announced Capri, Prime Sense were working on ever smaller 3D sensing chips that could eventually be integrated into mobile phones and tablets. This would effectively turn mobile devices into perceptual computing devices, capable of interpreting a wider range of information, as well as enhancing entertainment. An example is using your phone to record your children’s height simply by taking a photo, interior space mapping, scanning for 3d printing, or gestural interactions rather than touch screens.

The new Capri is 10 times smaller than the Carmine used in the first Kinects, and able to be used as an embedded system. It’s USB powered, with a VGA depth map of 640×480, a Field of View 57.5×45 and a range of .8m to 3.5m. And it’s now owned by Apple. As recently as January at CES2014, the robotics community was talking about the lack of replacements available in the immediate future. Ted Larson from OLogic, a robotics/consumer electronics R&D firm who have been working on Google’s Project Tango, said that there was nothing affordable and available right now to drop into the gap left by Prime Sense technologies.

“Soft Kinetic is about the only real competitor to Prime Sense. Their technology is in the new Intel data camera on display at CES2014, however the off-the-shelf Soft Kinetic solutions are still expensive, bulky and big. And the creative cam solution is really only suitable for a very short range, around about a meter. So like the Leap, it’s really only useful when you’re in front of your computer.”

“Since the Project Tango device allows of 3D depth sensing, mapping and localization, all in one package, OLogic is extremely hopeful and interested in applications of it as a replacement for many components of a robot, all-in-one, which negates the need for a 3D sensor in a stand-alone configuration.” said Larson.

Although it promises much for the future, Project Tango is not an immediate solution to the Prime Sense problem. Larson was hoping to find a ready-to-go, similarly priced alternative to Prime Sense in the new technologies on display at CES2014 but came away disappointed. One of the partial solutions that’s been creating some excitement is the LIDAR-Lite from PulsedLight, which improves distance sensing, but it doesn’t seem to offer any wide scanning ability.

The LIDAR-Lite is an optical distance measurement solution, with a range of between 4 and 40 meters depending on beam angle, at a price point of about $80. The form factor is small, measuring 51mm x 30mm by 39mm including optics. And the LIDAR-Lite will suit projects where low power consumption is important, using a 5vdc power source and drawing 100 milliamps when measuring and less than 10 milliamps when idle. This makes it perfect for some UAV and robotics applications and it’s no surprise that the crowdfunding campaign for the device has already met target with over a week still to run.

PulsedLight takes distance range finding from the expensive, closed solution that it is today to small, cheap and open. This is tremendously appealing to us at 3D Robotics, as is the potential this has for getting us to truly autonomous UAVs without relying solely on GPS.
– Chris Anderson, CEO of 3D Robotics

The LIDAR-Lite is on the Dragon Innovation crowd funding platform, the first crowd funding platform purely for hardware projects, including robotics. Some of Dragon’s previous success stories include Romotive, Sifteo, Orbotix and Dash Robotics, as well as the uber popular Pebble. No surprise really that robotics is featured, as Dragon is in many ways an iRobot spinoff. Cofounders Herman Pang and Scott Miller are both veterans of iRobot’s engineering team and manufacturing operations in Asia. They saw the need that hardware startups had for manufacturing assistance. Starting out as a consultancy in 2009, Dragon Innovation has added a crowd funding platform and offers design assistance and manufacturing support.

Companies on the Dragon crowd funding site have had to go through Dragon’s premanufacturing program, which should guarantee that they ship on time and under budget. Pebble is a great example of a company that succeeded wildly on Kickstarter and then annoyed backers by shipping very late. Pebble now use Dragon for manufacturing.

“We envision Dragon Innovation as the official home for hardware, providing entrepreneurs everything they need to launch products and scale their companies,” said Scott N. Miller, CEO and co-founder. “By working closely with great entrepreneurs from the very beginning, Dragon can provide a full spectrum of resources and experience to help companies succeed. It makes natural sense for us to extend this commitment in the form of funding to help hardware entrepreneurs achieve success.”

LIDAR-Lite is made by PulsedLight, a company founded in 2011 by optical engineer Bob Lewis and Dennis Corey, with the goal of creating a very small and cheap, low-power, high performance optical ranging sensor. Depth measurement techniques using light, as opposed to using microwave or ultrasonic waves, fall into four main areas: triangulation or stereoscopic vision, linear laser scanning, structured light coding or Time-of-Flight.

Where the Carmine and Capri from Prime Sense use structured light coding for depth measurement, the new generation of Microsoft Kinects and the LIDAR-Lite use Time-of-Flight, mapping by calculating the time it takes for light to bounce off objects. This requires powerful light, a very sensitive detector, and a very precise clock. Until now, this has meant using lasers in potentially dangerous and much more expensive devices. LIDAR-Lite uses edge-based signal reconstruction methods, similar to that used in radar and some special signal correlation processing, which enables them to:

• Use LEDs and off-the-shelf components
• Process very weak return signals
• Calculate distance without having to directly measure it
• Do it on a single system-on-chip solution
• Reconfigure via software control

The newly announced stretch goal for PulsedLight’s campaign will mean adding lasers if funding reaches $75,000. And at time of writing the project is at $61,424 with 7 days to go, so it’s possible that Pulsed Light will convert all LIDAR-Lites to lasers not LEDs. This will enable greater range, smaller beam and faster measurement. The extra funds will be needed to help the team obtain the eye safety certification that laser devices need.

Meanwhile, most Occipital backers are still waiting for their Structure Sensors. The Structure Sensor aims to let you capture the world in 3D on your iPad, for modeling or AR or games. Occipital launched the device on Kickstarter in November 2013 and raised more than a million dollars on a $100,000 campaign. This very popular device has impressed at Make, CES and SxSW, but it’s basically a commercially packaged Capri.

http://youtu.be/ZN-9Ub9S7Cs

Occipital was supposed to ship beta units in December and fulfil the most of the backers in Feb 2014 but are well behind schedule, with the beta units just going out. Occipital recently attempted to reassure backers by saying that they were working with Prime Sense to ensure that they could fulfill orders for now, although components may change in the future. So while we’re waiting for Google’s Project Tango to reach the real world, PulsedLight might help keep roboticists ‘on the map’, and we still seem to have a bit of a Prime Sense problem.

• 75% of the ROV vehicles are used as work-class vehicles; the remainder are for inspection.
• The combined purchase and operations expenditures over those years will be $9.7 billion.
• Insight into those numbers provided by Unmanned Vehicle Systems Consulting said that global ROV sales in 2010 were $850 million of which 50% were for energy use; 25% for defense and security and 25% for scientific research.
• Further, roughly 20% of the vehicles were autonomous underwater vehicles (AUVs); the rest were ROVs (remotely operated vehicles).

\

To get around unknown environments, most robots will need to build maps. To help them do so, robots can use the fact that human environments are often made of geometric shapes like circles, rectangles and lines. The latest paper in Autonomous Robots presents a flexible framework for geometrical robotic mapping in structured environments.

Most human designed environments, such as buildings, present regular geometrical properties that can be preserved in the maps that robots build and use. If some information about the general layout of the environment is available, it can be used to build more meaningful models and significantly improve the accuracy of the resulting maps. Human cognition exploits domain knowledge to a large extent, usually employing prior assumptions for the interpretation of situations and environments. When we see a wall, for example, we assume that it’s straight. We’ll probably also assume that it’s connected to another orthogonal wall.

This research presents a novel framework for the inference and incorporation of knowledge about the structure of the environment into the robotic mapping process. A hierarchical representation of geometrical elements (features) and relations between them (constraints) provides enhanced flexibility, also making it possible to correct wrong hypotheses. Various features and constraints are available, and it is very easy to add even more.

A variety of experiments with both synthetic and real data were conducted. The map below was generated from data measured by a robot navigating Killian Court at MIT using a laser scanner, and allows the geometrical properties of the environment to be well respected. You can easily tell that features are parallel, orthogonal and straight where needed.

For more information, you can read the paper Feature based graph-SLAM in structured environments ( P. de la Puente and D. Rodriguez-Losada , Autonomous Robots – Springer US, Feb 2014) or ask questions below! 

PackBots will be deployed in Brazil during the 2014 World Cup Soccer season to bring a high-tech approach to security. The nation’s government has secured a $7.2 million deal with PackBot’s creators for 30 of the military bots. The robots will be stationed throughout Brazil’s 12 host cities, during the soccer matches to boost security and help examine any suspicious objects.

The sleek PackBot robot system, designed by iRobot, a Bedford, Massachusetts-based company, made its debut in 2002 in Afghanistan, where it helped soldiers clear bunkers, caves, collapsed building and to cross minefields. The versatile machine was used again in 2003 in Iraq in urban warfare situations and to search vehicles. By 2007, more than 800 of these military robots were in use throughout Afghanistan, Iraq and several other countries. The PackBot was also the first remote controlled robot to enter the Fukushima nuclear facility after the East Japan Earthquake and tsunami in March 2011.

PackBot is a unique robotic propulsion system that can reach a road speed of up to 9 miles/hour. The bot’s in-built flippers allow it 360 degrees of rotation, enabling it to negotiate rough terrain and even obstacles like logs, rocks, rubble and stairs. PackBot has the capability to climb up to 60 percent grades and survive submersion in water up to two meters depth.

Its robust body can survive a fall from a two meter height to a concrete surface, being flung through a window and falling downstairs. This bot weight less than 40lbs, and is easily carried in a back pack, ready to be deployed within minutes.

PackBot’s other attributes include a state-of-the-art GPS, video image display, system monitoring, electronic compass, temperature sensors. The robot is manipulated with an integrated Pentium-based computer.

If you liked this article, you may also be interested in:

• Quadrotor allegedly seen spying on French team’s closed practice at World Cup
• Exoskeleton with haptic sensors helps paralysed man to kick off World Cup
• Table tennis champion Timo Boll to play against Kuka robot arm
• ShanghAI Lectures: Xiao’an Li “Action Skill Developmental Learning for Autonomous Soccer Robots”
• Why drones are the future of sports photography 
• German robot learns ping-pong from people
• Robot Launch 2014: The first global startup competition for robotics is here!

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Japanese start-up HiBot, a maker of various types of robots for commercial use such as industrial pipeline cleaning, also developed a high transmission line inspection robot name Expliner. Last week HiBot announced a partnership with Hitachi High-Tech Group to expand the development and distribution of that robot.

Hitachi High-Tech has been inventing and manufacturing railroad track/wire inspection equipment as part of its infrastructure inspection business. Looking ahead, Hitachi High-Tech will collaborate with HiBot to expand into the transmission line inspection business as well.

HiBot was at the forefront of the development of overhead transmission line robots and was about to do live trials when the Fukushima disaster occurred. It took all Tepco resources – including those used with HiBot to test and install their robots – and redirected them to the disaster area and recovery efforts – which is why this new partnership agreement is fortuitous for HiBot.

In the power industry, optimizing the use, performance and life of transmission lines, is a continuing need for the power industry. Hence the need for this type of inspection robot. Different methods have been explored to examine, assess and determine and/or perform maintenance where needed, and, in Canada, to de-ice those lines as well. Many existing methods, some of which involve turning off the power, are too expensive to continue and include helicopter inspections. The Expliner robot by HiBot, Québec’s LineScout and the EPRI’s robot are the farthest along in providing those services.

The American Electric Power Research Institute (EPRI) has developed and is testing their robot. It was originally planned to be installed for field trials in 2013 but seems to be running at least a year or so late.

In Canada, Hydro-Québec Research Institute created the LineScout robot which has been in the testing and upgrading phase for the last couple of years.

In October, 2014, the 3rd International Conference on Applied Robotics for the Power Industry will take place in Brazil.

If you liked this article, you may also be interested in:

• Students hope their ship inspection robot will significantly reduce dry-dock time for maritime transport industry, improve safety for inspectors
• Ship hull cleaning & inspection – ideal for robots
• Underwater ROV keeps its eye on the target

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

An Irish surveyor tested results (and had those results peer reviewed through the local university) of surveys done by drone and software vs. doing it by hand (which took days longer). The surveyor is now prepared to stand by drone and software results in a court of law. 

The Irish surveyor used a Slovenian UAS from C-Astral Aerospace and a Russian mapping system, Agisoft.

In Japan, they’ve been using Yamaha R-Max unmanned helicopters for a variety of agricultural spraying and monitoring activities for years. From Canada to Switzerland and all over Australia and New Zealand, farmers and agricultural providers are experimenting with UAS to build the best apps to fit their needs. America must wait until the FAA decides (late in 2015) what rules and requirements will be needed to fly UAS in US airspace and when, after that, UAS will be able to enter the airspace.

In countries with less tight restrictions than in the U.S. commercial uses for aerial robots have been popping up in the news and experimented with by businesses.

The surveying profession is just one of many that will be disrupted as aerial robots are allowed to enter civilian airspace for their various tasks. Others include aerial surveying of crops, acrobatic aerial footage in filmmaking, search and rescue operations, inspecting power lines and pipelines, counting wildlife, delivering medical supplies to remote or otherwise inaccessible regions and some of these:

• Remote sensing;
• Commercial aerial surveillance;
• Weather monitoring and surveillance;
• Commercial and motion picture filmmaking;
• Domestic policing;
• Oil, gas and mineral exploration and production;
• Transport of materials;
• Scientific research;
• Armed attacks;
• Aerial target practice in training of human pilots;
• Search and rescue;
• Conservation;
• Maritime patrol;
• Forest fire detection;
• Archaeology;
• Thievery (yes-burgulars in Hong Kong were arrested after flying Parrot AR.Drones up to condo windows on hi-rise buildings to see whether people were home and whether there were goodies to be had);
• Commercial surveying;
• Agricultural spraying.

UPDATE: New video of a collaborative, cloud-based mapping experiment. Mapping is essential for mobile robots and a cornerstone of many more robotics applications that require a robot to interact with its physical environment. It is widely considered the most difficult perceptual problem in robotics, both from an algorithmic but also from a computational perspective. Mapping essentially requires solving a huge optimization problem over a large amount of images and their extracted features. This requires beefy computers and high-end graphics cards – resulting in power-hungry and expensive robots.

Update

Here is a video of a collaborative mapping experiment using Rapyuta, the open source cloud robotics framework. Instead of just compressing each frame of the stream, we now run a dense visual odometry algorithm to produce keyframes and these keyframes are eventually sent to the cloud for optimization. This technique is a very natural way to compress since keyframes are produced only when the robot has moved a certain distance or angle. The Cloud, in addition to optimization, now also does merging of maps produced by multiple robots. The code is open under the Apache 2.0 license at https://github.com/rapyuta/rapyuta-mapping.

Together with our RoboEarth colleagues Dominique Hunziker, Dhananjay Sathe, Mayank Singh, we have set up an inexpensive, light weight robot so that it can perform full 3D mapping in real-time by offloading heavy computation to the RoboEarth Cloud Engine (nickname: Rapyuta) (see previous Robohub article). In the video below Master student Vlad Usenko shows how robots can sidestep this problem.

How it works
What actually happens here? As you can see from the video and the figure below, the robot is equipped with an RGBD sensor (ASUS Xtion PRO), an ARM-based single board computer (Odriod-U2), and an off-the-shelf wireless dongle. The robot transmits RGB images (QVGA resolution) and depth images (QVGA resolution) at 30 FPS to Rapyuta with a frame by frame compression (JPEG for RGB, and PNG for depth images). The single board computer is only used for compression, communication, and low-level control of the robot – the heavy computation all happens off-board. The optimized map shown at the end of the video is built by manually triggering the optimization process in the cloud after the demo (i.e., not in real time).

The main purpose of this demonstrator is to showcase that today’s wireless technologies allow robots to outsource computationally heavy data processing to the cloud. As shown in the figure below, and as I’ve laid out in an earlier article on Cloud Robotics, bandwidth is a key driver for cloud-based robot services and applications.

Most actual mapping scenarios call for a slightly different split in work-load: You’d want to perform visual odometry locally and only send keyframes to Rapyuta for global optimization. The good news is that we already supports this setup: Rapyuta provides the corresponding software packages to run on ODroid boards along with the necessary cloud computing infrastructure.

What’s next?
This demonstration stops short of actually using the data returned from the cloud: A map is created and available in real time, but not actively used to control the robot. Using the map to control the robot, e.g. to improve exploration, is one logical extension.

Another natural next step would be to use the computational resources in the cloud for localizing in an existing map. Imagine a robot entering a new building and trying to get its bearings. Such localization in an existing map requires performing a search over the entire map – essentially a very large set of images and their features – to find the best match between what the robot is seeing now and a part of the map.

But the cloud can be used to do much more interesting things than improve single robot navigation. Multiple robots can instantaneously share their maps, so applications like multi-robot mapping become much easier. This implies using the cloud as more than a pure computational resource – it becomes a medium to store, share, and collectively improve data. This is interesting for static scenes, but things get even more interesting for dynamic scenes, for example when multiple robots track dynamic objects in the map. With WIRE, some of my RoboEarth colleagues have developed algorithms to do just that. We are currently working on integrating this functionality with Rapyuta and the RoboEarth database and will make it available as a Rapyuta cloud service later this year.

Many of the key advantages of using this Cloud Robotics approach to mapping will only become apparent when maps are brought together with the vast amounts of knowledge required to understand environments. Performing mapping in the cloud not only allows the creation of maps but also the ability to understand them: By bringing computation close to the knowledge required to make sense of all of a robot’s sensor information, Cloud Robotics offers robots a very powerful way to understand the world around them. When working together, cloud computing frameworks like Rapyuta and online knowledge bases like RoboEarth can allow us to make robots that are both smart and inexpensive.

Additional information
For further information, see:

• Technical details and tutorial for this setup (available early July)
• Video of recent live Cloud Robotics Mapping demo at ROSCon 2013
• Cloud Robotics Workshop at IROS 2013 in November, Tokyo
• Homepage of the RoboEarth Cloud Engine (codename “Rapyuta”)
• RoboEarth homepage

Questions?
Feel free to post in the comments below.

With the global market for unmanned systems in agriculture on the rise, companies that make robotic ground and air systems are paying close attention.

More than 25,000 “field” or agriculture robots will be sold by 2015 — about the same as robots for military use, according to the International Federation of Robotics. Together, defense and agriculture make up the lion’s share of the nearly 94,000 “service robots for professional use” that the IFR believes will be sold in the next couple of years. Defense and agriculture are by far the two largest categories in IFR calculations, with robots for things like logistics, medicine and rescue coming in well behind.

IFR statistics for 2011 sales of “professional use” robots — as opposed to robots for the industrial sector — offer a snapshot of this market. Overall unit sales in 2011 were up 9 percent over 2010, with unmanned systems for armies around the world coming in at 6,570, or 40 percent of the total. Right behind were sales of unmanned systems for agriculture at 5,000 units, or 31 percent.

Going Commercial
For companies like Harvest Automation, such numbers confirm that they’re on the right track. The Billerica, Mass., company was founded in 2010 to look for ways to match robotic technology with sectors that still require difficult, repetitive manual labor, says CEO John M. Kawola. Meatpacking, warehousing, construction and a host of other areas were scrutinized.

But, he says, the founders “pretty quickly honed in on agriculture,” because even though there have been significant advances in automation in “big ag” — wheat, corn and soy, for instance — “there still are large sectors of agriculture that rely heavily on manual labor.”

One is horticulture, a category of specialty crops, which, according to the U.S. Department of Agriculture, is defined as “intensively cultivated plants that are used by people for food, medicinal purposes and for aesthetic gratification.” This includes cut flowers, evergreens and bushes — “anything that is grown and sold in a container,” Kawola says. It’s “the nursery and greenhouse sector.”

But horticultural work today, “from planting to maintenance to harvesting, is being done the way it was done 20 years ago, 100 years ago.” Harvest Automation’s goal, therefore, is to use robotic technology to address material handling, harvesting, information gathering and sensing, and information management. It’s worth the effort because the wholesale horticulture market in the U.S. is huge, $17 billion. The crop value per acre is also huge — $15,000 for blueberries, for instance, compared to about $500 per acre for big ag crops like corn and wheat. In addition, it’s a much more continuous business — while big ag crops are planted and harvested once a year, horticulture crops can turn over every six weeks or so.

All this presents several possibilities to robot makers. The one initially chosen by Harvest Automation is to take the relatively easy approach of picking up and moving containers or potted plants. Its first product, the HV-100, is designed to do this job in nurseries and greenhouses. Kawola says it’s easy to operate by farm workers, and, because it’s relatively small, weighing about 80 pounds and standing two feet tall, it’s a safe working companion. He says it can work individually or in teams, depending on size of an operation. A video of several of the robots at work shows their movements to be similar to those of iRobot’s household Roomba vacuum cleaner. In fact, says Kawola, Joseph Jones, cofounder and chief technology officer of Harvest Automation, was one of the inventors of the Roomba when he was at iRobot.

Another company, Robotic Harvesting LLC of Simi Valley, Calif., has chosen to approach the market with a technically more complex system, one that harvests strawberries. The idea, says CEO Joe Wickham, is to develop a robot that can “automatically go up and down the fields picking strawberries. It’s still in the prototype phase. It’s a complex piece of machinery. It’s not something that can be done overnight, but we’re very interested in bringing this to market and getting commercial acceptance for it.” The recession slowed interest in the project and has prompted Robotic Harvesting to put it on the back burner.

Wickham acknowledges that, at least for now, people are more discerning than robots when it comes to identifying strawberries that are not fully ripe or misshapen. On the other hand, he says, advances in sensors and image processing software, “like disparity mapping and other algorithms that are useful for these types of complex vision tasks,” are making the technology more feasible. Meanwhile, Wickham says, some universities are studying the idea of developing a breed of strawberry plant that’s nearly leafless, “and the thinner the leaf canopy, the easier it is to robotically harvest the berry,” Wickham says.

Challenges For Flight
While companies like Harvest Automation and Robotic Harvesting are dealing with their own sets of challenges in agriculture, companies that want to use robot aerial systems in this market face others. One big hurdle is approval by the Federal Aviation Administration. It’s now illegal to fly unmanned aerial systems for commercial purposes in U.S. airspace. Assuming the Federal Aviation Administration approves low-level, line-of-sight operations — like those applicable to the monitoring and spraying of crops, for instance — analysts expect the market to boom. An AUVSI report in March 2013 says that during the period 2015-2025, integrating UAS into national airspace will contribute $82 billion to the U.S. economy — a whopping $75.6 billion of which will go directly to agriculture. Another $3.2 billion will boost the public safety sector, and the remaining $3.2 billion will go to the “other” category. More than 100,000 jobs would be created.

The 38-page report, “The Economic Impact of Unmanned Aircraft Systems Integration in the United States,” says unit sales of UAS will rocket from just under 40,000 in 2015 — when FAA is expected to give its approval — to about 160,000 by 2025. Even so, the study’s authors take what they say is a conservative approach, using “100,000 unit sales per year as a conservative benchmark.” One of the authors, Bijan Vasigh, a professor of transportation at Embry Riddle Aeronautical University in Daytona Beach, Fla., says the numbers could be higher, or lower. “We are really projecting what will happen in a normal scenario. If the scenario is more conducive for [unmanned aircraft], the impact would be much stronger.” At the same time, unforeseen circumstances “could impede progress.”

Some aren’t sure how the FAA will rule, but Vasigh is optimistic. “Frankly, I don’t see that much problem with FAA. I’m a little bit positive on go ahead, especially at lower altitude.”

Optimism, with a bit of caution, is also expressed by Young Kim, general manager of BOSH Precision Agriculture, Newport News, Va. BOSH, which has been oriented to the military UAS market, is shifting its sights to UAS for agriculture because the market is big and steady — as opposed to defense, which is big but probably won’t be steady forever.
“As an entrepreneur, I’m an optimist,” Kim says. At the same time, he sees no large, single, organized group engaging the FAA. “If we can get the farmer, UAV vendors, universities, economic development people, ag trade organizations,” like state cotton and peanut commissions and citrus growers, all addressing the FAA with a unified voice on safety and certification and policy, possibly with AUVSI participation, success would be more certain.

Foreign Expertise
Japan is well ahead of the U.S. in the use of unmanned aerial systems for agriculture. In 1983, the Japanese government asked Yamaha to help it develop an unmanned helicopter for agricultural duties. The country was faced with an aging farming population and wanted to make things like spraying more efficient, according to Steve Markofski, a U.S.-based new business planner for the company. In 1991, he says, Yamaha began to market its first ag-oriented unmanned helo, the R50. Today, 2,150 Yamaha RMAX unmanned helos spray about 2.5 million acres a year in Japan, covering about 40 percent of the country’s rice paddies. The government backed the idea from the beginning, with the Japanese department of agriculture responsible for regulation.

Yamaha hopes to translate its success in Japan to the U.S. One area that it thinks is promising is spraying vineyards. Yamaha is testing the idea, under a certificate of authorization from the FAA, by flying over vineyards in Napa, Calif. They’re about the same size as rice paddies in Japan, five acres or so, and therefore a good match to the RMAX’s four-gallon chemical payload capacity, Markofski says. Tractors, traditionally used in Napa vineyards, can spray about two acres an hour, but RMAX can do 12 to 15 acres an hour. And with RMAX, Markofski says, there’s no soil compaction, no crop damage, the operator is not exposed to chemicals and it is safer, because he doesn’t have to drive in challenging terrain, like slippery hills. “We’re looking at areas [in the U.S.] that we think align directly to the RMAX, and vineyards is one that jumped out at us,” Markofski says.

Canada’s MicroPilot also is waiting for a green light from the FAA. The company, based in Winnipeg, Manitoba, is a leading manufacturer of UAS autopilots as well as the maker of CropCam, a camera-toting fixed-wing UAS that has made a name for itself in crop monitoring. CropCams have been sold in places like Kazakhstan, Southeast Asia and South America, but approval to fly in the U.S. would be a big boost, says Pierre R. Pepin, vice president of sales and marketing. He says one business model might be to work with those who sell pesticides, for example. They also could sell UAS services. A customer would therefore not only get a product from the seller, but also precise images of his crops.

“Once you’re allowed to fly,” Pepin says, “it doesn’t take a whole lot of imagination to see how you can use those little birds.”

[This article first appeared in AUVSI’s Mission Critical‘s agriculture issue.]

‘Drones in agriculture’ has been a topic hot lately, but is it a real business?

As a former Army UAV platoon leader and a current founder of an aerial data collection start-up, I believe so, but when I first got back from Afghanistan, I did not see how drones could be a big thing in agriculture. WHY? I still do not see how the “pilotless app” makes a decisive difference in most civilian applications.  Yet as I expanded my view of a drone to be any flying network node — be it piloted or unpiloted, in the air or in space — I started to see how drone technology could deliver data that creates real value.

Drones may someday perform all kinds of “work” in the field, like spraying or seeding.  However, today the product of the drone industry is not drones, it’s data. Two ideas persuaded me that there is a business here.   The first is that variability information has real value to growers — it is production control data.  The second is that there are places where drone technology allows real data to replace heuristics and estimation in order to drive action.

Agriculture is a very sophisticated business.  Even in specialty crops with less automation, the growers are extremely sophisticated and as scientifically inclined as one would expect of managers of multi-million dollar production facilities.  Putting production facilities out of doors and using production implements derived from nature only increases the complexity of these operations.   In particular, it introduces uncertainty, variation, and risk into the production process.  The managers of these operations, who  have responsibility for maximizing production, are the customers for this data.

When these managers are trying to squeeze the most production out of a given set of inputs, particularly land and labor, there is a natural cycle of input opportunities.  Data only has value if the manager can act on it, and this is where agriculture becomes extremely heterogeneous.  At one end of the spectrum you have dry farming of corn, where there might be three or four viable input opportunities (plant, a spray or two, then harvest), most of which are already done automatically — or robotically — on a combine.  In high value per acre specialty crops (like vegetables, grapes, or nuts), managers might make over 50 discrete, pre-planned input decisions each year, especially if the crop is deficit irrigated.  Even though these crops are in some ways “lower tech” in that they don’t use as much machine automation (such as combines), they do have a much more sophisticated ability to react to variation data.  These are where the drone’s fundamental advantages are most likely to be valued.

The tradeoff in data collection is a tradeoff between the cost of collection per geographic unit and “responsiveness.”  Responsiveness is the degree to which the system can deliver exactly the data needed to make better decisions, exactly when needed, and is a function of many factors including system reliability, resolution, revisit rate, sensor selection, controllability, and ease of deployment.

Drones and other overhead networked systems provide a continuum of data options.  At one end you have LANDSAT satellite data, which provides free, 30-meter pixels, un-interpreted data once every 16 days (if you are lucky enough not have clouds).  At the other end of the continuum, there are hand-launched drones, which have a high real cost per acre because of their short lifespan and the amount of labor required, but can be launched at anytime and are cheap enough to keep in the back of your truck.  In between you have a variety of commercial solutions.

There will always be a tradeoff between cost and responsiveness, but the idea of drones is that electronics will move the efficient frontier in this tradeoff towards more responsiveness at a given cost.  The challenge with this access to better data is not to overshoot the mark.  It is very expensive to overshoot the required responsiveness for the crop.  The trick is match responsiveness to the needs of the manager.  This is the task for the commercial drone industry: match responsiveness to the needs of the decision-maker.

As time goes on, every category of platform across the continuum will have to offer more responsiveness for the same money.  However, the customer responsiveness needs will not increase unless the managers are also given new mechanisms for taking action. Beyond being used to control fast-spreading crop diseases, I do not know what new markets will open up for drones — perhaps they aren’t in agriculture; the film industry certainly requires even more responsiveness than anything in this field  — but it is going to be fun to discover them.

One of the challenges that I would like to lay down to the ground robotics and manipulation communities is to reduce the cost of making an intervention.  Until it makes production sense to go out into a cornfield 20 or 30 times per season and take an action to improve growing, there won’t be a need for more responsive information.  As interventions become more profitable and crop managers decide to increase the number of interventions, there will be a greater need to automate data collection.  Then we can kick off a virtuous cycle of robotic technology.

Of course the military applications of drone technology were being protested against outside the conference. The other highlight of DARC, which was a very stimulating multidisciplinary conference, was the inclusion of the protestors into the program. Conference organizers had as broad a range of viewpoints in the program as possible and when the protestors outside saw what was actually on the schedule, there were many who felt that being on the inside was far better. Kudos to the conference organizers for building strong bridges.

To continue the discussion around drones, DARC organizers have put together a guidebook on selected law and policy issues related to UAVs and outstanding public interest questions, which were explored at more depth in the conference. If you couldn’t go, then read the guidebook. Or read some excerpts here.

DARC law and policy intro

Drones are no longer the exclusive domain of the military. Or even of the state. At all levels—hobby aircraft, remotely piloted aircraft, and fully autonomous drones—unmanned aerial systems are growing in sophistication and accessibility, and are almost universally expected to transform aerospace.

The Association for Unmanned Vehicle Systems International (AUSVI), an industry trade association, claims that flying robots will create 100,000 jobs and contribute $82 billion toward U.S. GDP by 2025 and the Federal Aviation Administration (FAA) estimates as many as 30,000 unmanned aerial vehicles (UAVs) in American skies by 2020.

In 2012, Congress passed the FAA Modernization and Reform Act, tasking the FAA with the development of a plan for safely integrating unmanned aerial vehicles into domestic airspace by 2015. Safely introducing drones into the domestic airspace is, in itself, a daunting task—800 million people fly U.S. skies annually, and an errant $900 drone could easily take down a commercial airliner and everyone on board. But we need to do more than safely introduce drones into our airspace. We need to safely introduce them into society.

Drones herald an explosion of technological innovation. They’re enabled by — and drive — innovation in peripheral technologies like cameras and sensors. They also open the door for exciting new applications of existing technology. Worldwide, people are experimenting with using drones for wildlife observation, geological survey and environmental science, precision agriculture, firefighting, building and bridge inspection, mapping, and filming for both news and entertainment.

No comprehensive framework for civilian use of UAVs

It is important to remember that any comprehensive framework will also be shaped by the expertise and work of an unprecedented number of otherwise seemingly disparate government agencies, from NASA and the National Transportation Safety Board, to the National Oceanic and Atmospheric Administration, US Customs and Border Protection, and the US Patent and Trademark Office.

Some would argue that drones simply fit within our existing legal and regulatory framework. Even if it is the case that there is nothing new to be said about “drones,” formally mapping existing legal frameworks and precedents onto the wide spectrum of drones would still be a worthwhile endeavor.

Either way, the coming formal integration of drones into domestic airspace presents a unique opportunity — and call to action — for guiding the development of relevant law and policy, ahead of their wide-scale, mainstream adoption. It is perhaps the first time that we have had the occasion or ability to bring the expertise and knowledge of so many to bear on the direction of a technology with such landscape-altering potential.

Privacy

Arguably drones don’t do anything that other new technologies aren’t doing. In fact, probably much less; but they are a lightning rod for the debate around privacy and increasingly ubiquitous communications technology.

As Justice Alito points out in his US v Jones concurrence, “[i]n the pre-computer age, the greatest protections were neither constitutional nor statutory, but practical.” The further erosion of those practical limitations by the proliferation and democratization of drones is certainly cause for concern.

Public Space

As drones proliferate, how will the concept of public space change? Public spaces have traditionally been common areas enjoyed by the community at large. Naturally, most conceive public space along a horizontal plane; identifying a house as “private” and a park as “public” is relatively straightforward. With the promise of drone technology, we may have to recalibrate our thinking along a vertical plane as well.

The precise boundaries of public and private space have always been contested; a private company may own a shopping mall, but in the US such spaces are treated in many ways as “public.” The lack of a clear legal definition of “public space” will become increasingly problematic as new technologies allow communities to harness national airspace. As technology raises our reach upwards, we may find ourselves carving new public and private airways.

Whether drones present legitimately new issues for “public space” jurisprudence or just issues of scale, considering how the law conceptualizes public space in relation to drones is critical. The spectrum of drone technologies that implicate questions of boundaries and sovereignty is much wider than any previous technology. From tiny, insect-sized drones all the way up to military grade UAVs, drone technology is set to interact with public and private space in unpredictable ways. Development of law in this space will need to balance the public’s interest in maintaining a “public highway” privacy rights.

How can the law reconcile a largely public airspace with diverse privacy expectations on the ground? Should there be restrictions on the use of thermal, infrared, millimeter, or other advanced sensor technologies in the airspaces around public and private spaces? And, to what extent can local and state authorities develop or effect tailored regulations surrounding personal and commercial use of airspace?

Journalism

In addition to the fundamental aviation regulations (i.e., fundamental regulations on aircraft worthiness, pilot competency and flight operations), there are things unique to the practice of reporting and journalism. Journalism drones will raise concerns over privacy and data protection, and the role of the fourth estate. The gradient between reporting, paparazzi stalking, and surveillance could become gray. Drone-pilots-as-journalists may raise additional questions about best practices: does the desire to “get the story” create an ethical conflict for an operator of a heavy, dangerous machine? If you thought the debate over whether “bloggers are journalists” was interesting, wait for the wave of citizen drone journalists covering protests, disasters, and other community happenings.

Value Sensitive Design

Values in Design (VID) is a way of considering human life that explores how the values we think of as societal may be expressed in technological designs, and how these designs in turn shape our social values. In other words, technology is never neutral: certain design decisions enable or restrict the ways in which material objects may be used, and those decisions feed back into the myths and symbols we think are meaningful. Physical objects we use on a daily basis shape the way we act: driving cars means we walk less; access to the Internet means we use our digital devices more.

Because architecture and design features may be systematically related to political, social, and ethical values (such as security, privacy, and freedom of expression), it’s important to focus attention on these values from the ground up, rather than retroactively. But it’s also not enough to demonstrate that values are expressed through design decisions; they should be included among the set of aspirations that guide and constrain the designs of all technology. Ethical and political values should be placed on the same plane of importance alongside technical and functional specifications and constraints, and “engineered” into system components.

Safety

Protecting public safety with regard to drones is in no small part about safe design and sound engineering. Technical standards must be developed to ensure the airworthiness of unmanned systems, as well as standards for guaranteeing the quality and performance of the individual parts that keep those systems airborne.

But safeguarding against disaster also relies on the ability of humans to develop with the technology. That UAVs are becoming increasingly more available to consumers is not proof positive of the general public’s readiness to responsibly operate them. The safe operation of unmanned systems necessitates thinking about education and training, user interfaces, flight planning, safety check processes, and a host of other variables necessary for attaining an acceptable level of even recreational safety.

Registration and Monitoring

In 2006, a US Customs and Border Protection Predator B drone (often referred to as a Reaper) crashed into a hillside nearby homes just outside Nogales, AZ. A series of failures culminated in the complete loss of communication with the drone, during which time operators were unable to determine the location of the drone—including whether other aircraft were in danger of colliding with it—or where it would ultimately crash. The harrowing incident highlights just one of the reasons it can be terrifying to not know where a drone is and what it’s doing.

The lack of an onboard pilot allows UAVs to operate under dramatically different circumstances, such as flying at extreme altitudes and staying airborne for extended periods of time. For example, the Reaper involved in the Nogales incident had the capability to stay airborne for over 30 hours at altitudes up to 50,000 ft. This functionality opens the door to exciting new possibilities for scientific research and disaster reconnaissance, but also presents a new problem for people on the ground.

Encouraging Innovation

The proliferation of UAVs represents a new wave of technological innovation — not just in aerial robotics, but also peripheral technologies like cameras and sensors. And the mainstream availability of UAVs also opens the door for exciting new applications of existing technology. Yet, conspicuously absent from the Congressional Research Service’s recent report on legal issues relevant to the integration of drones into domestic airspace was any discussion about law and policy related to innovation, such as intellectual property and information law.

Intellectual property law will almost certainly play a significant role in the development of drones. Intellectual property protection of the physical embodiment of drones falls largely within the realm of patents, which give certain exclusive rights to the grantees of the patents, such as the right to make, use, and sell their invention. Of course patenting hardware is nothing new, and as a general matter, leaving aside questions of whether drones constitute patentable subject matter or what might qualify as prior art, mapping the existing framework for granting and challenging patents onto the drone landscape is fairly straightforward. Of course drones are not a singular piece of hardware, and depending on the sophistication of the drone, it might be comprised of hundreds of other pieces of hardware, any number of which might be patented, and therein lies the potential for trouble. The more patented components that a technology is comprised of, the greater the potential for patent thickets, where so many of the individual components are patented by different people that it becomes difficult to build anything without running afoul of someone’s patent rights.

Like computers and smartphones, drones are also a platform technology, allowing for the development of APIs and applications developed by third parties, invariably paving the way for UAVs to be the next battlefield for software patents and open source development. The promise of drones is not realized simply because we can each have one and operate it independently of each other. Much like the way the Internet unlocked the vast potential of computers, and your smartphone is far more interesting when you have a signal, networks that support UAVs can dramatically increase their usefulness, allowing drones to share information and communicate with each other.

For example, a drone studying weather might be able to relay real-time, granular information about approaching air turbulence to a drone being used for photogrammetry, allowing it to move to a safer altitude or position. Or a network of GPS waypoints might allow drones to stay on course on their way to delivering tacos. As a practical matter, some level of networking is necessitated by limitations in sense-and-avoid technologies that would require powerful, yet-to-be-developed sensors, as well as a lack of UAV maneuverability that limits their ability to get out of the way of fast-approaching objects. At the very least UAVs need to be able to know where other nearby drones are in order to avoid mid-air collisions.

But as beneficial as it might be to network technologies together, networks create other significant challenges for innovation. With a spectrum of users that stretches from backyard hobbyists, to university researchers, to long-established aerospace giants, concepts related to innovation, like net neutrality, standards, and interoperability, will likely need to be considered in a new light.

Negligence and Liability

The FAA estimates as many as 30,000 UAVs flying overhead by 2020. No technology is immune to failure, and even the most fervent proponent will admit that drones are no exception. High profile drone crashes—like the 2006 Reaper crash outside Nogales, AZ and this year’s QF-4 crash near Panama City, FL—are clear reminders of the possibility of drones falling from the sky. Indeed, just days before DARC, a quadcopter smashed into the sidewalk in Manhattan, narrowly missing a pedestrian. While the drone community is hard at work improving the reliability of UAVs, efforts to integrate drones into domestic airspace clearly envision an acceptable failure rate greater than zero.

Tort law is an area of civil law that addresses violations of duties and responsibilities owed to others. Via tort law, parties suffering injuries— and not just physical injuries—resulting from the actions of others can seek compensation, shifting the cost of those injuries to the party responsible for inflicting them.

The law has long dealt with attacks, accidents, dangerous actions, and defective products that cause damage to people and property, and allowing victims to seek redress helps to discourage careless and risky behavior. What remains to be seen is how well these laws work in a world filled with flying robots.

Secrecy

The fallout over the NSA’s Internet communications surveillance demonstrates the impact of procedural bottlenecks inside government when a culture of excessive secrecy meets vast computing power. As an unprecedented dual-use technology, drones may well be the next frontier in the battle for information.

Regulators tasked with developing regulatory frameworks for drones face a significant challenge. Having never encountered a technology with such a wide breadth of uses and implications, they are at a severe experiential deficit as they attempt to craft rules for governing the civilian use of a technology that is equally applicable to Domino’s Pizza and the US Air Force, and can be purchased by anyone on Amazon.com for a few hundred dollars.

Given these circumstances, there is a clear need for discussion about what regulators and the public at large need to know, would like to know, and shouldn’t know because of commercial concerns. As we inch towards the 2015 integration deadline, our ability to have an intelligent conversation about the contours of civilian UAV use could be hampered by undue and unnecessary secrecy.

Comparative Law (Globalization)

UAV use is an international phenomenon, and the spatial issues it raises domestically around property and airspace don’t end at our borders. Every country that has tackled the issue of domestic UV integration has faced similar challenges. Around the world, legislatures and administrative agencies have crafted integration plans that anticipate how their sovereign national airspace may be affected by the proliferation of unmanned, commercially owned aerial systems.

Additionally, surveying national UAS integration policies around the world could reveal developing international norms. The complex challenge of anticipating how AV proliferation will affect life from country to country is dwarfed by the fact that the spatial issues drones raise for airspace and property transcend borders. Drones could render the line between French airspace and German airspace as permeable as the line between public and private domestic space.

Full DARC guidebook, references, acknowledgements and additional reports.

Hear the word “drone” today and you’ll probably picture some kind of flying weapon, snooping or raining Hellfire missiles from above. But in reality, the first drones you’re likely to see actually in use are more likely to be closer to crop dusters, buzzing over farms. Rather than taking pictures and videos of people, they’ll be surveying fields, using their high-resolution sensors to improve crop yield and decrease agricultural water and chemical use.

Why farms? Because agriculture is a big data problem without the big data .  About half of the “inputs” in farming (from fluids to pesticides, fungicides and herbicides) are typically wasted because they’re applied in greater amount than needed or in the wrong place, such as the ground between plants rather than the plants themselves. That’s considered unavoidable, due to the nature of spray application or the need to avoid under-use of water and chemicals, which can be catastrophic, from disease outbreak to total crop loss.

Soon farmers will know what’s going on with every plant, spotting problems before they spread, and applying chemicals with honeybee precision. They’ll use pesticides and fungicides only when needed and in the smallest amounts necessary, lowering the chemical load in both food and environment and saving money. On a small farm, you can get that level of precision with hand-tending. But on a big farm, the answer is more likely to be robotics, including flying robots — drones.

For the past year, my team at 3D Robotics has been flying our drones on farms, gathering data and talking to farmers about what they want and how they work. These are some of the lessons we’ve learned:

1. Every crop is different! It goes without saying, but grapes are not tomatoes, and tomatoes are not corn. There are hundreds of different kinds of farms, ranging from trees to roots (and that’s not even including livestock and ranching). Each crop needs to be measured differently to generate actionable data. There is no universal crop survey solution, and it will probably be specialists in each particular crop type who ultimately deliver solutions to farmers.
2. Multicopters, not planes. We started with fixed-wing UAVs, but quickly realized that most farms don’t have landing strips. Even short takeoff-and-landing planes get battered fast in regular use without dedicated landing areas, which few farms have. Meanwhile, multicopters, which can take off and land anywhere, are gaining endurance and can now fly for as much as 40 minutes and cover miles. Planes are only suited for the largest farms, and even then missions need to be planned very carefully to find places they can reliably land.
3. Phones/tablets, not laptops. Farmers don’t want to drag laptops into the fields. Any drone that is expected to be used by regular consumers should be entirely operated by a standard Apple or Android smartphone or tablet.
4. One-click auto missions, not “flying”. Likewise, farmers don’t want to have to fly things . Agricultural UAVs should be fully-autonomous, from takeoff to landing. The experience should be as simple as pressing a “Start” button on a phone and the drone flies the entire mission on its own.
5. Fly the camera, not the aircraft: What the farmer is interested in is a picture — not the acquisition of the picture.  Let sophisticated planning tools figure out precisely how to gather the right images, let autonomy take care of the nitty gritty details of flight dynamics, and let humans do what humans do best — specify high-level desires.
6. Video can be worth more than stills. Don’t discount how good farmers are at spotting things with their own eyes. Sometimes a first-person-view live video feed will allow them to spot issues and direct the vehicle to more closely inspect the problem area. (Needless to say, this is only really practical with multicopters). Indeed, farmers may not even know what they’re looking for initially.  Sometimes general situational awareness is the task, rather than delivering a specific data product (such as a mosaic).
7. NDVI is surprisingly easy to do. The gold standard of crop surveying is a “Normalized Differential Vegetation Index”, which shows the difference between regular red light reflected from plants and near-infrared light. Healthy chlorophyll absorbs red and reflects near-IR, while damaged chlorophyll reflect both. It doesn’t take expensive cameras to gather this data. A regular camera slightly modified with a blue bit of plastic becomes a near-IR camera. Take a cheap consumer 3D camera with two lenses, modify one for near-IR, and you’ve got a NDVI camera for less than $200.
8. Aim for crop consultants, not farmers. Most crop data services are provided by local consultants, such as agronomists, not the farmers themselves. At the moment, FAA regulations ban most commercial use of UAVs, defined as anything where money changes hands, so most are used by farmers themselves for their own purposes on their own land. But Congress has mandated that the FAA introduce regulations to allow wider commercial use by 2015 (although it will probably be later than that before this happens). At that point, expect most users to be those local service providers, not the farmers themselves.
9. Time is money. Drones can get answers fast and cheaply, taking advantage of their “anywhere, anytime access to the sky” abilities. That means “timely data on time”, such as daily surveys to find exactly the right time to harvest. Likewise, changes over time can be equally illuminating. The aim of crop surveying is to show the farmers something they can’t see with their own eyes, and the time dimension is a great example of that. By doing regular crop surveys, say every day or week, and using software to highlight differences over time, it’s possible to zero in on growing differences between areas of a field, which may be directly correlated to productivity.
10. Data can be marketing. Some seed companies already offer to do aerial crop surveys for free as part of a sales process, much as they once “walked the field” as part of a free crop analysis process. Similarly, crop survey data can do more than simply guide a farmer into making different crop management decisions. It can also allow the farmer to market their harvest more effectively, pitching such high-tech precision agriculture as a differentiating quality in a commodity field. If data-driven crop management lead farmers to use less chemicals and water, perhaps someday “drone-guided agricultural” will be something consumers could be willing to pay more for.   Done right, big data agriculture means “greener” crops and food.  If consumers will pay a premium for organic, why not for this?

If you liked this article, you may also be interested in:

• Snap 2014: Aerial drones and 3D mapping software for civil engineering surveying
• Down on the farm with drones
• Matching technology to value creation: Drones in agriculture
• UAV innovation is in the Cloud; 3 precision Ag innovations to watch for in 2014

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Strapped for cash and with limited resources to allocate, municipalities are faced with the same type of problems globally competitive businesses have: the need to use automation where it can either augment or replace limited manpower. There are about 18,000 police departments in the U.S. and about 133,000 K-12 public and private schools. These days, almost all those schools want some form of police monitoring, if not physical presence, further straining city and police resources.

The robot is a Segway-based platform with a payload of sensors, cameras and communication devices and a navigation and collision-avoidance system customized to fit various types of community needs. It has a nifty design, enough weight to thwart all but the huskiest of thieves, and sensors for day and nighttime imaging, air quality and temperature monitoring, and cameras for license plate recognition, heat mapping and much more. Linked with other devices such as external cameras, the robot’s own streaming cameras, communications with local authorities, and a set of smart software, the robot can patrol and add a wealth of data and support to those other resources.

Stacy Stephens, Knightscope’s VP for Marketing and Sales, is an ex-policeman; William Santana Li, the CEO, has design and auto industry and successful start-ups in his background.

What struck me as unusual were the backgrounds of the people involved and their focus on solving a clear and growing problem within our cities. Their approach, which includes many sciences, and cooperation between those sciences and local authorities, is to augment the manpower of those agencies where criteria can be met using sensors, quick analysis of the data, and alarms, rather than feet on the street.

Yet current investment in robotic solutions by the agricultural sector is a fraction of its potential, and many financial, regulatory, practical and social obstacles prevent farmers from implementing these at a commercial scale.

In order to delve into the complex ecosystem of obstacles and pathways to agricultural robotic innovation, Robohub is launching a focus series “Agricultural Robotics” featuring original articles and interviews from leading experts in the fields of robotics, agricultural automation and farming.

The series will look at:

• How and where robotics is currently being used to augment and influence agricultural practice;
• What new innovations are in the R&D pipeline, and what obstacles await these;
• What are the potential business models and pathways for bringing these innovations to the point of commercial viability;
• What farmers and consumers think about incorporating robotics into farming practice; and
• Specific technologies such as UAVs, tractor automation and picking solutions.

We will be covering a whole range of perspectives — R&D development, regulation, business and investment, practical implementation, and sustainable farming — from an excellent line-up of contributors.

Not surprisingly, drone technology features heavily in this series:

• Based on his experience developing drones for farmers, Chris Anderson of 3D Robotics gives us Ten lessons for farm drones
• Ernest Earon of PrecisionHawk argues that, in agriculture, the future of UAV innovation is in the cloud
• Rich Tuttle from AUVSI looks at how the market for field drones, both in the US and internationally, will change when the  FAA lifts its regulations in 2015 in Unmanned systems coming to commercial agriculture
• Robert Morris of TerrAvion looks at drone data and the importance of Matching technology to value creation
• Similarly, Patrick Egan of sUASNews looks at the value proposition of UAVs in Down on the farm with drones

But of course there is more to high-tech farming than drones!

• We will learn about small autonomous farming robots from David Dorhout of Dorhout R&D
• Peter Corke of QUT will discuss the use of robotics for zero-tillage agriculture
• Jorge Heraud of Blue River Technology will talk about using computer vision and robotics to build intelligent solutions for lettuce-thinning
• Kendra Kerrisk of FutureDairy discusses robotic milking and herding technology
• Joseph Jones of Harvest Automation looks at the the forces that drive where commercial robots will be applied in agriculture, using as an example Harvey: A working robot for container crops that distributes and collects container-grown plants in greenhouses and nursery farms
• Jeremy Brown of Jaybridge Robotics explores the trajectory from precision farming to autonomous farming using the Kinze Autonomous Grain Harvesting System as a case study
• Our own John Payne writes about the power of robotics to radically transform the way we think about agriculture
• And Nikolaus Correll from the University of Colorado will take us far beyond earth’s green fields to look at robotic systems for horticulture in outer space

Many thanks to AUVSI for sharing their enlightening infographic on the history of machinery in the fields from their recent issue of Mission Critical. Thanks also to our podcast team, and especially to Ron Vanderkley, who has organized so many of podcast interviews for the series. Thanks Ron!

Hope you enjoy!

… Two events, from different parts of the world, on the same day and subject: geographic information systems (GIS) and global positioning systems (GPS) and the use of “layering” to map, analyze and design.
… Today is 4-H National Youth Science Day, and 4-H Maps & Apps will be introduced to millions of American youth to exhibit and examine the power of geospatial technologies. “We created this year’s experiment to inspire young people to be critical thinkers by embracing a technology that can help change and shape communities in innovative ways.”
… Also today, senseFly, the Swiss start-up recently acquired by French wireless devices provider Parrot (also famed for providing the AR.Drone quadcopter), announced that they will begin providing swarm technology for mapping. The new swarm capability has been incorporated into senseFly’s ground control software and includes in-flight collision avoidance and shared waypoints.
… Spectacular video showing how mapping the Paris Air Show is done using two ebee drones and senseFly’s new swarm technology.

A team of students from ETH Zurich and ZHdK have developed a prototype for a robotic ship inspection unit that is capable of conducting visual inspections of ship ballasts. Ballast inspection – which involves navigating hard-to-reach spots with no line of sight, often in the presence of intense heat, humidity, and hazardous gases – is normally done by human inspectors, and represents a significant cost to ship-owners who must pay for dry-docking and who face lost income when they cannot operate their ships during the inspection period. Because robotic ship inspection can occur while the ship is in operation, it could significantly reduce dry-dock time. The Ship Inspection Robot (SIR), which was developed in conjunction with Alstom Inspection Robotics and which uses magnetic wheels to navigate the I-beams and other awkward obstacles found inside ship ballast, is relatively compact and does not require any cables for power or communication, and thus offers significant mobility improvements over other robotic ship inspection prototypes. Project leaders anticipate that a per unit production cost could be as low as €4K, enabling shipping companies to operate several units in parallel as an additional time-saving measure. The robot was developed as part of a Bachelor-level project that aims to give students practical experience and promote contacts with industry.

Cargo ships, which transport everything from bananas, coffee, grain, coal, heavy equipment, chemicals, oil and gas, are the backbone of intercontinental trade, and the shipping industry represents an annual estimated average of more than €300bn worldwide. According to the International Shipping – Facts and Figures report published by the International Maritime Organisation, there were 103 392 commercial ships in service by January 2011. With an average age of 19 years and an expected lifespan of 25 years, regular inspection and repair is an important part of the global fleet maintenance.

The mechanical stress of loading and unloading, and the corrosive properties of seawater all take their toll on a vessel’s structure. This is especially true for a ship’s ballast tanks, which are filled with seawater to compensate for weight changes as cargo is loaded and unloaded. Located between the outer hull and the cargo holds, and consisting of beams and stiffeners separated by bulkheads, ballast tanks form much of a ship’s supporting structure.

The surface of ballast tanks is subject to saltwater exposure and algae accumulation, and as a result, must be inspected for damage every five years. A standard visual inspection, which can only be conducted with the ship in dry-dock after about a two-day venting period, serves as a kind of triage for determining what repairs are needed, and where. During inspection and repair, time is of the essence. Ship owners must pay for every day in dry-dock (€16K to €25K per day) and, in addition, they face loss of income when goods cannot be transported (€16K – €50K or more per day, depending on the value of the goods; oil and its derivatives, for example, can double in value within weeks in the event of a oil crisis).

Visual inspection is normally carried out by humans; however, intense heat and humidity, and hazardous gases make ship inspection a difficult and dangerous task. Ballast inspection is therefore an ideal task for robots, but the inside of the ballast, which has both convex and concave edges, as well as T-, L-, bulb- and I-profiles, presents a significant navigation challenge for any robotic inspection system. A few prototypes have been developed to date, however most of these systems have been large and heavy, and have required cables (for communication and power) that significantly limit their range and mobility.

The project
SIR was developed as part of an industrial-academic collaboration between Alstom Inspection Robotics, ETH Zurich and ZHdK. Alstom Inspection Robotics proposed a project that would result in a lightweight, low-cost robot for inspecting ships and similar structures, and provided some of the technology; a team of ten undergraduate students (six mechanical engineers and two electrical engineers from ETH Zurich, and two industrial designers from ZHdK) completed the project from concept to functional prototype as part of a nine-month bachelor-level focus project.

Developing the inspection scenario

In order to research the state-of-the-art of the ship inspection process, we went to a shipyard in Rotterdam to interview ship owners, inspectors, and maintenance workers, and to explore the inside of bulk carriers — SIR’s eventual workplace. This visit helped us to understand everyone’s priorities, how an inspection robot might be used to address them, and allowed us to develop, together with Alstom Inspection Robotics, a realistic potential inspection scenario that would serve as a guide for SIR’s eventual design. Here is the scenario we came up with:

As part of a round-the-year continuous inspection, the ship owner decides to use a four-day cruise to inspect part of the ballast tank. The inspection area is split into sections 3m wide and 50m long, which are bordered by stiffeners that can be climbed, but which limit sight. Calculations done on power consumption suggest a run time of about three hours; with an assumed average speed of 4.5 cm/s (calculated as a weighted mix of maximum and obstacle speed), ten of these sections could be inspected on a single battery charge.

Once inspection is complete, the data is analysed and suspicious areas are marked. Available data from earlier inspections is used to detect changes, and the damage map is updated. The next time a ship is due for inspection, the damage map is checked, and necessary equipment and material are ordered before the ship even enters the dry dock. 

Once in dry dock, the ship is vented and known repairs begin without delay at the same time as small or complexly shaped areas (like the bow) are manually inspected. A full inspection is shifted to the end of the dry-docking: since the inspectors already have access to the data, they can identify critical problems in advance and shorten the time they spend inspecting undamaged areas.

The prototype

The focus project resulted in a prototype for a relatively small, cable-less robotic ship inspection system that we call the Ship Inspection Robot (SIR). The prototype uses magnetic wheels to crawl along the metal ballast surface; SIR is controlled by a human inspector via wireless transmitter, and a powerful battery allows for approximately three hours of run time on a single charge. We envision that future prototypes will be able to map damaged areas. Below are some of SIR’s most important features:

1. Continuous inspection for early damage detection
   Because, unlike its human counterparts, SIR is immune to poor air quality and does not require scaffolding to climb, inspection could be undertaken while the ship is cruising. It could thus be used as part of an on-going year-round inspection program, allowing ship owners to identify and monitor suspicious areas, and order materials and plan repairs ahead of a scheduled dry-docking. While SIR is not meant as a replacement for a human visual inspection, it would allow human inspectors in dry-dock to focus on damaged areas, and on areas that are not accessible by robot, such as the ship’s bow.
2. Low cost of entry
   Because of its structural simplicity and its use of relatively inexpensive peripheral equipment, the target per-unit production cost of a commercially produced version of SIR is expected to be about €4K, allowing ship-owners or their subcontractors to treat them as a consumable rather than an asset. This low cost would allow operators to own and operate several units in parallel. The prototype cost about €12K to produce.
3. Low degree of autonomy
   Conceptually, SIR has a low degree of autonomy. Ship inspections are carried out according to very strict regulations and procedures, leaving no margin for errors, and one way to reduce errors is to keep things simple. This feel of being ‘in control’ also increases market acceptability.

How it works

SIR features four overlapping magnetic wheels, which allow the unit to maintain continual contact with the ship’s surface, even as it navigates sharp angles, vertical surfaces and overhangs. A gamepad allows the human operator to control the device, and a top-mounted camera and lighting system allow for visual inspection. The robot has almost no on-board computing power, making it lightweight and power efficient; almost all processing and control tasks (with the exception of a micro-controller for bundling sensor data) are executed on a base station that runs the Robot Operating System.

The robot uses four Sharp infra-red distance sensors for edge and obstacle detection, and an Invensense MPU 6050 IMU provides the operator with orientation data. These sensors are currently only used for user feedback. The camera used on the robot is a GoPro Hero 3 where it serves as a demonstrator for a visual sensor. It is assisted by three lights, one pointing in the viewing direction while the other two illuminate the ground in a flat angle, so as to make bumps and small edges more visible thanks to the longer shadow they cast on the ground.

Communication is carried out by two systems. The first system is an analogue wireless device. It uses the same 2.4GHz frequency as normal WLAN devices, however the signal is analogue, giving it more stability. It can be exchanged for an ethernet cable at any time. This device is used to control the robot, since delay through this channel is barely perceivable. The second system is a wireless video transmitter for the GoPro camera that is often used in model airplanes.

Performance
Working as part of an interdisciplinary team and organising a project all at once was certainly a challenge, however SIR proved itself to be a successful focus project. Not only did SIR perform as planned, it also showed that the mechanical and electrical design can be integrated into a robust and compact design; this was achieved by working closely together to integrate our various areas of expertise.

Our primary goal was for SIR to be able to overcome obstacles, and thanks to its compact wheelbase, SIR is able to do this with a high reliability, a thing no other robot known to us can do! Overcoming obstacles still needs some work in terms of reliability and smoothness, however, the robot never detached from the surface during a test and, in most cases, was recoverable without interaction from outside. We are proud that we were able to raise the bar in this respect.

So far, tests have shown that many of the other desired tasks can also be fulfilled by the current prototype, such as the ability to carry a variety of sensors.

SIR’s high range of mobility is the key value we added to the state-of-the-art of inspection robots, and thus the biggest achievement of our project. The ability to climb all these additional obstacles is not only beneficial for ship inspection, but also useful for many other robot applications such as power plant inspections.

Further development of SIR lies in Alstom Inspection Robotics’ hands.

Links
SIR website
Product Development Group at ETH Zurich
Alstom Inspection Robotics
ZHdK

If you liked this article, you may also be interested in:

• Ship hull cleaning & inspection – ideal for robots
• Beginner’s guide to the humanoid robot challenge (DRC)
• Underwater ROV keeps its eye on the target

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

To understand what the Drones for Schools program is, and how it came about, it helps to know a few things about my background. I didn’t originally set out to be a STEM educator. I had some science education from studying mechanical engineering as an undergraduate, but eventually wound up with a master’s in journalism. I didn’t think the two halves of my life would merge until I finished my master’s and took a job at a K-12 STEM education grant, at the University of Illinois – they needed someone with a scientific background to handle communications and study the social networks that teachers were forming as part of the grant.

At the time, I was researching how journalists could use aerial robots, commonly called “drones”, to capture data during natural and man-made disasters. As part of this research, I was spending a great deal of time in my basement “hacking” together a small drone from a radio controlled airplane and open-source hardware. It became apparent that this learning experience mirrored what teachers were doing in the STEM grant, and so I approached my boss, a department head in the College of Education one of the co-principal investigators for the grant, with a pitch: let’s have students design, build, and fly drones at our partner schools. He assembled a team of STEM educators to guide the process, and the “Drones for Schools” initiative was born.

Two students collaborate to build a “drone”.

In some ways, the program resembles a nesting doll for concepts in science, technology, engineering and mathematics. A drone is a complex machine, and once you peek inside, one STEM concept leads to another. To design the aircraft, you begin with concepts like Bernoulli’s principle, which when applied through a mathematical formula, determines parameters such as stall speed (how slow the plane can fly before falling out of the sky). Stall speed dictates the power you need to fly, which opens an opportunity to deliver content about electricity and electric motors. And delivering content about electricity and electric motors segues nicely into lessons about battery chemistry and capacity. All the while, students are applying this content knowledge as they make key decisions about how this aircraft will be designed and deployed.

Because this is a semi-autonomous aircraft that is dependent on a small microcontroller, computer programming is interwoven into the Drones for Schools curriculum. Students take this full-function microcontroller, which uses hackable open-source hardware and software, and learn programming by getting it to do some things it wasn’t intended to do. They learn about electronics prototyping by wiring up the microcontroller to external LEDs, and by writing C code to enabe the microcontroller to blink out a Morse code, they learn about variables, arguments, loops, functions, and libraries.

Soldering is one of the many skills that students learn through the Drones for Schools program.

These individual lessons are valuable on their own, but the program’s biggest gains come from students assuming the roles of engineers. This program was designed in part to meet the demands of the Next Generation Science Standards (NGSS), which were developed by the National Research Council to help prepare K-12 students for an world where science literacy is increasingly necessary. Perhaps one of the biggest challenges and opportunities presented by NGSS, is the incorporation of engineering education into the K-12 science curriculum, something which hasn’t been attempted on this scale. To help students understand what makes them engineers in the Drones for Schools program, and how that’s different from what scientists do, we’ve had a graduate engineering student explain how engineers make models, design solutions, and test those solutions. The engineering student also provided high school students with information on how their interests in the Drones for Schools program could lead to an education in engineering, and what an engineering career path looks like.

Drones for Schools is just one of around 30 collaborations at the National Science Foundation grant EnLiST, which stands for Entrepreneurial Leadership in STEM Teaching and learning. Each collaboration is unique, but they are all “entrepreneurial,” meaning they draw on resources from other classrooms, university faculty, community organizations, and businesses. Drones for Schools is no different than the other collaborations in the sense that we rely on community partners to provide a useful application for the drone. The drone is not built for the sake of building a drone; rather it is built because it can provide high-resolution images of the ground which can be stitched together in a map. And that data has use for farmers, city planners, researchers, first responders, and many others. Simply put, we think there is a need for drones in the community, and our students are trying to fill that need.

Testing out one of the class’ inventions

AUVSI, the Association for Unmanned Vehicles and Systems International, predicts that 30,000 drones will be flying by 2020, with the majority of those serving as farm equipment. Depending on the definition of a drone, the US may already have exceeded that number. The goal of this program isn’t to ensure that one of our students will engineer one of those drones someday, although technically they’re already building and flying one of those 30,000 when take part of our initiative. Rather, we’re interested in our classroom data, which suggests the initiative increases interest in engineering. And in that sense, I think we’ve succeeded.

UIUC faculty and staff associated with this grant are currently seeking other grants to specifically enrich middle school STEM education. Because of the tremendous learning opportunities packed into these flying robots (some of which are small enough to fly in a classroom), and because of their real-world importance, we believe that drones will be an integral part of STEM opportunities for some time.

… Once again National Geographic photographers have used innovative techniques to capture amazing images. Remember the polar bear cam in 2011 that got crushed by the curious bear as he examined and stepped on it?
… This time photo journalist Michael Nichols used camouflaged SuperDroid Robots’ AGVs and quiet drones to capture the lives of lions on the Serengeti. Did you know they are the only cat that is truly social?
… The August issue of National Geographic has the story and a terrific online component as well.

Space Exploration Technologies (SpaceX) announced today that, in 2018, it will be carrying three satellites that will make up Canada’s RADARSAT Constellation Mission (RCM) into orbit aboard its Falcon 9 rocket.

The launch reservation contract was awarded by  MacDonald, Dettwiler and Associates Ltd. (MDA Robotics) to support Canada’s largest space program to date in Canada. The satellite mission, which will be built on previous missions RADARSAT-1 and RADARSAT-2, will “support Canada’s need for maritime surveillance, disaster management and ecosystem monitoring”.

Two small drones, Insitu‘s Scan Eagle X200 and AeroVironment‘s PUMA, have become the first federally certified unmanned aerial vehicles (UAV) for civilian use in the US. One will operate off the Alaska coast to survey ice floats and wildlife; the other will conduct commercial environmental monitoring in the Arctic Circle, assist emergency response teams in oil spill monitoring and conduct wildlife observations.

Both drones have fixed wings, weigh less than 55 pounds each, and have wingspans of less than 10 feet.

General Atomics Aeronautical Systems – Predator A /B

These are probably the most widely recognizable military robots; their characteristic design frequently appears in pop culture, art and even in protests. Originated by the work of Abe Karem that later developed into DARPA’s project Amber, under the ownership of the Aeronautical division of General Atomics it evolved into a family of UAS. Based on the GNAT drone, the first RQ-1A Predators were initially used for reconnaissance in the Balkan wars during the 90s. The later MQ-1 armed version in 2002 flying over Afghanistan and operated by CIA was the first drone to shoot a missile at a car convoy. MQ-1 is similar in size to a light passenger plane (it has a 6cyl. reciprocating engine) and is still in use today by many countries while its later -C variant (with a diesel engine) recently entered service with US Army.

The similar in concept but much bigger MQ-9 (Predator B according to GAASI or Reaper as called by USAF) is the main large drone currently used by the US in almost every place there is military action. It has a turboprop engine and can carry a bigger payload. A completely new model, the turbofan stealthy Predator C is currently under development. In a similar fashion to its predecessors it is funded internally — an unusual model for the defense sector but financially very successful for the company.

https://www.ga-asi.com/products/aircraft/

Northrop Grumman RQ-4 Global Hawk

Another DARPA project, the RQ-4 was designed by Ryan Aeronautical (later absorbed by Northrop Grumman). It is one of the larger UAVs and it operates in high altitude / long endurance flight profiles. Although it is probably the most capable system of its kind, it is also one of the most expensive to the point that USAF is on the verge of dropping it in favor of the manned U-2 aircraft that it was supposed to replace. However newer versions with more capabilities are still being developed (and even delivered) while an armed version (MQ-4C Triton) is being developed for the US Navy.

http://www.northropgrumman.com/Capabilities/GlobalHawk/Pages/default.aspx

Samsung-Techwin SGR-A1

Military robots are still in their early days and this is a very characteristic example of a system that was developed for very specific needs. The SGR-A1 is an automated gun turret that is designed to replace an armed guard. It has an electro-optical detection system and a machine gun. It can detect a human, shout warning messages and even exchange passwords orally or understand gestures (like raising hands for surrender). Of course it can also shoot bullets, but usually (though not necessarily) a human is in the loop for that decision. It will be fielded along the borders between North and South Korea. The de-militarized zone here isn’t the biggest in the world lengthwise, but due to local tension it is guarded for its entire length.

http://www.samsungtechwin.com/product/product_01_01.asp
http://www.youtube.com/watch?v=pMkV8E2re9U
http://www.globalsecurity.org/military/world/rok/sgr-a1.htm

Northrop Grumman MQ-8 Fire Scout

Fire Scout is one of the first VTOL UAVs. The MQ-8B is an improved and armed version of the initial RQ-8A and both are based on the manned commercial Schweizer helicopters (330 and 333 respectively). MQ-8 is already in use by the US Navy and it had been deployed to Afghanistan and Libya. The next more capable version, the MQ-8C will be based on the Bell 407, a completely different aircraft, providing a good example of the hierarchy inside an unmanned system where the aerial platform isn’t always (or usually) the primary component.

http://www.northropgrumman.com/Capabilities/FireScout/Pages/default.aspx

Lockheed Martin RQ-170 Sentinel

This is probably the first UAV designed to operate in a hostile environment where air-superiority isn’t a given. The RQ-170 was revealed from ‘paparazzi’ photos shot in the Kandahar airfield in 2009 that gave it the “beast of Kandahar” nickname. It was officially declared RQ-170 (an out of line number and a non-too accurate RQ- designation, in similar fashion to the F-117 stealth bomber) but most details were revealed when Iran captured one in late 2011. From the images and video released from Iran media, the RQ-170 appeared to be much smaller than initially estimated and not as high-tech. It has a low rcs design but it appears to be simple in construction, for example it has a mesh in the engine inlet (like the first generation stealth aircraft) instead of a serpentine duct. Even so it is probably the only large UAV that can operate undetected and it is rumored that it was used in the operation that lead to the kill of Osama bin Laden.

http://www.af.mil/information/factsheets/factsheet.asp?id=16001

iRobot Packbot

Packbot is a very successful robot made by iRobot. It uses a patented variable geometry track system that enables it to overcome obstacles like climbing stairs while being very stable. There are many versions and it can be equipped with various accessories like manipulators and cameras. More than 2000 packbots are already in use in several roles such as disposing IEDs, reconnaissance and more.

http://www.irobot.com/us/learn/defense/packbot.aspx

Insitu Scan Eagle

ScanEagle is a military derivative of SeaScan, a small UAV designed by Insitu for the fishing industry. A patented system that uses a catapult with a slingshot for launch, and a wire-hook pole called Skyhook, enable its deployment and recovery from a small ship. This concept is the core of Insitu products and it is very useful for military use as a portable system. Insitu is now owned by Boeing, and although SeaScan wasn’t successful commercially, the Scan Eagle is widely used and it has developed into a family of UAVs that includes the larger ‘integrator’ and others.

http://www.insitu.com/systems/scaneagle

Qinetiq-NA Talon

Talon is a small tracked robot initially developed by Foster-Miller (now part of Qinetiq North America). It is a versatile platform that can carry many sensors and actuators, and it can move through difficult terrain like a small tank. An armed version called SWORDS is equipped with a machine gun, a first for a robot like this, although it is not yet in use.

https://www.qinetiq-na.com/products/unmanned-systems/talon/

Aerovironment RQ-11 Raven

The smallest system in this post, the RQ-11 looks like a small R/C plane. It is hand launched by throwing and it ‘crash’ lands with the wing detaching from the fuselage. It has many capabilities for its size, and can be easily transported. Almost 20 000 have been produced and are in use by many militaries around the world.

http://www.avinc.com/uas/small_uas/raven/