RBO Team from TU Berlin wins the Amazon Picking Challenge at ICRA 2015. Photo Credit: RBO.

Capturing and processing camera and sensor data and recognizing various shapes to determine a set of robotic actions is conceptually easy. Yet Amazon challenged the industry to do a selecting and picking task robotically and 28 teams from around the world rose to it.

Perception isn’t just about cameras and sensors. Software has to convert the data and infer as to what it “sees”. In the case of the Amazon Picking Challenge held last week at the IEEE International Conference on Robotics & Automation (ICRA), each team’s robot was to pick from a shopping list of consumer items of varying shapes and sizes – from pencils, to toys, tennis balls, cookies and cereal boxes – which were haphazardly stored on shelves, and then place their selected items in a bin. They could use any robot, mobile or not, and any arm and end-of-arm grasping tool or tools to accomplish the task.

It’s tricky for robots using sensors to identify and locate objects that can be confused by plastic packaging within the shelf or storage area. Rodney Brooks, of iRobot, MIT and Rethink Robotics fame, often speaks of an industry-wide aspirational goal regarding perception in robotics: “If we were only able to provide the visual capabilities of a 2-year old child, robots would quickly get a lot better.” That is what this contest is all about.

Software has to first identify the item to pick and then figure out the best way to grab it and move it out of the storage area. Amazon, with its acquisition of Kiva Systems, has mastered bringing goods to the picker/packer and now wants to automate the remaining process of picking the correct goods from the shelves and placing them in the packing box, hence their Amazon Picking Challenge.

The top three winners were the teams from Technical University (TU) – Berlin, with 148 points; MIT in 2nd place with 88 points; and the 3rd place finisher (Oakland U and Dataspeed), which only got 35 points. Teams were scored on how many items were correctly selected, picked and placed.

Many commercial companies with proprietary software for just this type of application (such as Tenessee-based Universal Robotics and their Neocortex Vision System, and Silicon Valley startup Fetch Robotics who were premiering their new Fetch and Freight system at the same ICRA conference) chose not to enter because the terms of the challenge included that the software be open sourced.

Team RBO from TU-Berlin wrote their own vision system software and will soon be working on a paper on the subject. They used a Barrett WAM arm because it was the most flexible device for this task amongst the arms that they had access to in their lab. They used a vacuum cleaner tool augmented with a suction cup and a vacuum cleaner to power the suction. For a base, they decided they needed to go mobile and used an old Nomadic Technologies platform, which they upgraded to fit the needs of the contest. Nomadic no longer exists. It was acquired by 3Com in 2000. TU’s photo at right shows robot’s camera view and random placement of items in cubby holes. Watch a video of their winning run here:

Noriko Takiguchi, a Japanese reporter for RoboNews.net, who was at the contest, observed the TU team and said that their approach was torque-based plus position control of the arm and the mobile base, consequently they had good torque control that that gave them flexibility in choosing where to place the suction cup and how much suction to apply.

Team RBO received a 1st place prize of $20,000 plus travel costs for the equipment and team members.

In this episode, Audrow Nash interviews Christoph Stiller from the Karlsruhe Institute of Technology. Stiller speaks about the sensors required for various level of autonomous driving, as well as the ethics of autonomous cars, and his experience in the Defense Advanced Research Projects Agency (DARPA) Grand Challenge.

Christoph Stiller

Christoph Stiller studied Electrical Engineering in Aachen, Germany and Trondheim, Norway, and received the Diploma degree and the Dr.-Ing. degree (Ph.D.) from Aachen University of Technology in 1988 and 1994, respectively. He worked with INRS-Telecommunications in Montreal, Canada for a post-doctoral year as Member of the Scientific Staff in 1994/1995. In 1995 he joined the Corporate Research and Advanced Development of Robert Bosch GmbH, Germany. In 2001 he became chaired professor and director of the Institute for Measurement and Control Systems at Karlsruhe Institute of Technology, Germany.

Dr. Stiller serves as immediate Past President of the IEEE Intelligent Transportation Systems Society, Associate Editor for the IEEE Transactions on Intelligent Transportation Systems (2004-ongoing), IEEE Transactions on Image Processing (1999-2003) and for the IEEE Intelligent Transportation Systems Magazine (2012-ongoing). He served as Editor-in-Chief of the IEEE Intelligent Transportation Systems Magazine (2009-2011). He has been program chair of the IEEE Intelligent Vehicles Symposium 2004 in Italy and General Chair of the IEEE Intelligent Vehicles Symposium 2011 in Germany. His automated driving team AnnieWAY has been finalist in the Darpa Urban Challenge 2007 and winner of the Grand Cooperative Driving Challenge in 2011.

Links:

• Download mp3 (13.1 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Christoph Stiller’s website

Transcript

Audrow Nash:  Hi, welcome to Robots Podcast. Can you introduce yourself?

Christoph Stiller:  My name’s Christoph Stiller. I’m a professor in Mechanical Engineering at Karlsruhe Institute of Technology in Germany.

Audrow Nash:  Can you tell me the goal and motivation behind your research?

Christoph Stiller:  We’re working on autonomous cars, and in particular we focus on computer vision for vehicles (cars that see their environment), and on trajectory planning for vehicles (the decision on where to drive, given the world that the car sees).

Audrow Nash:  What kind of sensors do you use on your vehicles?

Christoph Stiller:  Our main sensors are video cameras, but we also do experiments with LIDAR and radar sensors.

Audrow Nash:  You have a car named Bertha, and it does not have roof-mounted sensors. Can you tell me a bit about that?

Christoph Stiller:  Bertha was developed together with Mercedes Benz and the goal was to use only close-to-market sensors. In particular, we only used cameras that were looking around the vehicle, and radar sensors that came from serious production levels. No expensive GPS, no expensive LIDAR sensors, and in particular, nothing on the roof.

Audrow Nash:  What is the motivation behind not using roof sensors?

Christoph Stiller:  Our other vehicle was driving with roof sensors and those sensors were quite expensive; we had LIDAR sensors that cost us about $70,000 each, and we had a high precision GPS and with an inertial measurement unit and that cost us about the same amount of money on top. That’s far from serious production.

Audrow Nash:  Are there other disadvantages to using roof sensors? For example they get dirty, was that a concern?

Christoph Stiller:  Yes, of course. If a roof sensor is not cleaned it will get dirty, just like how the windshield on your car gets so dirty that you can’t look out anymore after a few hundred kilometers of driving. The same happens to a camera lens: if you drive with your camera lens for a long time, or with LIDAR sensors (which are also optical systems), the lens of that system gets contaminated with dirt or insects, and so the sensor gets blind. In the best case you notice that you can’t drive autonomously anymore.

Audrow Nash:  If your sensors are not on the roof, where are they in the car?

Christoph Stiller:  [Human drivers look out the window. In our system] they’re very close to the roof behind the windshield. When the driver gets irritated by a dirty window, and he engages the wipers, and then our cameras are also cleaned because it’s in the wiper area.

Audrow Nash:  What other sensors do you have in addition to the vision system that watches what’s happening ahead of the car?

Christoph Stiller:  We also have radar sensors. Many cars already have one radar sensor, but we equipped our vehicle with three radar sensors. These came from serious production because we needed the larger viewing angle for those sensors so that we could look into the traffic area and see whether [there was] an oncoming vehicle that we would need to consider.

We also use a standard GPS unit, but it has a precision of about 20 meters, similar to what you have in your smartphone. It’s not an expensive unit, it’s only a low precision unit that gives us a very coarse positioning. We prerecord that tell us where to drive, which lanes we’re supposed to take, what possibilities we have, who has precedence at what intersection, where traffic lights are … all this information is stored in maps.

Audrow Nash:  You mentioned in your abstract that these sensors, the ones you’re using now on Bertha, are close to market. What do you mean by this?

Christoph Stiller:  [The radar sensors are in the market, but not in high numbers.] The video cameras are in the market already, but ours of course require particular algorithms to analyze the data and that’s done on a personal computer in the trunk and of course we would need to bring that down to some embedded system hardware for the processing. That’s something that could be done in a reasonable amount of time.

Audrow Nash:  What are some of the major barriers to bringing these sensors to market?

Christoph Stiller:  The sensors could be brought to market, but the main barriers to bring the whole system to market would be that right now we need a safety driver who could intervene in case of very rare situations that were not predicted. For example if you have an emergency vehicle coming on to you or some other situation that is very difficult to detect, you need human intelligence to keep the vehicle in a safe state.

Audrow Nash:  Can you tell me a bit about the real-time decision-making that your systems do?

Christoph Stiller:  The vehicle first analyses all the sensor signals. It looks first of all where it is on the map with high precision; it uses GPS for a coarse 20-meter precise position, and then it does visual localization, looking for landmarks in the environment such as a window of a house or a tree. Then, like humans do, the system is able to localize itself, except that our visual localization system is very precise: we get a precision of about five-centimeter accuracy, so we know with high accuracy where we are. Then we look at the map to see what lane we are in, the lane boundaries and any obstacle that is in our driving corridor (such as other cars, pedestrians, bicyclists, parked cars or any other static obstacles), and then we plan a path that does not collide with any static or moving obstacles if possible.

We first try to stay in our lane, but if that’s not possible we have a decision unit that allows us to move to another lane. This might be an adjacent lane that moves in the same direction if there’s a gap, and if that’s not possible either, we could even decide to go into the [opposing] lane if there is no vehicle coming in that direction. If none of this is possible, of course, we’ll stay and wait until the situation resolves.

Audrow Nash:  In the real world, the maps that you correlate all of your sensor data with are sometimes unreliable. Can you talk about those challenges?

Christoph Stiller:  Right now if our map is wrong, for example a new construction zone is built in the area, our supervising driver has to react. We have a safety driver onboard who doesn’t do anything, except in an unexpected situation like when the map is wrong, or when we have an emergency vehicle. Of course [in the marketplace] you can’t have a safety driver delivered with every car, and so the long-term goal is then to keep the map up to date, and that will be done with community mapping. There will be a whole crowd of cars that are equipped with sensors that communicate slight changes in the map to an infrastructure server, and the infrastructure server would accumulate that information. If many vehicles [report a change to the map], let’s say lane has moved slightly due to a construction, then the map will signal that to the vehicles that follow.

Each vehicle has an accurate map and an up-to-date map of the environment.

Audrow Nash:  So it’s essentially crowdsourcing the development of the map for all the cars to pull from?

Christoph Stiller:  Yes, it’s a crowdsourcing method.

Audrow Nash:  What was your experience with the grand cooperative driving challenges, the series of them?

Christoph Stiller:  There’s a large series that was started in the US by DARPA in 2004 and 2005. These were grand challenges where the goal was to drive through the desert autonomously. Then in 2007 there was an urban challenge that was through an area with low houses, almost no trees, very few traffic lights, very few traffic signs, and no bicycles. So it was not a fully normal traffic situation but more a mock-up of that.

In 2011 there was the first cooperative driving challenge, which took place in Holland. The goal there was to drive in a tight platoon of vehicles, keeping a very short distance – 6 meters – [between the cars]. If you are driving 100km/h that is about 34 meters per second, and that means it’s 0.2 seconds [between you and] your predecessor. Of course it’s impossible for a human driver to react in such a short time, so these vehicles could only be driven autonomously.

The first vehicle would [break hard], and all the other vehicles then of course had to react in real-time to avoid collision. The vehicles needed to communicate – that’s why it was called cooperative challenge. They had to communicate their position and velocity, and if they wanted to, they could also communicate their acceleration, which of course is highly beneficial to following vehicles.

Audrow Nash:  What were some of the major challenges you encountered in the DARPA cooperative driving challenge?

Christoph Stiller:  The major challenges were the short reaction time, and the fact that the teams were heterogeneous. It was not a system approach, developing one control law for all the vehicles, which would then have a guaranteed stability for the whole platoon. We didn’t know the control laws of the other [vehicles]. The challenge was to have a very robust system that could drive safe without knowing what strategy other [vehicles] have.

Audrow Nash:  What did you learn from this?

Christoph Stiller:  We learned that it was possible in such scenarios to damp very severe oscillations, allowing the first car to [break almost to a full stop] and have all the vehicles in the platoon (from small cars to trucks) avoid a collision through timely braking and communication. We learned that communication could help [prevent] what we call shockwaves in traffic. [This can happen] when one car brakes in a platoon, and each following car has to brake stronger than its predecessor and eventually either one car can’t break as strong as it should, so there’s a collision, or it comes to full stop and then you have traffic congestion.

Audrow Nash:  I’m interested in your perspective on the ethics of driving autonomous vehicles. What would you think about the delegation of responsibility between the driver and the manufacturers of the autonomous car sensors?

Christoph Stiller:  That will shift. Right now for most accidents, the car driver is responsible, so most accidents in traffic are due to human fault. A very small number of traffic accidents come from the responsibility of construction fault by the car manufacturer. Of course, the more autonomous maneuvers introduced into the market, the greater the likelihood that one of those maneuvers is ill-designed and could cause a crash. In that case of course the car manufacturer would be liable. If in the long term the vehicle is fully autonomous, so that [a person] could read a newspaper, then of course the car manufacturer is responsible for all the traffic accidents that happens while he’s driving autonomously.

Audrow Nash:  So, while the user is driving, they’re responsible if the car gets in an accident, but when when the autonomous car takes over, then it’s me manufacturer’s liability?

Christoph Stiller:  Yes, that’s my understanding of liability. That you’re only liable for what you really do. If the car manufacturer automates a car, then of course the driver to my understanding cannot be responsible. There maybe a situation, with highly automated cars where the driver is allowed to read a newspaper but before some situation comes that cannot be handled by the car like a construction site, the vehicle tells the driver, “You have to take over in 10 seconds,”  … then of course for the portion where the vehicle drives autonomously, the car manufacturer is responsible, and for the [portion] that the driver drives manually, the driver is responsible for his driving. And in between, at the times when the vehicle says you have to take over, this is a responsibility of the driver to be ready to take over; he can’t sleep in the rear seat or be drunk. He has to be ready to take over in such a situation.

Audrow Nash:  What kind of timeline do you believe we will have for autonomous cars?

Christoph Stiller:  This is a very difficult question. We do have some autonomous functions in the market already. Most [high end] cars already do automated emergency braking before an imminent collision, and for pedestrians they even do automated evasive steering. Those functions will merge. Today there are very rare situations where the vehicle is certain that it can handle the situation better than the driver; the number of these situations will grow, the vehicle will get into more and more situations and understand them better than the driver, and if it understands that the situation is critical, at least then it will take over and avoid a collision, or at least minimize the velocity of the collision and therefore the severity of the collision. Eventually all those functions will come to the point where the car can drive autonomously everywhere and any time and the driver doesn’t even need to be in the car, but that will take a long time.

Audrow Nash:  I know there are many major challenges before autonomous cars are integrated into society. What do you believe is one of the big ones?

Christoph Stiller:  In my experience, the most difficult thing is to understand the situation correctly. An example would be to consider a pedestrian walking to the curb and stopping for a moment. A human driver can predict, with very high precision, whether or not the pedestrian will walk on the road or will stop and wait for the vehicle to pass. You look at whether a pedestrian looks towards you, you consider their age, and you consider whether the person is wearing earphones and not paying attention. Taken together, all those small factors together tell you whether you should break, blow your horn, or just drive through. To do the same would be very difficult with the current state-of-the-art autonomous vehicle.

Audrow Nash:  Wrapping up, what do you think is the future of robotics?

Christoph Stiller:  As far as cars are concerned, I’m certain that autonomy will come, and in parallel, we will see cooperation, with vehicles communicating with each other and using that to harmonize their trajectories. Cars will follow each other at a closer distance and have maneuvers that are communicated with each other, so they will know exactly what the others are doing. That would harmonize traffic flow, and of course improve safety. In the very long term, I expect that traffic would then be harmonized so much that the flow of traffic would look like a fish swarm movement rather than the chaotic movement that we have on the roads today.

*Transcripts are edited for clarity with great care. However, we cannot assume any responsibility for their accuracy.

Skysense is a Berlin-based start up designing and manufacturing a range of products to enable the remote operation of drones. Their first two products are the Skysense Charging pad and Skysense Hangar. 

The Skysense Charging Pad is a charging pad for drones that can be remotely operated, it provides fast charging and wide landing area, it supports nearly all existing multicopters and VTOL aircrafts and it can charge multiple drones at the same time.

The Skysense Hangar is a protective hangar to store and recharge the drone between flights.

These products are the start of the infrastructure needed if you are planning to maintain routine operations. Now Skysense have launched a range of multicopters designed for professional photographers. The SS700 hexacopter is the entry level model starting from €1,498. The SS750 Hexacopter is the mid level model starting from €2.998. The SS900 Octocopter is the top level model and starts from €4,498.

To celebrate Skysense has launched this showreel:

 Voith GmbH, among the biggest family-owned businesses in Europe, acquired 25% of robot maker Kuka AG by buying the shares held by Grenzebach GmbH.

The transaction was valued at €555 million ($683 million), based on Bloomberg closing share prices.

KUKA has been active financially this last year:

• KUKA acquired the engineering services, toolmaking and integration divisions of US-based Utica Enterprises for an estimated $25 million in May, 2013. 300 people were involved in the acquisition.
• KUKA acquired 51% of the shares of Reis Robotics, a 57-year old family-owned Germany-based robotics provider and systems integrator with sales of $178 million, in December, 2013. 1,300 employes worked for Reis.
• KUKA acquired Alema Automation SAS, a French automation solutions provider to the aviation industry, in March, 2014, for an undisclosed amount.
• KUKA purchased Swisslog, a healthcare and materials handling robot company, for $378 million in October, 2014. Grenzebach GmbH was a 30% shareholder of Swisslog at the time of the acquisition.

Till Reuter, CEO of KUKA, said: “This is good news for our employees, as this new anchor shareholder will provide us with a stable basis for further growth.”

“Kuka is focused on mechanics, electronics, sensor systems and software,” Voith Chief Executive Officer Hubert Lienhard said. “This makes it an ideal, forward-looking investment for us.

Will there be a consolidation of KUKA, Grenzebach and Swisslog mobility R&D and products? And will there be a mobile manipulator robot in the near-term future using KUKA’s new lightweight IBR iiwa arm and Swisslog’s and Grenzebach’s consolidated mobile platform? Interesting times …

If you liked this article, you may also be interested in:

• KUKA acquires Swisslog for $357 million
• KUKA buys 51% of Reis Robotics with option to buy remainder
• KUKA opens Shanghai factory and announces orders
• KUKA acquires US integrator and toolmaker

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

DHL launches first authorized autonomous delivery flights by “parcelcopter” to and from the North Sea island of Juist, 7.5 miles off the coast of Northern Germany.

The DHL parcelcopter has been approved for testing delivery of medications and other urgently needed goods to the small North Sea island of Juist – one of a string of islands just off the coast of Northern Germany and The Netherlands. Daily deliveries began today.

DHL is working together with RWTH Aachen University and Microdrones GmbH along with the German Ministry of Transport. This research project represents the first and only time in Europe that a flight by an unmanned aircraft will be operated outside of the pilot’s field of vision in a real-life mission.

The research team has optimized the copter and software for flight duration, flight range and speed to address the special challenges posed by the wind and marine-weather conditions of the North Sea.

“Our DHL parcelcopter 2.0 is already one of the safest and most reliable flight systems in its class that meets the requirements needed to fulfill such a mission,” said Jürgen Gerdes, CEO of Deutsche Post DHL’s Post – eCommerce – Parcel Division. “We are proud that this additional service can create added value for the residents of and visitors to the island of Juist and are pleased with the support we have received from the involved communities and agencies.”

Read the full press release here.