Michael Aeberhard: Automated Driving with ROS at BMW. Source: OSRF Blog

ROSCon is an annual conference focused on ROS, the Robot Operating System. Every year, hundreds of ROS developers of all skill levels and backgrounds, from industry to academia, come together to teach, learn, and show off their latest projects.

Here is the next set of posts from the OSRF blog, along with videos.

Amit Moran (Intel): Introducing ROS-RealSense: 3D Empowered Robotics Innovation Platform

While Intel is best known for making computer processors, the company is also interested in how people interact with all of the computing devices that have Intel inside. In other words, Intel makes brains, but they need senses to enable those brains to understand the world around them. Intel has developed two very small and very cheap 3D cameras (one long range and one short range) called RealSense, with the initial intent of putting them into devices like laptops and tablets for applications such as facial recognition and gesture tracking.

Robots are also in dire need of capable and affordable 3D sensors for navigation and object recognition, and fortunately, Intel understands this, and they’ve created the RealSense Robotics Innovation Program to help drive innovation using their hardware. Intel itself isn’t a robotics company, but as Amit explains in his ROSCon talk, they want to be a part of the robotics future, which is why they prioritized ROS integration for their RealSense cameras.

A RealSense ROS package has been available since 2015, and Intel has been listening to feedback from roboticists and steadily adding more features. The package provides access to the RealSense camera data (RGB, depth, IR, and point cloud), and will eventually include basic computer vision functions (including plane analysis and blob detection) as well as more advanced functions like skeleton tracking, object recognition, and localization and mapping tools.

Intel RealSense 3D camera developer kits are available now, and you can order one for as little as $99.

Michael Aeberhard (BMW): Automated Driving with ROS at BMW

BMW has been working on automated driving for the last decade, steadily implementing more advanced features ranging from emergency stop assistance and autonomous highway driving to fully automated valet parking and 360° collision avoidance. Several of these projects were presented at the 2015 Consumer Electronics Show, and as it turns out, the cars were running ROS for both environment detection and planning.

BMW, being BMW, has no problem getting new research hardware. Their latest development platform is the 335I G. This model comes with an advanced driver assistance system based around cameras and radar. The car has been outfitted with four low-profile laser scanners and one long-range radar, but otherwise, it’s pretty close (in terms of hardware) to what’s available in production BMWs.

Why did BMW choose to move from their internally developed software architecture to ROS? Michael explains how ROS’ reputation in the robotics research community prompted his team to give it a try, and they were impressed with its open source nature, distributed architecture, existing selection of software packages, as well as its helpful community. “A large user base means stability and reliability,” Michael says, “because somebody else probably already solved the problem you’re having.” Additionally, using ROS rather than a commercial software platform makes it much easier for BMW to cooperate with universities and research institutions.

Michael discusses the ROS software architecture that BMW is using to do its autonomous car development, and shows how the software interprets the sensor data to identify obstacles and lane markings and do localization and trajectory planning to enable full highway autonomy, based on a combination of lane keeping and dynamic cruise control. BMW also created their own suite of RQT and rviz plugins specifically designed for autonomous vehicle development.

After about two years of experience with ROS, BMW likes a lot of things about it, but Michael and his team do have some constructive criticisms: message transport needs more work (although ROS 2 should help with this), managing configurations for different robots is problematic, and it’s difficult to enforce compliance with industry standards like ISO.

Jerry Towler (SwRI): Mapviz – An Extensible 2D Visualization Tool for Automated Vehicles

ROS already comes with a fantastic built-in visualization tool called rviz, so why would you want to use anything else? At Southwest Research Institute, Jerry Towler explains how they’ve created a new visualization tool called Mapviz that’s specifically designed for the kind of large-scale outdoor environments necessary for autonomous vehicle development. Specifically, Mapviz is able to integrate all of the sensor data that you need on top of a variety of two-dimensional maps, such as road maps or satellite imagery.

As an autonomous vehicle visualization tool, Mapviz works just like you’d expect that it would, which Jerry demonstrated with several demos at ROSCon. Mapviz shows you a top-down view of where your vehicle is, and tracks it across a basemap that seamlessly pulls image tiles at multiple resolutions from a wide variety of local or networked map servers, including Open MapQuest and Bing Maps. Mapviz is, of course, very plugin-friendly. You can add things like stereo disparity feeds, GPS fixes, odometry, grids, pathing data, image overlays, projected laser scans, markers (including textured markers) from most sensor types, and more. It can’t really handle three dimensional data (although it’ll do two-and-a-half dimensions via color gradients), but for interactive tracking of your vehicle’s navigation and path planning behavior, Mapviz should offer most of what you need.

For a variety of non-technical reasons, SwRI hasn’t been able to release all of its tools and plugins as open source quite yet, but they’re working on getting approval as fast as they can. They’re also in the process of developing even more enhancements for Mapviz, and you can keep up to date with the latest version of the software on GitHub.

Matt Vollrath and Wojciech Ziniew (End Point): ROS-Driven User Applications in Idempotent Environments

Matt Vollrath and Wojciech Ziniew work at an ecommerce consultancy called End Point, where they provide support for Liquid Galaxy; a product that’s almost as cool as it sounds. Originally an open source project begun by Google engineers on their twenty percent time, Liquid Galaxy is a data visualization system consisting of a collection of large vertical displays that wrap around you horizontally. The displays show an immersive (up to 270°) image that’s ideal for data presentations, virtual tours, Google Earth, or anywhere you want a visually engaging environment. Think events, trade shows, offices, museums, galleries, and the like.

Last year, End Point decided to take all of the ad hoc services and protocols that they’d been using to support Liquid Galaxy and move everything over to ROS. The primary reason to do this was ROS support for input devices: you can use just about anything to control a Liquid Galaxy display system, from basic touchscreens to Space Navigator 3D mice to Leap Motions to depth cameras. The modularity of ROS is inherently friendly to all kinds of different hardware.

Check out this week’s ROSCon15 video as Matt and Wojciech take a deep dive into their efforts in bringing ROS to bear for these unique environments.

Tom Moore: Working with the Robot Localization Package

Clearpath Robotics is best known for building yellow and black robots that are the research platforms you’d build for yourself; that is, if it wasn’t much easier to just get them from Clearpath Robotics. All of their robots run ROS, and Clearpath has been heavily involved in the ROS community for years. Now with Locus Robotics, Tom Moore spent seven months as an autonomy developer at Clearpath. He is the author and maintainer of the robot_localization ROS package, and gave a presentation about it at ROSCon 2015.

robot_localization is a general purpose state estimation package that’s used to give you (and your robot) an accurate sense of where it is and what it’s doing, based on input from as many sensors as you want. The more sensors that you’re able to use for a state estimate, the better that estimate is going to be, especially if you’re dealing with real-worldish things like unreliable GPS or hardware that flakes out on you from time to time. robot_localization has been specifically designed to be able to handle cases like these, in an easy to use and highly customizable way. It has state estimation in 3D space, gives you per-sensor message control, allows for an unlimited number of sensors (just in case you have 42 IMUs and nothing better to do), and more.

Tom’s ROSCon talk takes us through some typical use cases for robot_localization, describes where the package fits in with the ROS navigation stack, explains how to prepare your sensor data, and how to configure estimation nodes for localization. The talk ends with a live(ish) demo, followed by a quick tutorial on how to convert data from your GPS into your robot’s world frame.

The robot_localization package is up to date and very well documented, and you can learn more about it on the ROS Wiki.

Moritz Tenorth (Magazino): Maru and Toru — Item-Specific Logistics Solutions Based on ROS

It’s not sexy, but the next big thing for robots is starting to look like warehouse logistics. The potential market is huge, and a number of startups are developing mobile platforms to automate dull and tedious order fulfillment tasks. Transporting products is just one problem worth solving: picking those products off of shelves is another. Magazino is a German startup that’s developing a robot called Toru that can grasp individual objects off of warehouse shelves, a particularly tricky task that Magazino is tackling with ROS.

Moritz Tenorth is Head of Software Development at Magazino. In his ROSCon talk, Moritz describes Magazino’s Toru as “a mobile pick and place robot that works together with humans in a shared environment,” which is exactly what you’d want in an e-commerce warehouse. The reason that picking is a hard problem, as Moritz explains, is perception coupled with dynamic environments and high uncertainty: if you want a robot that can pick a wide range of objects, it needs to be able to flexibly understand and react to its environment; something that robots are notoriously bad at. ROS is particularly well suited to this, since it’s easy to intelligently integrate as much sensing as you need into your platform.

Magazino’s experience building and deploying their robots has given them a unique perspective on warehouse commercialization with ROS. For example, databases and persistent storage are crucial (as opposed to a focus on runtime), and real-time control turns out to be less important than being able to quickly and easily develop planning algorithms and reducing system complexity. Software components in the ROS ecosystem can vary wildly in quality and upkeep, although ROS-Industrial is working hard to develop code quality metrics. Magazino is also working on remote support and analysis tools, and trying to determine how much communication is required in a multi-robot system, which native ROS isn’t very good at.

Even with those (few) constructive criticisms in mind, Magazino says that ROS is a fantastic way to quickly iterate on both software and hardware in parallel, especially when combined with 3D printed prototypes for testing. Most importantly, Magazino feels comfortable with ROS: it has a familiar workflow, versatile build system, flexible development architecture, robust community that makes hiring a cinch, and it’s still (somehow) easy to use.

Michael Ferguson (Fetch Robotics): Accelerating Your Robotics Startup with ROS

Michael Ferguson spent a year as a software engineer at Willow Garage, helping rewrite the ROS calibration system, among other projects. In 2013, he co-founded Unbounded Robotics, and is currently the CTO of Fetch Robotics. At Fetch, Michael is one of the primary people responsible for making sure that Fetch’s robots reliably fetch things. Mike’s ROSCon talk is about how to effectively use ROS as an integral part of your robotics business, including best practices, potential issues to avoid, and how you should handle open source and intellectual property.

Because of how ROS works, much of your software development (commercial or otherwise) is dependent on many external packages. These packages are constantly being changed for the better — and sometimes for the worse — at unpredictable intervals that are completely out of your control. Using continuous integration, consisting of systems that can handle automated builds, testing, and deployment, can help you catch new problems as early as possible. Michael also shares that a useful way to avoid new problems is to not immediately switch over to new software as soon as they are available: instead, stick with long-term support releases, such as Ubuntu 14.04 and ROS Indigo.

While the foundation of ROS is built on open source, using ROS doesn’t mean that all of the software magic that you create for your robotics company has to be given away for free. ROS supports many different kinds of licenses, some of which your lawyers will be more happy with than others, but there are enough options with enough flexibility that it doesn’t have to be an issue. Using Fetch Robotics as an example, Mike discusses what components of ROS his company uses in their commercial products, including ROS Navigation and MoveIt. With these established packages as a base, Fetch was able to quickly put together operational demos, and then iterate on an operating platform by developing custom plugins optimized for their specific use cases.

When considering how to use ROS as part of your company, it’s important to look closely at the packages you decide to incorporate, to make sure that they have a friendly license, good documentation, recent updates, built-in tests, and a standardized interface. Keeping track of all of this will make your startup life easier in the long run. As long as you’re careful, relying on ROS can make your company more agile, more productive, and ready to make a whole bunch of money off of the future of robotics.

If you enjoyed this article, you may also want to read:

• ROSCon 2015 recap and videos – Part 4
• ROSCon 2015 recap and videos – Part 3
• ROSCon 2015 recap and videos – Part 2
• ROSCon 2015 recap and videos – Part 1

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

OATV

Interview with former OATV Principal Roger Chen

Edited for clarity

What’s OATV’s investment thesis and how does that make you look at robotics companies?

Roger Chen, former OATV Principal

We like to look on the edge. If you take a look at our portfolio, there’s a wide array of companies in different categories. You have anything from consumer internet companies like Foursquare to satellite companies like Planet Labs to drone companies like 3D Robotics and logistics companies like Fetch Robotics. We made each investment when we sensed the emergence of a new category. When we think a “thing” is going to become a “thing,” we try to find companies and entrepreneurs in those categories very early on and back them just before the categories are really created. Before 3D Robotics, drones wasn’t really much of a category. And neither was space before Planet Labs and other pioneering companies like SkyBox.

Our focus has been on that strategy applied at the seed stage. We’re a little bit different than other seed firms in that we invest in fewer companies, about six a year. We concentrate more capital into those companies and take more of an all-in approach. Our personal philosophy and style is to try to work more closely with the companies we invest in, and it becomes hard and unwieldy to do that if you invest in too many companies within one year.

When it comes to robotics, we are seeing a lot of interesting things happen. There has been a confluence of technologies enabling new forms of robotics, from innovations in actuators enabling compliance and collaborative robotics to innovations in sensors and software.

ROS, or Robot Operating System, only emerged these last few years, and before that, software development for robotics was exceedingly difficult. The advent of open source communities and platforms like ROS has really catalyzed the field. So those are some of the enablers on the technology side.

There’s also a lot happening on the market side, and market pull is just as important as technology. Just to give one example, let’s take a look at e-commerce and what’s happening with consumers. They want things faster, cheaper, and personalized, and this just creates so much pressure on a lot of these manufacturing, supply chain, and logistics companies.

At the same time, there are macro trends within the labor economy, as baby boomers start getting older and labor supply is expected to drop significantly. The confluence of all these factors puts logistics companies in a tough spot. People tend to forget that someone somewhere still needs to make, package, and ship things as part of e-commerce’s backend. Suddenly, automation and robots make a lot of sense and are economical.

That’s how we see the robotics industry, and while that example is specific to the logistics industry, I think there are a lot of industries where automation and robotics are going to come into play in similar ways. It’s going to be a collision between technology enablers on one side and intense emerging market demand on the other.

Could you expand on some of the trends enabling robotics and automation in logistics?

I’ll talk a little more about market pull. Depending on country, online sales make up somewhere around 10% of overall retail. You can see how massive companies like Amazon and Alibaba are, and that 10% online share of retail will only continue to grow. It speaks to the growing volume of work that has to be fulfilled on the backend of e-commerce.

I think a lot of people, especially consumers, don’t see how much work has to go into fulfilling those online orders. There is a box that has to be moved. There is something that has to be packed. There is something that has to shipped and transported. That’s kind of shielded away from consumers’ eyes. But it all has to be done, and it’s becoming more and more challenging for logistics companies to fulfill all these operations economically.

I can give you another statistic on the labor side. A lot of people have concerns about how robotics and automation will disrupt labor, which I think is valid and true to a certain extent. But I think you also have to be nuanced about it because if you actually look at the manufacturing and materials handling industry, particularly in the US, there is a huge job gap of 600,000 people because there is not enough sufficiently skilled labor to execute on the work to be done. That then presents an opportunity for robotics and automation to come in and fill that gap.

These are the powerful forces we see driving robotics: the really intense demand for logistics fulfillment, and simultaneously a lack of people to do all the work at an economical cost.

What other areas of the commerce value chain can robotics, smart automation and AI potentially improve?

There’s a ton of room for cost reduction via automation, but it’s not necessarily with just physical automation. There are software solutions as well that can make supply chains a lot more productive. For example, our portfolio company Fetch Robotics is tackling the logistics problem by streamlining operations in factories and warehouses with a mobile robotics platform. However, Fetch will be just as much about its future operations management software and application data as its material handling robots.

At some point, all the goods that a company like Fetch moves around will need to be packaged into containers to be shipped all around the world. Another OATV company called Haven is creating a marketplace for streamlining how container shipping is booked. If you think about it, it’s rather ridiculous that people still have to call one another and use manual paper-based processes to mix and match which containers should go on which ship. It’s just very inefficient, and it hurts business by not maximizing fulfillment of shipping capacity. This is a case where relatively simple automation through purely software and a web application can go a very long way in driving up productivity in the supply chain.

The take home message here is that when I think about automation for improving supply chains, it’s not necessarily just robots with arms that move around and pick things up. It’s as much about the software as it is about the hardware.

What are some examples of OATV portfolio companies, perhaps not robotics companies but where the lessons can be applied to robotics?

I just talked about Haven a little bit – it’s essentially an online marketplace for more efficiently filling capacity on ships for shipping things.

There are a couple other OATV companies that come to mind. One is Sight Machine. It’s
a data platform company. They aggregate data streams across the manufacturing floor, perform analytics on them, and offer a frontend dashboard for customers to understand exactly what’s happening along their manufacturing lines. That has a ton of value because that kind of intelligence is what will allow decision makers overseeing operations to keep things up and running efficiently.

https://www.youtube.com/watch?v=x8hOqzBFkRk

Another example is Riffyn. They are similar to Sight Machine in that they are also a data platform that aggregates data streams, but their focus is on the R&D lab for life science companies. Currently 70-90% of R&D results in the life sciences are not reproducible. If you are a pharma company, imagine the egregious amounts of R&D money wasted due to poor process control. Companies like Riffyn perform data automation to give science-driven enterprises control of their processes again. They automate data collection, root cause analysis and continuous deployment of improved process designs to drive up productivity for the R&D pipeline.

Both Sight Machine and Riffyn automate workflows to enable superior operational intelligence, flexibility, and performance to drive up productivity. While neither company is a robotics company in the traditional sense, that exact value proposition very much applies to robotics as well.

How is robotics today different from robotics in the past?

Once upon a time, robotics was about stationary, highly repetitive, high performance, and generally expensive automation. Industrial robot arms would repeat the same action again and again with extreme precision. We’re not as excited about those applications. We think a lot more about flexible robotics. We think about programmable robotics.

It’s no longer just about robotics automating specific human tasks. It’s more about how flexible robotics will enable superior operations overall. I would actually draw a parallel with what’s happening in the software world with the DevOps movement, which I think we’re shortly going to see in the physical world. With DevOps, the idea is to leverage a closed feedback loop to quickly iterate on software development, deployment and operations in a continuous fashion. I think this concept of continuous development and deployment will extend beyond virtual environments to optimize physical operations as well.

Companies like Fetch Robotics have a physical hardware platform that automates some process, which it will do well, but probably not perfectly at first. However, it will continuously improve and learn about the process because of all the operational data and analytics that result from prior deployments. That data will come back to decision makers and engineers and inform them how to redesign and improve process. Now, imagine a simple firmware or a software push that instantly reorganizes and improves operations

That’s the sort of robotics we’re seeing these days, and I think that’s what makes these companies really exciting. Robots are no longer one-trick ponies that just automate a single task.

Do you think these trends are leading to a change in the business model away from robots as capital expenditure? Will we see robots as a service? Or robots as a delivery mechanism for sensors and analytics?

At the end of the day, the decision to use robotics is still at its core a question of ROI. For some companies the ROI math doesn’t add up, but for an increasing number of companies, I think it does.

I do think though that there will be some interesting and innovative business models for robotics. How will future robotics products be priced? Some companies will make money selling hardware like more traditional industrial robotics companies. A company like Fetch Robotics makes a physical product that it will sell, but at the same time, there are potential SaaS-like revenue streams through the data applications and operations software that the company will offer. I’m definitely excited to see what new business models come out of all this.

And in conclusion?

I’m excited about this new wave of robotics companies evolving from traditional robotics that have primarily been about static, repetitive processes. We are witnessing the emergence of the first platforms to offer flexible and programmable robotics that we haven’t seen before.

I’m also really excited about learning what other verticals robotics will address. Most of the conversation about robotics has been centered on logistics and e-commerce, but there will be several other verticals as well. Applications like exploration, search and rescue, caregiving and more. I can’t wait.

The rest of our free report is available here — or in installments at Silicon Valley Robotics — featuring case studies and analysis from industry experts and investors.

If you enjoyed reading this post, you may also like..

• SVR Case Studies: Fetch Robotics bringing warehouses to life
• SVR Case Studies: Introduction to service robots
• SVR Case Studies: Lessons for robotics from enterprise computing
• SVR Case Studies: Hardware may be hard, but strategic investing can make it smart
• SVR Case Studies: Adept’s versatile mobile service platform

Transcript below

In this episode, Audrow Nash interviews several companies from the International Conferences on Robotics and Automation (ICRA) showcase. The companies span the following applications: mobile robots for military and commercial uses, warehouse solutions, robotic arms and manipulators, and robotic systems to assist surgery.

Companies are listed in the order of their interview with a picture of their booth at ICRA 2015. All photos were taken by Robohub and can be found here.

American Robot Company (AMBOT) specializes in advanced electronics, unmanned systems, autonomous vehicle solutions and intelligent platforms for research, commercial and military robotic applications.

Fetch Robotics provides warehouse solutions.

Intuitive Surgical designs and builds the da Vinci Surgical System for robotically assisted minimally invasive surgery.

Kinova designs and manufactures robotics platforms and components that are simple and safe under two business units: Assistive Robotics for  people with disabilities and Service Robotics for humans and robots working in the same environments.

Applied Dexterity seeks to advance the field of robotically assisted surgery by creating a research surgical robot that can serve as a standard platform allowing researchers to share their software and improvements.

Interviewees

Jens Hurley is a Senior Roboticist at the American Robot Company.

Michael Ferguson is a Co-Founder and the Chief Technical Officer (CTO) of Fetch Robotics. Previously, Ferguson was the founder and CTO of Vanadium lab, developing low-cost education and hobby robot controllers, and a Software Engineer at Willow Garage.

Simon DiMaio is a Senior Manager of Research and Advanced Systems Development at Intuitive Surgical. Previously, DiMaio was an Instructor of Radiology and Postdoctoral Research Fellow at Harvard Medical School.

François Boucher is the Vice President of Business Development at Kinova Inc. in Boisbriand, Canada. He received his MBA and a Bachelor degree in Physics Engineering from Laval University. After his graduation, he worked as the General Manager of a technology transfer and investment company before joining Kinova in its early days to develop the service robotics market. In 2014 and 2015, Kinova was listed as one of the 50 most influential public & private companies in the global robotics industry by the Robotics Business Review and one of the fastest-growing company by Profit 500 in Canada.

Andrew Lewis is a Roboticist at Applied Dexterity. Previously, Lewis was a mechanical engineering graduate student in the BioRobotics lab at the University of Washington. His work in the BioRobotics Lab focused on the development of surgical robots, dynamically evaluated gravity compensation, and electromyography controlled iRobot Roombas. Lewis earned his BS in Robotics Engineering from Worcester Polytechnic Institute in 2011. His interests include mechanical design, systems development, and ethics in robotics engineering.

Links:

• Download mp3 (22.3 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Transcript

Audrow Nash: Hi, can you introduce yourself.

Jens Hurley: My name is Jens Hurley. I’m a senior roboticist at AMBOT.

Audrow Nash: Can you tell me what you have set up at this booth?

Jens Hurley: Our booth is based on research mobility platforms. We’re doing a lot of ground robotics with a lot of autonomous function built into them.

Audrow Nash:This one right here, can you describe it a bit and then talk about its applications?

Jens Hurley: Absolutely, this guy is a ruggedized outdoor, high-speed, mobility platform. It weighs about 525 pounds. It can carry about 550 pounds, so it’s got a one-to-one payload ratio. It’s got four-wheel, steer independent suspension and four-wheel drive so it’s highly manipulative which makes it great for research applications. It also served in the military for a light and load application and medical evac. It has high-powered lithium ion batteries and a 8-10 hour runtime over difficult terrain.

Audrow Nash: How large is it and how tall?

Jens Hurley: Overall, I believe it’s about 52 inches long, about 42 inches wide and its ground clearance is about eight inches. The whole vehicle itself is about 19 inches from the ground.

Audrow Nash: It’s a large remote operated vehicle?

Jens Hurley: It can be. This particular one is set up for remote, a man in the loop. We also offer systems with autonomous functionality, like soldier following or vision based systems.

Audrow Nash: I see, and so it has these large carbon fibre capsules on its side. What do those do?

Jens Hurley: Those are actually our battery packs. They are quickly removable.

Audrow Nash: Why electric?

Jens Hurley: Silent operation, clean, quiet, overall a very nice function to have. A lot of applications in military require silent functionality.

Audrow Nash: What’s the max distance it could travel? Do you have an idea?

Jens Hurley: No, I couldn’t tell you the max distance. Runtime dependent about seven miles per hour for 10 hours, moving continuously.

Audrow Nash: 70 miles, okay. This vehicle looks like it has a very heavy suspension system.

Jens Hurley: It does.

Audrow Nash: Can you talk a bit about that?

Jens Hurley: Absolutely, it was developed to carry a high payload, heavy weight, under almost any conditions, so it’s got six inches of independent suspension at each corner which makes it contact with the ground. It’s very rugged, very durable.

Audrow Nash: How fast does it go?

Jens Hurley: This particular model has a top speed of about 40 miles per hour.

Audrow Nash: At those speeds why is good suspension valuable?

Jens Hurley: It’s key. If the payload’s too high – high centre of gravity – it’s just not a good thing to have at high speeds. The active suspension keeps the payload low, very stable, it’s physics.

Audrow Nash: Yes. Are there governments interested in this or are there other applications?

Jens Hurley: This particular robot here was developed for a military function, basically, to lighten the load of a soldier. The European space agency is using it for teleop, and universities such like John Hopkins and the Applied Physics Laboratory, are using it.

Audrow Nash: What do you mean by teleop?

Jens Hurley: It’s basically a man in the loop. So they’ll be operating from miles away to perform some sort of action.

Audrow Nash: Okay, so teleop and research applications. What about agriculture?

Jens Hurley: It’s something we’re looking into. There’s been a lot of interest here in agricultural purposes.

Audrow Nash: I see. What’s the purpose of the buttons?

Jens Hurley: It’s pretty standard that every robot comes with an emergency shut off: the red buttons. The rest of these are there for easy setup and control when you get it out of the package. A lot of the customers, when they get it up and going, move the onboard control system because they’re going to be controlling it from a distance.

Audrow Nash: I don’t really see how you put a payload on it, so describe it a little bit more. The wheels are a little bit taller than the body and the battery packs are a little higher than the body. Do you mount some sort of platform on it and then use the payload that way?

Jens Hurley: It depends on the application. A lot of payloads are being used just with manipulative arms and torsos and they do fit on the current payload space. If it’s something larger, like a palette size, we do offer mounting. There are hard mounting points located at each corner of the vehicle where we mount racks to elevate it up and off.

Audrow Nash: What about repair in the field?

Jens Hurley: Depending on what exactly has just broken down, it’s a very simple platform. Everything is open architecture, so it’s easy to get into, easy to repair. It’s a very straightforward, simple machine.

Audrow Nash: Are there standards in creating this? Do you use, for instance, one single size screw for accessing almost everything, or is there one tool to take apart most of it? Is there any modularity built in, so that you can quickly replace components?

Jens Hurley: Definitely. Most of the stuff is done in two metric sizes, M6 and M5, so it’s pretty much built together with the same hardware.

Audrow Nash: Thank you.

Jens Hurley: No problem.

Audrow Nash: Hi, can you introduce yourself.

Michael Ferguson: Yes, I’m Michael Ferguson. I’m the chief technology officer at Fetch Robotics.

Audrow Nash: Can you tell me about what you have set up at your booth?

Michael Ferguson: This week we are showing our new robot platform, Fetch. We’re letting people get up close and see the robot in person. Most people here at this show are particularly interested in the research edition of Fetch.

Audrow Nash: Can you tell me a bit about the robot and describe it.

Michael Ferguson: Sure. So, Fetch is a mobile manipulation platform, it has a mobile base that can drive around with a scanning laser in it to give it a view of the world and keep it localized. It has a torso that can rise up and down and attached to that we have a seven degrees of freedom arm. The gripper is a modularity point. Currently, we have a simple two-finger pincher on here.

Audrow Nash: What do you mean by modularity point, can you swap it?

Michael Ferguson: You can swap it, yeah. We have a standard specification on this, an ISO standard.

Audrow Nash: International Standards Organization?

Michael Ferguson: Yes. There are a number of vendors here at the show with really cool grippers that have, you know, sensors and all sorts of other stuff in them. And many of those will fit on our robot.

And then, lastly, the robot has a pan and tilt head, so that you can look in different directions, that’s carrying an RGB-D sensor. Colour and depth data are in 3D so you can see the whole world around you.

Audrow Nash: It’s a Microsoft Kinect or very similar?

Michael Ferguson: It’s a prime sense sensor, yeah, similar.

Audrow Nash: The robot has a wider base and that’s how it moves around. Then it has the centre part, that has the robotic arm and then above that is the vision, the depth sensor. What’s at the very base?

Michael Ferguson: It’s a scanning laser range finder.

Audrow Nash: Laser range finder? Is it only on a plane?

Michael Ferguson: It is on a plane, yeah. That’s actually manufactured by SIC.

Audrow Nash: Why seven degrees of freedom?

Michael Ferguson: Seven degrees of freedom give you really good dextrous manipulation. If you’re just picking things up off a table top, you probably don’t need a really dextrous arm, but if you want to reach into things, if you’ve got a complex cluttered environment, you really need the extra degrees of freedom there.

Audrow Nash: I see, and so it only has one arm and the arm is at the centre of the robot. Why this configuration? Is this so you can reach things?

Michael Ferguson: Yeah, there are two aspects to it. One is that two arms would make this robot significantly larger, heavier and bulkier. You want to navigate around human environments. We have skinny doors and things like that. The other is that very few applications really require two arms and, in many cases, if you did need a second arm, you just have a second robot come up and help and they can coordinate with each other. As far as the centre, really that’s for stability. There’s a very nice little tuck configuration for the arm that makes the platform extremely stable as the robot’s driving around – so that you can move quickly in the environment and not have to worry about your robot falling over.

Audrow Nash: So will this, primarily, be used for warehouse solutions to pick up products and put them into boxes to be shipped off somewhere? Is that what this has been designed for?

Michael Ferguson: We’re really designing to go into the logistics space, warehouse automation, pick and pack for items. There are videos on our website where you can see the robots doing those sorts of tasks.

Audrow Nash: If I’m not mistaken, the whole robot can rise up. Its spine kind of grows longer and that allows it to reach higher?

Michael Ferguson: Yes.

Audrow Nash: Okay, and how tall can it be?

Michael Ferguson: It gets up to about 1.7 meters or so. The exact specs are on our website. When you extend the arm all the way up you can still have some dextrous manipulability at about six foot.

Audrow Nash: You also mentioned that this could be a research platform. Can you tell me a bit about that?

Michael Ferguson: Yeah, we need a stable robot platform built on a operating system or ROS. We’re now making available our research edition, which has the robot hardware plus the open source side of navigation and some manipulation software, so that university labs can get a real jump-start. They don’t have to build robots by bringing pieces together. They have something that’s designed for mobile manipulation from the get go.

Audrow Nash: Thank you.

Michael Ferguson: Well you’re welcome.

Audrow Nash: Hi, can you introduce yourself.

Simon DiMaio: Sure, my name is Simon DiMaio. I work in our R&D group at Intuitive Surgical.

Audrow Nash: Now, can you tell me a bit about what you have set up at this booth?

Simon DiMaio: Sure, so this is called the Da Vinci Surgical system. It’s a system for minimally invasive surgery, for assisting surgeons to perform in a minimally invasive way. It does that by allowing the surgeons to control instruments inside the patient’s body from what we call a surgeon console. They can get very dextrous manipulation of surgical instruments while minimizing the invasiveness of the procedure for the patients. They fit these instruments through very small incisions in the patient’s abdomen, but still allow the surgeon to have control and dexterity.

To see the field in 3D, we have a stereo endoscope that allows the surgeon to see the inside, with very high quality HD images to control surgical instruments, like graspers and forceps and cutting instruments and electric quarter instruments, in a very natural way, almost as if they were inside the patient.

Audrow Nash: Now can you describe the Da Vinci a little bit.

Simon DiMaio: Absolutely, so there are really two main components in the Da Vinci. There’s the part that’s at the patient and this is where there are robotic manipulators for instruments that are inserted inside the patient. And that is a cart. It’s a mobile cart that carries four robotic arms. Usually, one of those arms carries the camera and the surgeon can control the pointing direction and the view that the camera sees. Then there are three additional arms that are usually used to manipulate instruments. Those can be for cutting, grasping, suturing and things like that.

Then, the other main piece is where the surgeon sits. They sit in an ergonomic console. You can think of it as kind of like a surgeon’s cockpit where they see the images from inside the patient. They see information from the system and they can interact and control by moving some really fancy joysticks, to control the movement of the instruments. They can interact with tissue from the other side of the room. Usually, the console is in the same room, connected to the rest of the system through a cable, but the surgeon controls the system from it.

Audrow Nash: The patient and cart, it kind of looks like a large hang man.

Simon DiMaio: Yeah.

Audrow Nash: It’s high, high above with the arms coming down.

Simon DiMaio: Correct.

Audrow Nash: It has four arms and each has a large tube that is inserted into the patient and that’s where the manipulators are and the camera.

Simon DiMaio: That’s right and that tube is called a trocar or a cannula and that’s the point of entry into the patient’s body. In order to create space for the surgeon to work in, the abdomen, or whichever cavity within the anatomy the surgeon is working on, would be inflated using carbon dioxide gas to create kind of a bubble. Those tubes that you referred to, the trocars or cannulas, have a seal on them, so that the carbon dioxide is trapped inside and maintained and so the instruments pass in and out of the body through that cannula, or trocar, through a seal.

Audrow Nash: I see, and so if they want different attachments, do they remove the inserted part from the arm and then swap in a new one?

Simon DiMaio: That’s correct. Yeah. They’re surgical articulated instruments that have wrists on them to mimic what the surgeon would want to do inside the body. Then there are a variety of end effectors or jaws that the instruments have. Some of them are for grasping tissue, like bowel, liver or kidney. Some of them are for cutting. Some of them are for suturing, so for handling needles. Those instruments are interchangeable on the system so they can be removed and replaced with different instruments depending on the task.

So, if the beginning of the procedure is a dissection, where you’re trying to expose tissues of interest, then you’ve got to remove some tissue and suture things back together. In each of those stages you would potentially use a different combination of instruments. So the system allows you to swap instruments during the procedure very quickly.

Audrow Nash: Is that the purpose of the fourth arm?

Simon DiMaio: The fourth arm could be managing retraction, for example, so it could be holding some tissue out of the way. You might be working underneath a liver or you might be working in an area where there is bowel and you might need to hold the bowel out of the way. It could be what’s called a suction irrigation instrument, that flushes the field with saline and then vacuums it out to manage blood and to keep the field dry and visibility good.
There are a variety of different things that you could use the fourth arm for. It could be used to manipulate an ultrasound probe. If the surgeon is looking at ultrasound images of the tissue and trying to see beyond visible surfaces in the endoscopic image, they might insert an ultrasound probe and manipulate that with the fourth arm. So it’s kind of an auxiliary arm that’s used for many different things.

Audrow Nash: I see.

Simon DiMaio: It can also be controlled from a second console so you can connect two consoles to the robot and have two surgeons.

Audrow Nash: When is that useful? Why?

Simon DiMaio: Training in one instance, if you have a proctor that’s training a mentor, a mentor that’s training a mentee, the proctor might be on one console and the trainee might be on the other. They can virtually swap instruments back and forth. So, a mentor might demonstrate a technique and then give control of the instruments to the trainee who can replicate those motions.

Another instance is if you have two experienced surgeons and they’re collaborating on a complex case, so that you can have all of the arms operating at the same time.

Audrow Nash: I see. How do you manage collisions between the arms? Does that happen?

Simon DiMaio: It does happen, the arm is working in very close proximity. Part of the design was to make the arms as skinny as possible so that they can work closer together. They’re pretty robust, so they can collide and they shouldn’t break.

Audrow Nash: Do they collide during surgery?

Simon DiMaio: They can collide during surgery and usually that happens when the surgeon’s trying to get the instruments very close together through a very narrow entryway. Some of the forces that occur during collisions are fed back into the console, so surgeons can feel some of them on the outside of the robot.

Audrow Nash: So there is some haptics.

Simon DiMaio: So there is some haptics. They don’t feel the very delicate interactions with tissue, but they can feel external collisions between the arms or perhaps with some rigid object. They get some feedback and they stop and investigate. There’s usually an assistant or nursing staff at the patient’s side that can warn them. But it happens quite often, and instruments work very closely together, it’s almost like if we were to stand side by side and our elbows were to knock against each other. It’s not a big deal, you keep going and that’s the nature of working in a confined space.

Audrow Nash: Now, can you tell me about the demo that you have set up at the booth?

Simon DiMaio: So we have a full Da Vinci system here and people are able to test-drive it. We’ve got a mannequin that has a simple plastic environment people can interact with. We’ve got some elastic bands so that they can get a sense of the intuitiveness of the motion of the arms. We also have a virtual reality-based simulator that’s used for training, basically a skills simulator that surgeons can use to learn how to tie knots and suture. They learn how to control the camera and the various instruments, and so are able to repeat the same skills and drills over and over again to develop muscle memory and familiarity with the system in a way that doesn’t require them to operate on patients or animals.

It’s a convenient way for people to use virtual reality models to hone their skills or to warm up before a procedure. So there are some surgeons that use the simulator like an athlete would do before a game. They might warm up and they might do some practice.

Audrow Nash: Interesting. Is there a certification procedure for doctors that are using the Da Vinci system in operations?

Simon DiMaio: There is a training path. I’m less familiar with that side of things but there is some training. I don’t believe that Intuitive directly certifies surgeons for using the system at their hospitals. That’s the responsibility of the hospital. But we provide some training, but we don’t train them on how to perform surgery, that’s something that they learn in med school, but we teach them how to use the tool and then hospitals decide how to certify.

Audrow Nash: What kind of feedback have you received from surgeons on the Da Vinci?

Simon DiMaio: Well, a variety of feedback. Obviously, there’s lots of interest in what they can do with it and what they can offer their patients. It’s a way of allowing surgeons to offer their patients an option for minimally invasive procedure where sometimes that’s not available. We get feedback related to the kinds of procedures that they can perform and the value of those procedures.

We also get feedback on the ergonomics of the system. For them, it’s being able to sit at a console in a fairly comfortable posture versus sometimes working in fairly awkward poses and configurations at the operating room table.
Obviously, there are lots of features that surgeons would like and so we get feedback on things that they’d like to see. It’s a work in progress, so we’re always interested in feedback; things that work well, things that they would like improved.

Audrow Nash: Thank you.

Simon DiMaio: You’re welcome.

Audrow Nash: Hi can you introduce yourself.

François Boucher: François Boucher, VP Business Development for Kinova Robotics.

Audrow Nash: Can you tell me what we have at the booth here?

François Boucher: Lighter, smooth, compact, efficient sets of service robotic arms. We have two different models, JACO and MICO with different ranges and size and weight.

Audrow Nash: First, what do you mean by service robotic arm?

François Boucher: Basically, it’s the opposite of industrial robotics. We’re not extremely fast and extremely precise, but we’re good at working around humans, in collaboration with them or with a human in the loop.

Audrow Nash: That means that they’re compliant to being touched, so if they bash into a human when it’s moving, it won’t hurt the human?

François Boucher: Yeah, exactly, but the arms are on top of all the features that we added: software and sensors. The arm itself weighs only five kilograms, including gripper control, so it won’t hurt anyone at that speed and that weight.

Audrow Nash: Very light, very cool. So how do you make them compliant? Is there a spring in them, like with Baxter?

François Boucher: No, it’s sensors in every actuator.

Audrow Nash: It generates a small electrical signal when you have force on the arm and it’s in the opposite direction?

François Boucher: Exactly.

Audrow Nash: How come you have a different approach? What’s the advantage to doing that over having like a spring in series with the actuator or pillow action?

François Boucher: Well, the fork gives us a safety feature but it also gives us a whole new range of applications, since the arm can now be speech controlled and not only just position controlled. With this whole new field of service robotics, that’s the kind of control that will be needed.

Audrow Nash: You mentioned two robots. What are they? Can you introduce me?

François Boucher: Yeah, it’s JACO and MICO. We started a company for assistive robotics, to mount robots on a wheelchair so that people can eat, drink, open doors, all by themselves. JACO was our first product five years ago.

Audrow Nash: How did you get its name?

François Boucher: It’s the name of the uncle of the founder of the company. He, himself, had muscular dystrophy. So, basically, you’re in a power wheelchair and you have limited motion in your arms. He was an inventor and he developed and built some sort of mechanical arm, with a cable from bicycle brakes and a motor from a windshield washer. It was our inspiration to start the company. His nickname was Jaco, so that’s why the robot’s named after him.

Audrow Nash: I see, and how did MICO get his name?

François Boucher: That’s an easy one. It’s a contraction of Mini and JACO so it turns out to be MICO.

Audrow Nash: Very cool. Can you tell me a bit about the grip and endefactors on these robotic arms?

François Boucher: Yeah, so for the assistive robotics purpose, we had to figure out how we could grip hundreds of different objects of different size and weight. We had to find a really versatile solution. We went with an under actuated gripper.

Audrow Nash: Tell me a bit about how it’s under actuated.

François Boucher: Basically, there’s one actuator per finger and, if you imagine, two tendons links with a spring.

Audrow Nash: I see. Okay.

François Boucher: So the great advantage with this gripper is that they are mechanically intelligent. We don’t have to worry much about which force we’re going to apply, how we’re going to grasp the object, since the flexible gripper adapts to its shape. So, with the precise force needed, we can grab an egg like we can grab a bottle of water.

Audrow Nash: You can grab an egg?

François Boucher: Yeah, exactly.

Audrow Nash: Surprising, that’s great. So, to grab an egg, do you have to tell it that it’s a soft object?

François Boucher: No, it’s really just a great mix of mechanical design and force imitation based on current. Sometimes people try to overdo things. We keep it simple and it works pretty well.

Audrow Nash: That’s interesting. Can you talk a bit more about that or is that IP?

François Boucher: It’s more experience than IP, I would say.

Audrow Nash: That’s kind of mind blowing for me. All right. So some of these don’t have any hands, robotic arms or graspers. What are you demonstrating there?

François Boucher: A lot of our arms are used in maintenance logistics, inspections or replacing a tool, a sensor or a camera. So the arm’s not only used for manipulation, but it can also be combined with a mobile mount platform to perform inspection and probing.

Audrow Nash: I see. Thank you.

Audrow Nash: Hi, could you introduce yourself.

Andrew Lewis: Sure, my name is Andrew Lewis and I’m a roboticist with Applied Dexterity and I’m here representing the Raven II and the Surgical Cockpit.

Audrow Nash: Tell me about the Raven II.

Andrew Lewis: The Raven II is an open source surgical robotics research platform. It was developed here at the University of Washington and then spun out into our start-up company, Applied Dexterity. We’ve helped to ship it around the world. Now there are about 18 or 19 labs around the world using Raven, or something similar.

Audrow Nash: Now, what stands out to me is open source surgical. Can you explain that a bit?

Andrew Lewis: Sure. So all the software is open source. It’s up on GitHub if you want to fork it and play around with it; throw it in your own version of Raven, if you’re that brave. The tooler face is also open source. We have a team, one of the original Raven teams that developed a Da Vinci adapter, and now we can use clinical Da Vinci tools on our robot and have access to their 150 plus tools without having to make them all ourselves.

Audrow Nash: Is most of the value you get from open sourcing this technology in low level parts of the software?

Andrew Lewis: Right. We’ve done all the low level parts and we’ve opened it all up so that researchers can look at anything they want. If they want to implement their own impedance control, or some other control techniques, they’re welcome to dive deep into the PID loops and what have you. You can just use some automation software, with some python scripts in ROS and, since all the source code is available to you, if something’s not working quite right you can change it yourself, or you shoot me an email and I’ll tell you how to change it, or change it for you.

Then, with the common platform, all of the schools or labs that have this robot can really lean on each other for advice on how to approach a problem.
If they’re having technical difficulties, they can check the forums and see that “oh yeah Harvard dealt with it a few years ago and they just unplugged this other thing.”

Audrow Nash: Is it largely research communities that are benefitting from this being open source?
Andrew Lewis: Yeah, so far we’ve distributed just to research labs to help us get in touch with hospitals about using this for research or training. But that would probably be a tighter, ready to roll, don’t-need-to-play-with-the-code version.

Audrow Nash: When we spoke about this a little bit ago, you mentioned that it’s not FDA approved.

Andrew Lewis: Right.

Audrow Nash: Can you talk a bit about what that means in term of its application for surgical, is it just a robot that is used in surgery first of all?

Andrew Lewis: The original version, the Raven I was used for Cholecystectomy, about a decade ago I guess.

Audrow Nash: I don’t know what that is.

Andrew Lewis: It’s gall bladder removal in a pig, and it was successful and I think that’s been the only real life surgery that’s been done with the robot. Harvard has started doing some work with it on time delay and tele-operation.
Since it’s not FDA approved, it means that, when a lab is looking to do some research with the platform that’s ready to go, they’re not paying for that FDA approval. The Da Vinci or another clinical systems is going run into millions of dollars because they’ve had to put time and a lot of effort into getting it approved.

Audrow Nash: That’s the advantage of it, the cost is kept down because it’s not FDA approved.

Andrew Lewis: Right.

Audrow Nash: I see. What’s kind of the goal do you have, where is the company heading?

Andrew Lewis: Our slogan is: “pushing the boundaries of surgical robotics.” The idea is to be able to make this platform available to research labs that are really pushing the boundaries and to support them wherever we can. Then we can make a lot of great connections in all these great labs.

And maybe try to license back some of the technologies that are coming out of the research community. And certainly learn a lot by just being around everyone, so we keep developing either surgical or tele-operated or autonomous systems, and gain huge advantages by being familiar with all these research topics.

Audrow Nash: Thank you.

Andrew Lewis: Sure.

Amazon announced that at the end of September they had 30,000 Kiva robots at work in 13 fulfillment centers, effectively doubling the number of Kiva bots that it had in 2014. “Our intent is to use [Kiva robots and the Amazon Robotics subsidiary] more widely, so stay tuned.”

UPDATE 11-1-2015: Added Grenzebach’s G-Com storage and picking system to the list of vendors offering competing systems to Kiva’s.

Phil Hardin, Amazon’s director of investor relations also said:

The wider use of robots at fulfillment centers has had a direct impact on productivity. It’s a bit of an investment that has implications for a lot of elements of our cost structure, but we’re happy with Kiva. It has been a great innovation for us, and we think it makes the warehouse jobs better, and we think it makes our warehouses more productive.

Amazon acquired Kiva Systems for $775 million in 2012 and recently renamed it Amazon Robotics. It is now a wholly owned subsidiary of Amazon. This left a big gap in the marketplace for robotic innovation in distribution and fulfillment centers because Kiva’s goods-to-man technology was taken in house by Amazon.

Distribution centers (also referred to as warehouses, DCs, fulfillment centers and package handling centers) are principal parts of the order processing and fulfillment process. As the e-commerce boom continues to grow, warehouse managers need to find new suppliers to meet their automation needs for their DCs. Amazon is just one of many online e-commerce providers in need of robots and automation; Alibaba and Jingdong in China and Flipkart and Snapdeal in India are also big e-commerce companies adding robots to their warehouses.

eCommerce is the fastest growing segment of retail. In 2013 it represented less than 10% of overall retail, but has been growing at nearly 15% annually and is expected to continue at that rate for the remainder of the decade. Online retailers are having to change their operations to meet this growing demand.

A few startups are beginning to enter the market to fill the gap left by Kiva:

Fetch Robotics, a Silicon Valley startup, recently received $23 million in equity funding from SoftBank to scale up operations. The Fetch robot is a mobile manipulator that can be autonomously deployed to pick orders and place items into a pod on the companion Freight robot, which then autonomously navigates to a packing station. Multiple Freight robots can support Fetch robots and all are coordinated and monitored by a real-time web-based communications and information system that allows warehouse managers to know where robots are, how orders are being fulfilled and inventory status.

GreyOrange, a Singapore-based Indian startup, is beginning to distribute their Butler robots — square versions of Kiva’s round robots — along with their goods-to-man warehousing system, which is also very similar to Kiva’s system of moving portable shelving to a picker/packer station for picking and then returning them to a dynamic warehouse location. GreyOrange offers exactly the same process and sells the mobile robots, mobile storage units, pick/put stations, charging devices and warehouse control system. GreyOrange received a $30 million equity investment from a group of VCs to enable international expansion to Japan and China, and hire 25 additional people in Hong Kong and Singapore.

Harvest Automation, a startup initially focused on moving potted plants from one location to another — a very labor-intensive task in the nursery segment of the agriculture industry. Their HV-100 mobile robots reduce labor headcount and costs and provide needed pot manipulation at very low cost. Now that Harvest Automation has mastered the task and successfully brought their product to market, they are expanding into the warehouse with their new TM-100 mobile material handling platform. The rugged TM-100 enables workers to pick and place goods in a cart, which then autonomously whisks off and navigates to a selected destination for further processing.

In addition to the startups mentioned above, old-line companies are also entering the market. The 55-year old German-headquartered Grenzebach Group is a good example. Grenzebach has held interests in KUKA, Inos Automation, SnoxStellar AGVs, and Swisslog – all involved in robotics, vision systems and materials handling as well as their new G-Com mobile storage and picking system. Grenzebach is still the controlling partner in all but KUKA and Swisslog. [KUKA bought back it’s shares from Grenzebach and acquired Swisslog for $357m in September, 2014.] BLG Logistics was awarded the German Logistics Award 2015 for an installation in the warehouse of a trading company of 74 Grenzebach G-Com robots, delivering goods from 800 shelves to five picking stations.

Bottom line

Amazon isn’t the only e-commerce warehouse operator adding robots to their automation arsenal.  Think Coca Cola, Walmart, Nike, Ikea, FedEx, and UPS. And there are hundreds of name-brand suppliers of warehousing automation technology such as Westfalia, Egemin, Schaefer, Swisslog, Vanderlande, Daifuku, Intelligrated, JBT, Knapp and Thiele to name just a few. There is a transition in technology to reduce fixed racking and conveyor costs in favor of dynamic storage and mobility methods. Thus far only Kiva and these new startups are offering that level of disruptive system, but the big companies are sure to enter the fray soon with competing products.

If you liked this article, you may also be interested in:

• SoftBank invests in Fetch Robotics
• Competing robotic warehouse systems
• GreyOrange gets $30m for mobile robots
• Team RBO from Berlin wins Amazon Picking Challenge convincingly
• Robots in Depth: Melonee Wise on building robots, and robot companies

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Melonee shares how she first got into building things at a young age and how that led to studying mechanical engineering and leaving her PhD project behind to become the second employee of Willow Garage. She shares some personal anecdotes from the first few years at Willow Garage, including both successes like the PR2 and some less successful moments.

Melonee also gives her perspective on the development phase robotics is in now and what the remaining challenges are. Related to that, she discusses what is feasible to deliver in the next five years vs. what her dream robot would be.

You can support Robots in Depth on Patreon. Check out all the Robots in Depth videos here.

The MIT Technology Review’s annual list of 35 Innovators Under 35 aims to illustrate the most important emerging technologies of the moment. Released this August, the 2015 list features a number of robotics and AI visionaries. Check them out.

Lars Blackmore, Visionary, 34: SpaceX engineer, working on reusable rockets that could be flown back to Earth in reverse.

Yunji Chen, Inventor, 32: designing dedicated deep-learning processors, optimized “to compute the basic blocks of machine learning.”

Adam Coates, Visionary, 34: designing artificial intelligence for Chinese search company Baidu, to make the Internet more useful to millions.

Canan Dagdeviren, Inventor, 30: working on flexible sensors and batteries for a new class of medical device.

Travis Deyle, Inventor, 32: builder of robots that can be powered wirelessly. Now at Google X working on tech to improve health care.

You may remember our Robots Podcast interview with Travis about his research on Finding objects with RFID.

Rikky Muller, Entrepreneur, 34: designing the implantable hardware intended to interact directly with the brain and help treat mental disorders.

Jaime Shotton, Inventor, 34: computer visionary at Microsoft Research. Giving computers a new way to see the world.

Ilya Sutskever, Visionary, 29: pushing the frontiers of deep-learning for the Google Brain.

You may remember the Talking Machines Podcast interview with Ilya about Machine Learning and magical thinking.

Benjamin Tee, Inventor, 33: developing medical diagnostic and treatment technologies using a synthetic sense of touch.

Conor Walsh, Inventor, 33: working on robots that are soft, lightweight, and flexible so people can wear them to enhance their abilities.

You may remember our Robots Podcast interview with Conor’s colleague Donal Holland about their Soft Robotics Toolkit.

Melonee Wise, Entrepreneur, 33: Fetch Robotics CEO wants affordable robots for the warehouse and beyond.

Check out our coverage of Melonee and Fetch Robotics on Robohub.

Credit: Illustration by Magoz; Color portraits by Matthew Hollister; black-and-white portraits by Ping Zhu. Source: MIT Technology Review.

Fetch Robotics, which just demo’d their Fetch and Freight robot system at ICRA in May, announced receiving $20 million from SoftBank, O’Reilly AlphaTech and Shasta Ventures.

The Series A round of funding brings their total funding to date to $23 million. SoftBank was the lead investor in the round. SoftBank, a Tokyo-based phone and Internet service provider, recently invested $100M in Aldebaran, the developer of the popular Nao robot. As part of their deal, SoftBank commissioned the new Pepper robot, which is now being rolled out in Japan. Aldebaran and Pepper are SoftBank’s first and second foray into robotics; Fetch Robotics is their third.

“As businesses look for solutions to streamline operations and meet the needs of an on-demand economy, we see a tremendous opportunity for robotics to solve that problem,” says Kabir Misra, Managing Director at SoftBank Group US, Inc. “The team, the robots, and the timing all lead us to Fetch Robotics and we are happy to join them in bringing Fetch and Freight to market.”

Fetch Robotics, a San Jose startup, unveiled their mobile manipulator and mobile platform (Fetch and Freight respectively) to wide acclaim at ICRA in Seattle last month. The Fetch team has a colorful history first as a spinoff from Willow Garage called Unbounded Robotics charged with making a low-cost version of Willow Garage’s PR2 robot. They succeeded with an orange, white and grey one-armed mobile robot that they could build and sell for $35,000 compared to the PR2’s $250,000+ cost. That operation failed to get agreeable funding terms and folded but the team went on to work with FYS Systems (fetch your stuff), which was recently renamed to Fetch Robotics.

The Fetch Robotics system is composed of a mobile base (called Freight) and an advanced mobile manipulator (called Fetch).  Fetch and Freight can autonomously find and use a charging dock for continuous operations. In addition, the system includes accompanying software to support the robots and integrate with the warehouse environment. Both robots are built upon the open source robot operating system, ROS.

The robots are designed to work autonomously alongside workers, performing repetitive tasks such as warehouse delivery, pick and pack, and more.  Fetch and Freight used in tandem are capable of handling a vast range of items in a typical warehouse.

“I’m delighted to have SoftBank join the team and help us change the world of logistics and material handling,” said Melonee Wise, CEO of Fetch Robotics. “SoftBank’s expertise and worldwide resources with respect to technology, production, distribution and more will be a big help to our growing organization.”

• Fetch and SRI show that useful humanoids are on the horizon
• What’s hot in robotics? Four trends to watch
• Competing robotic warehouse systems
• Team RBO from Berlin wins Amazon Picking Challenge convincingly

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Ex-Willow Garage and Unbounded Robotics veterans get $3 million to bring mobile manipulator and mobile platform to market.

Funding came from O’Reilly AlphaTech Ventures and Shasta Ventures. Melonee Wise was named CEO. Fetch Robotics is the new name for FYS Systems. FYS stood for Fetch Your Stuff. The new name, Fetch Robotics, seems more appropriate for a materials handling logistics market line of products.

Things are moving fast at Fetch. They’ve announced that the first robots will be introduced in the second quarter of this year and are searching to hire up to 10 positions in addition to their present 10-member team. They will launch a research edition of the robot first and probably show it at ICRA in May in Seattle.

From comments made by CEO Wise to IEEE/Spectrum, it appears that they are making more than one product.

“One is a mobile manipulator and one is a mobile base, and they’ll work together. We’re targeting the logistics market, looking to do order fulfillment with robots.”

No pictures are available of the platform or the robot; the software however, will use and be built upon ROS.

Evan Ackerman said it best when he wrote:

There was a lot of interest in Unbounded and the UBR-1, and I’m sure there’s going to be a lot of interest in what Fetch is working on. It’s not often that companies get second chances like this, and I’m certain that the Unbounded core team (and the rest of the roboticists at Fetch) are looking forward to proving that they can deliver the next generation of mobile manipulators.

As advances in actuators, processors and machine vision enable robots to be mobile and do more, and do it safely alongside humans, many companies are working on and beginning to offer mobile manipulator robots. We wish Fetch well in their new venture.

Fetch Robotics has just announced $3 million funding from Shasta Ventures and O’Reilly AlphaTech Ventures , which will be used to bring their new mobile manipulator robots to market in Q2 2015.Fetch Robotics is looking at the light industrial and logistics sectors, although researchers will also find the robots affordable. This comes hot on the heels of PAL’s February 4th launch of Tiago, also a mobile manipulator. And it seems very possible that Google will announce something similar very soon. After all, they have now had plenty of time (and experts) to finish what Redwood Robotics had started.

The innovator’s dilemma is at work here. The companies producing new mobile manipulators are not the big names in robotics. Companies like ABB, Kuka, Fanuc, etc. cannot really afford to produce a new style of robot that might damage their brand if it doesn’t perform to the same high standards. For example, ABB has been working on a collaborative manipulator for at least four years but in spite of unveiling the concept robot in 2011, ABB is only officially releasing YuMi for market on April 13 2015. This creates opportunity for faster-moving small companies. It looks like the technological conditions are ripe for the development of affordable mobile manipulators. Now, we’ll all get to see if the market is also ready.

Who’s who in the new light industrial manipulator sector?

These new robots are like the affordable compliant offspring of a PR2 and Motoman’s SDA series robots. Only these robots are safe for working together with humans and won’t break the bank. We will also be seeing combinations of robots where a base from one company, ie. Fetch or Adept is used with an arm from another, ie. Kuka or Universal.

Firstly, Fetch Robotics is not Unbounded Robotics, and the robots will be different (although Melonee Wise is now the CEO of Fetch Robotics and the core robot building team is the same). Unbounded Robotics ceased operation in 2014 just prior to marketing the UBR-1 (pictured). Where the UBR-1 was like a smaller version of Willow Garage’s PR2, the new robot(s) from Fetch Robotics will have separable base and arm(s). More information will be available in Q2 2015.

PAL’s Tiago is also modular, available as a base, as a base-and-arm package, and in other configurations. Tiago has a 7DOF arm capable of lifting a 3 Kg payload and a five-finger underactuated hand. Similar to the UBR-1, Tiago has a rising torso, USB and ethernet ports, and is also ROS compliant. The various Tiago combinations will cost between 30,000 and 60,000 euros.

Baxter from Rethink Robotics is not autonomously mobile but you can add a wheeled base to your Baxter and push it around. And Baxter is definitely affordable at $25,000 USD, although the various gripper options are extra.

YuMi from ABB is another stationery dual armed collaborative robot, developed for small parts assembly, especially in the consumer electronics sector. YuMi is part of a total small parts assembly solution that includes adaptable hands, flexible parts feeders, force control sensing and vision guidance.

Nextage from Kawada has similar specifications to Baxter and YuMi (although only 6DOF in the arms), but with an additional 1DOF in the waist. Nextage has been designed to be wheeled from place to place in an assembly line and has a compact configuration.

There’s also the Care-O-bot 3 from Fraunhoffer Institute, which is using industrial robotics components to create a service robot. Care-O-bot 3 has an attractive form factor for human interaction. Quite a difference from earlier versions, which were arms on mobile bases, suitable for tasks like welding assistance, not emptying garbage bins or serving coffee. (added 2/12/15 hat tip to Hizook)

Who knows for sure what Google is about to announce, although they have the IP and the experts to complete Redwood Robotics and Meka’s vision of an affordable lightweight compliant robot arm. They’re also well positioned to develop an autonomous mobile base.

And who knows how many other startups still in stealth will announce their light industrial/logistics manipulators this year? I’m betting that there’s at least one more.

As well as some robot arm/base combos being mix-n-match, the end effectors are also a whole other story. Perhaps 2015 will really be the year of the end effector. 

If you liked this article, you may also be interested in:

• Robot Startup Series #7: Interview with Soft Robotics’ Carl Vause

• Just launched: Fellow Robots’s OSHbot
• Amazon Picking Challenge
• Grasping objects in a way that is suitable for manipulation

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.