Shadow Robot’s Dexterous Hand can do more than cook up a storm

Rich Walker, Managing Director, The Shadow Robot Company

The Shadow Robot Company has been perfecting its robot hands for almost two decades, but it took just a short demo by the world’s first robot chef last month to propel this small, London-based startup into the global limelight. Despite the chef’s obvious attractions, Robohub’s Adriana Hamacher wanted to delve deeper: to discover what future robot hands have in the manipulator market and why they could soon turn up in a strawberry patch.

With his waist-length dreadlocks and goatee, Rich Walker certainly stands out at the funding and robo-business events, where I’ve often bumped into him. His workshop is just down the road from my house, so when the robot chef was unveiled at Hannover Messe last month, cooking up a storm for the world’s media, it seemed like a good time to pay Rich a visit.

Despite its prowess at making crab bisque, it’s fair to say that the robot chef is defined by its highly sophisticated manipulators – the “Dexterous Hands” in which Shadow specialises.

But the project is actually a collaboration between Shadow and Moley Robotics, the company funding the venture. Last year, in order to fulfil his vision of creating a chef replicator, Moley’s Russian-born founder, scientist and engineer Mark Oleynik, went on the hunt for a company to build the hands for his robot kitchen. He found there was a fairly short list, and it really only came down to one company: Shadow.

“What we built was almost a classic piece of industrial automation,” says Rich, taking up the story. “It’s a couple of robot arms with tooling on the end to be able to interact with a set of well-defined, fixed objects and perform a complex procedure with them. It just happens they’re in a kitchen, the tooling is a pair of fully human-like robot hands and the procedure they’re performing is cooking a crab bisque – it takes 26 minutes. The things you start with are not nuts and bolts and metal flanges, they’re chopped garlic and chicken stock, and the end result is a bowl of soup, plated up exactly as a chef would do it.

“And it’s worth noting that every single person in this company now knows how to cook a crab bisque, because we’ve seen it done seven million times!”

Masterchef winner Tim Anderson, preparing crab bisque. Source: Moley Robotics

The now infamous crab bisque recipe was prepared copying the exact movements of 2011 BBC MasterChef winner Tim Anderson. The robot switches on a hob, scrapes butter into the pan and adds prepared and measured ingredients – from a specific place on the worktop. But, it doesn’t do the chopping… That, says Rich, is an AI problem.

For Rich, corner cutting or finding ways around the problem is second nature, as becomes increasingly obvious in our conversation. Not only is he able to solve problems, he’s also able to turn them to his advantage. Rich believes that supermarkets will want to supply ingredients for the robot kitchen – at a cost, of course – and it’s already becoming common to see chopped salad, vegetables and meat on a shop’s shelves.

Even without the chopping there’s still a lot to do, to get the chef ready by 2017 and by far the biggest challenge is that something like this has never been done before.

“Nobody has built a mass manufacturable humanoid hand,” says Rich. “We have to go from making a small number a year, for state-of-the-art research to something we could have a factory producing in large numbers. So that’s the longest arch of the project, the critical path.”

The robot chef in action. Source: Moley Robotics

The team is also looking at equipment options and trying to keep costs down. But there’s no denying that at around £10,000 ($14,000) – the robot chef will have a hefty price tag, although it’s effectively a complete kitchen, comprising an integrated oven, hob, dishwasher and sink. The market for consumer and office robots is predicted to  grow at a CAGR of 17% between 2014 and 2019, seven times faster than the market for manufacturing robots, so does Shadow’s boss think it’s an area ripe for disruption?

“It’s interesting that domestic robotics has been the thing that inspires people,” says Rich. “But there’s never been anything above the level of the vacuum cleaner. Part of the reason for this is that, when you talk about unstructured environments, homes are about as unstructured as you can imagine. They’re full of random things, things are never where anybody left them, there are small animals running around, there are children, there’s jam – all kinds of things that are really difficult for a robot to see, interact with and understand. And it’s very hard to paint a white line around it and say ‘nobody cross that white line while the robot’s running.’ A factory’s actually a really nice clean, safe, structured environment, when compared with a domestic one.

“Frankly, we’re still a long way from solving the really big problems there. The kitchen is interesting because, there, we’ve constrained it down and said: ‘we’re going to put this robot in the kitchen, above the kitchen, as part of the kitchen; it’s going to be part of the equipment, not wandering around the house. It doesn’t have to find things, it doesn’t have to avoid people, it’s just either the robot is running, in which case the kitchen is closed, or the kitchen is open and the robot is tucked away. So that’s a fantastic simplification, it kind of eliminates all the real challenges of robots in the home.”

“Nobody has built a mass manufacturable humanoid hand. We have to go from making a small number a year, for state-of-the-art research, to something we could have a factory producing in large numbers”

Rich Walker started off as a computer scientist and joined Shadow in 1989 to write software for robots, but he discovered the robots needed building first. He now leads the company, having taken over from founder and non-executive director Richard Greenhill. In the early days, in the mid-90’s, Shadow were primarily working on biped robots and soft pneumatic muscles, but it became apparent that things weren’t working out.

“Bipeds are, I hesitate to say, useless,” says Rich cautiously. “It’s very hard to see an application for a biped that’s commercially viable at any point in the next 10 years. Twenty years ago even more so, because you just need so much infrastructure and ecosystem and capability to make it worth having a hugely expensive biped robot that is risky and might fall over.”

So, they put their heads together and the concept of the human hand came up. That was nearly twenty years ago. So why has it taken them so long?

The Dexterous Hand. Source: The Shadow Robot Company

“We’ve been looking around at ways to get additional resources,” explains Rich. Shadow has become adroit at not only slotting its technology into the supply chain, but also fitting it into funding schemes in both the UK and the EU.

Some of the first applications for their hand included assistive technology. But, fifteen years ago, most of the other elements necessary – for the hand to be able to see, to navigate, to interact with objects and to communicate – were simply not there, which was a big problem.

Recently, Shadow has revisited this space; it’s involved in a European project, RAMCIP, building an assistive robot for people with mild cognitive impairment. With advanced infrastructure, it’s now possible to enable a mobile robot, with a hand, to navigate via 3D mapping, to sense people’s emotions and handle objects. No one calls them crazy anymore.

Applications in manufacturing are looking equally attractive, with the possibility of integrating the Dexterous Hand in a new generation of lightweight, relatively mobile, flexible and safe factory robots. The price point works too, as it’s comparable to the two-year fully loaded labour cost of an employee on a factory line.

“There’s a lot of people with automation type problems who are solving them by having humans do the work,” says Rich, “because you can’t get a traditional robot to do them. But this next generation, of flexible, dexterous, manipulation capable robots, should be able to make an impact on that.”

With both the domestic and industrial markets to consider, you’d think Shadow would have its hands full (pardon the pun), but there’s another sector beckoning too.

“If you can pick the fruit then all the other things can be improved and adapted, but today, there are no machines that you can use in the UK to pick fruit.”

Shadow’s business model is opportunistic in that the company looks for strong partners who have problems and European or UK funding calls to provide the finance to solve those issues. The big challenges are addressed, largely, with EU funding. For shorter term, more commercially focussed or more manageable problem areas, they target government agencies such as Innovate UK, assuming the partner organisation is UK-based.

A recent example is a joint Innovate UK and Biotechnology and Biological Sciences Research Council-funded agri-tech project the company is involved in, AUTOPIC. Led by the UK’s Harper Adams University, the project consortium includes BerryWorld, which  is the leading supplier of berries to UK supermarkets.

Modern strawberry trestles. Source: AUTOPIC Consortium

As might be expected, BerryWorld suffers from seasonal labour issues. The company is also very open to new technology. It’s already using innovative techniques that involve raising the growing area so that the strawberries are now waist height and rest on substrates instead of soil. Rich believes that this is a game changer; it’s opened the door to automatic harvesting.

“We actually focused on the commercial step changes,”he says. “If you can pick the fruit then all the other things can be improved and adapted, but today, there are no machines that you can use in the UK to pick fruit.”

He believes that previous solutions have over complicated the process. “They’ve been very interested in problems like: ‘how do you see the berry in a complex space?’ The whole research challenge.”

But modern day farming means that strawberry trestles are now managed in such a way that it should be – not easy – but more possible for a robot hand to access the fruit:

“The leaves are raised up so they’re out of the way and the berries hang down so they’re very, very visible. The challenges are getting to the less accessible fruit at the back, not damaging the plant and not colliding with things, while doing it all at speed – picking a strawberry every four or five seconds – to have a chance of being commercially viable.”

The system must also only pick grade one fruit, because, unlike apples, for instance, strawberries won’t ripen after they’re picked.

There’s an impressive amount of technology involved in Shadow’s hand: 20 motors, 26 micro-controllers, 24 movements and 129 sensors. So, just how close is it to a real human one?

“We’ve managed to capture almost all of the interesting capabilities,” says Rich.  “Human joints have a property of dislocating and relocating and we don’t do that. The human hand, because of the way the bones run through the palm, can do this kind of wrapping and curling activity. What we do on our robot hand is just a finger bend that comes up to oppose the thumb, rather than a natural wrapping and curling. But, other than that, we think we’ve pretty much captured all of the detail of the way the human hand works. We haven’t yet got anything like the sensitivity of human skin and finger tip sensing. I think it’s going to be another decade before you have all-over high resolution sensing. But, in terms of the mechanical movement and operations, we seem to have about the same sensitivity, the same precision and better repeatability, because it’s a robot. We haven’t found any manipulation task that, if we could program it, we couldn’t use it so for.”

What stands out from talking to Rich is just how multidisciplinary the work Shadow does is; it’s not just about the hands, the software, the hardware, 3D navigation and AI, it’s about supply chains, dealing with chefs, users and funding bodies. How would Rich advise a young start-up to approach all the work that needs to be done to get something like this off the ground?

The Dexterous Hand. Source: The Shadow Robot Company

“We’ve already done robot kitchens so find a different problem,” he quips. “There is a lovely concept that you get taught in business school: it’s the cycle of resources and the cycle of constraints. You say: ‘I want to do something, what have I got to make available to me? How well do I understand the problem I’m trying to solve?’ If you don’t have enough resources, there are two things you can do. You can either go out and raise money, bringing partners in, developing technologies… or you can iterate on a loop of understanding the constraints and refining them around the problem. That means saying things like: ‘suppose the ingredients are supplied in package form, so that we don’t have to chop them. Is that feasible? Yeah, great. So, we’ll do that.’

“So you simultaneously increase the amount of capability that you have and decrease the problem, until you hit the sweet spot, where you say: ‘I have enough resources and the problem is well enough constrained.’ It’s iterating on those two loops that gets you from: ‘I have an idea and I’m going to need £100 million to do it and I don’t know who the customer is,’ to: ‘I can see all the customers, I’ve actually met most of them, the problem is well constrained, I only need two external partners, I know who they are and I’ve got some funding to execute the project.’ It’s all achievable and everyone we need is in the room.”

He makes it sound so easy. But anyone who’s attempted to build a robot of any complexity will know that it’s not. However, it is possible, and the robot chef, with his hands that were nearly twenty years in the making, is testament to that.

If you liked this article, you may also be interested in:

• 2015 sees mobile manipulators coming to market
• Governmental funding of strategic robotic projects
• OpenBionics prosthetic hands: Open source, affordable, lightweight, anthropomorphic
• DARPA’s HAPTIX project hopes to provide prosthetic hands with sense of touch
• Grasping objects in a way that is suitable for manipulation

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Adriana Hamacher Associate Editor at Robohub and the UK's Knowledge Transfer Network and a contributor to Economist Insights

Adriana Hamacher Associate Editor at Robohub and the UK's Knowledge Transfer Network and a contributor to Economist Insights