BERT2, a humanoid robot assistant. Credit: University of Bristol

Making an assistive robot partner expressive and communicative is likely to make it more satisfying to work with and lead to users trusting it more, even if it makes mistakes, a new study suggests.

But the research also shows that giving robots human-like traits could have a flip side – users may even lie to the robot in order to avoid hurting its feelings.

These were the main findings of the study I undertook as part of my MSc in Human Computer Interaction at University College London (UCL), with the objective of designing robotic assistants that people can trust. I’m presenting the research at the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) later this month.

With the help of my supervisors, Professors Nadia Berthouze at UCL and Kerstin Eder at the University of Bristol, I constructed an experiment with a humanoid assistive robot helping users to make an omelette. The robot was tasked with passing the eggs, salt and oil but dropped one of the polystyrene eggs in two of the conditions and then attempted to make amends.

The aim was to investigate how a robot may recover a users’ trust when it makes a mistake and how it can communicate its erroneous behaviour to somebody who is working with it, either at home or at work.

BERT2, a humanoid robot assistant. Credit: University of Bristol

The somewhat surprising result suggests that a communicative, expressive robot is preferable for the majority of users to a more efficient, less error prone one, despite it taking 50 per cent longer to complete the task.

Users reacted well to an apology from the robot that was able to communicate, and were particularly receptive to its sad facial expression which is likely to have reassured them that it ‘knew’ it had made a mistake.

At the end of the interaction, the communicative robot was programmed to ask participants whether they would give it the job of kitchen assistant, but they could only answer yes or no and were unable to qualify their answers.

Some were reluctant to answer and most appeared very uncomfortable. One person was under the impression that the robot looked sad when he said ‘no’, when it had not been programmed to appear so. Another complained of emotional blackmail and a third went as far as to lie to the robot.

Their reactions would suggest that, having seen the robot display human-like emotion when the egg dropped, many participants were now pre-conditioned to expect a similar reaction and therefore hesitated to say no; they were mindful of the possibility of a display of further human-like distress.

The research underlines that human-like attributes, such as regret, can be powerful tools in negating dissatisfaction but we must identify with care which specific traits we want to focus on and replicate. If there are no ground rules then we may end up with robots with different personalities, just like the people designing them.

BERT2, a humanoid robot assistant. Credit: University of Bristol

“Trust in our counterparts is fundamental for successful interaction, says Kerstin Eder, who leads the Verification and Validation for Safety in Robots research theme at the Bristol Robotics Laboratory. “This study gives key insights into how communication and emotional expressions from robots can mitigate the impact of unexpected behaviour in collaborative robotics. Complementing thorough verification and validation with sound understanding of these human factors will help engineers design robotic assistants that people can trust.”

The study was aligned with the EPSRC funded project Trustworthy Robotic Assistants, where new verification and validation techniques are being developed to ensure safety and trustworthiness of the machines that will enhance our quality of life in the future.

The IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), takes place from 26 to 31 August in New York City, and the study – Believing in BERT: Using expressive communication to enhance trust and counteract operational error in physical Human-Robot Interaction’ by Adriana Hamacher, Nadia Bianchi-Berthouze, Anthony G. Pipe and Kerstin Eder – will be published by the IEEE as part of the conference proceedings, available via the IEEE Xplore Digital Library.

A pre-publication copy of the research paper is available at: https://arxiv.org/pdf/1605.08817.pdf

David Mindell and the cover of “Our Robots, Ourselves” (Viking/Penguin). Photo (Mindell): Len Rubenstein

by Peter Dizikes | MIT News Office

If you follow technology news — or even if you don’t — you have probably heard that numerous companies have been trying to develop driverless cars for a decade or more. These fully automated vehicles could potentially be safer than regular cars, and might add various efficiencies to our roads, like smoother-flowing traffic.

Or so it is often claimed. But the promise of artificial intelligence, advanced sensors, and self-driving cars could be achieved without full autonomy, argue scholars with deep expertise in automation and technology — including David Mindell, an MIT professor and author of a new book on the subject.

If robotics in extreme environments are any guide, Mindell says, self-driving cars should not be fully self-driving. That idea, he notes, is belied by decades of examples involving spacecraft, underwater exploration, air travel, and more. In each of those spheres, fully automated vehicles have frequently been promised, yet the most state-of-the-art products still have a driver or pilot somewhere in the network. This is one reason Mindell thinks cars are not on the road to complete automation.

“That’s just proven to be a loser of an approach in a lot of other domains,” Mindell says. “I’m not arguing this from first principles. There are 40 years’ worth of examples.”

Now Mindell, the Frances and David Dibner Professor of the History of Engineering and Manufacturing in MIT’s Program in Science, Technology, and Society, and also a professor in MIT’s Department of Aeronautics and Astronautics, has detailed the history in his new book, “Our Robots, Ourselves,” released on Oct. 13 by Viking Books.

To be clear, Mindell thinks that “it’s reasonable to hope” that technology will help cars “reduce the workload” of drivers in incremental ways in the future. But total automation, he thinks, is not the logical endpoint of vehicle development.

“The book is about a different idea of progress,” Mindell says. “There’s an idea that progress in robotics leads to full autonomy. That may be a valuable idea to guide research … but when automated and autonomous systems get into the real world, that’s not the direction they head. We need to rethink the notion of progress, not as progress toward full autonomy, but as progress toward trusted, transparent, reliable, safe autonomy that is fully interactive: The car does what I want it to do, and only when I want it to do it.”

Shooting for the “perfect 5”

To see why Mindell thinks history shows us that automation is not the endpoint of vehicular development, consider the case of undersea exploration. For decades, engineers and scientists thought that fully automated submersibles would be a step forward from the seemingly risky work of deep-sea journeys.

Instead, something unexpected happened with submersibles: Technological progress, including improved communications technologies, made it less useful to have fully automated vehicles sweeping across the sea floor. Instead, Mindell notes, submersibles “are more effective when they have even a little communication” with people monitoring and controlling them.

Or consider the Apollo program, which put U.S. astronauts on the moon six different times. Originally, Mindell notes, the expectation was that moon missions would be fully automated, with astronauts nothing more than passengers. But in the end — and partly due to the feedback of the astronauts themselves — astronauts handled many critical functions, including the moon landings.

“The sophistication of the computer and the software was used not to push people out, but to give them true control over the landing,” Mindell says.

And then there are airplanes. Commercial airliners do have many automated systems, such as cruise control-type features and even systems that can automate landings in certain circumstances. But it still takes highly trained pilots to manage those systems, make critical decisions in the cockpit — and, yes, frequently to steer the planes.

“Commercial aviation is incredibly safe,” says Mindell, himself a qualified civil aviation pilot with more than 1,000 hours of flying time to his credit. “Part of the reason is there are a lot of highly technical systems, but those systems are all imperfect, and the people are the glue that hold the system together. Airline pilots are constantly making small corrections, picking up mistakes, correcting the air traffic controllers.”

Drawing on a concept developed by MIT professor of mechanical engineering Tom Sheridan, Mindell notes that the level of automation in a project can be judged on a scale from 1 to 10 — and aiming for 10, he contends, does not necessarily lead to more success in any given endeavor, compared to a happy medium of technology and person. In the space program, Mindell reflects, “The digital computer in Apollo allowed them to make a less automated spacecraft that was closer to the perfect 5.”

Full automation a “20th-century narrative”

So why, in the case of cars, are we back to a point where many people are envisioning a driverless future? In a way, Mindell says, this vision of the future belongs squarely to the past.

“I think the narrative of full autonomy is a 20th-century narrative,” Mindell says. “It’s a narrative of industrial mechanization that’s kind of filtered its way through the 20th century, supported by 20th-century science fiction. These narratives can and should change.”

Still, the idea of total automation is the approach taken by Google, most notably, in its development of self-driving cars. Yet as Mindell also observes, there are many challenges to the Google model: Its cars must identify all nearby objects correctly, need perfectly updated mapping systems, and must avoid all software glitches.

Ultimately, Mindell writes, “Google’s utopian autonomy is a more brittle, less functional solution than a rich, human-centered automation.” He predicts that the fully driverless model will not be the most successful, both for technical and social reasons.

“The notion of ceding control of something as fundamental to life as driving to a big, opaque corporation — people are not comfortable with that,” he says. Additionally, other companies and research groups looking at automating cars are “very clearly not going for the Google approach to fully driverless cars.”

Other scholars have found “Our Robots, Ourselves” to be valuable. Ian Bogost, a professor of media studies and interactive computing at Georgia Tech, calls the book “a lucid, hype-free exploration of how robotic automation really works” in tandem with human design and operation.

Mindell says he is eager to see how technologists, especially robotics engineers, react to the book. Among the places where Mindell is scheduled to speak on his current book tour are Microsoft and, yes, Google. In time, Mindell says, he believes his perspective will come to be more widely accepted, and that full automation on the roads will not seem as desirable a goal.

“I think the public discourse is slowly coming around to there is another way to do it,” Mindell concludes.

Reprinted with permission of MIT News.

Virtual assistants are setting the bar for the kind of user experience people will expect from the social robots of tomorrow.

At the Apple party earlier this month, at the Bill Graham Civic Auditorium in San Francisco, CEO Tim Cook and other key execs took to the stage to announce their next-generation product offerings. Apple’s been setting trends in user interface (UI) and technical design for over two decades now and its latest products didn’t disappoint: Siri already inhabits our iPhones and responds to voice commands, but now she will soon be embedded in Apple’s TV remotes too, so that users can change channels, find weather forecasts and get sports scores simply asking for them.

Siri demonstrates just how far we’ve come from the early days of interfacing with computers, databases and other information systems. Whereas systems used to be able only to respond to constrained queries, such as a Google search, Siri is now intelligent enough to respond appropriately to fairly detailed, human-like, requests, such as “Show me the Modern Family episode with Edward Norton” or “Skip ahead seven minutes.” She can even respond to questions such as: “What did he say?” Siri will simply skip back 15 seconds to the previous line the actor said. It’s an innovative new interface for controlling TV and a host of other media, and it’s likely to change how we interface with computing even more significantly than the introduction of the GUI (which, if we remember, Apple also had a hand in rolling out).

In addition to Siri, there’s a swarm of other software robots descending from the clouds: Google’s Google Now, Microsoft’s Cortana, Nuance’s Nina, and the Amazon Echo, which plays music, reads books and can buy stuff listed on their site. All of them keep track of what you like, when, and are able to draw massive amounts of data from both your voice (gender, age, region and other vectors) as well as the words you use. Baidu announced Duer in the first week of September with the intention of providing a voice interface for the home and internet of things (IoT) services and healthcare support. The company also  plans to integrate it into self-driving cars.

And recently Facebook’s M was touted as a concierge service available through its messaging app. This is a smart play, as Facebook is manning the assistant with a crowd of  employees (called M trainers) to simultaneously answer requests and train the system to improve responses. It seems a short step for Facebook to place ads in M (much as they do in site content today), but the real revenue stream will be in collecting user data. For the time being, as with Baidu’s Duer, Facebook’s M is not voice-driven, but uses text as a feature within the Messenger app.

Our mobile devices are becoming natural language interface hubs for life management and, as a result, having a gravitational pull on an increasingly complex buzz of connected services and APIs.

There are many others: Samsung’s S Voice, LG’s VoiceMate, BlackBerry’s Assistant, Sirius, HTC’s Hidi, Silvia and Braina, to name a few.

We can also foretell the future by looking at less advanced natural language systems. Bots – essentially natural language oriented scripts – are a good indicator of where the robotics industry is at because bots are pervasive, useful, and simple to author. TwitterBots and FacebookBots crawl through these systems like bees in a hive, industriously providing retweets, reposts, summaries, aggregations, starting fights and flocking to followers. They can be bought, auctioned, sold, and deleted; you can buy 30,000 Twitter followers on eBay for as little as for $20, provided they’re all bots.

Several years ago Facebook estimated that around five percent of all accounts are bogus – this would put the number of Facebook fakes at around 50 million – while other, now-antique estimates range as high as 27 percent (that’s about 200 million trash-trawling bots). They’re relevant because emergent technology arrives from the fringe.

According to Gartner’s 2015 Hype Cycle chart, Intelligent Virtual Assistants still have a ways to go before they get good enough to go mainstream (likely, 5-10 years). But also according Gartner, roughly 38% of American consumers have recently used virtual assistant services. They predict that, by the end of 2016, around two-thirds of consumers will be using them daily. Other sources forecast that the global Virtual Assistant market will grow at a CAGR of 39.3% between 2015-2018, and the total market is projected to climb to more than $2.1bn by 2019. These numbers cluster around the trends in natural language interfaces.

These trends show no sign of altering their flight path. Our mobile devices are becoming natural language interface hubs for life management and, as a result, having a gravitational pull on an increasingly complex buzz of connected services and APIs. This means that things like search will change: we will no longer have to speak Googlese; paper and page metaphors will be supplanted by the more dynamic (and cognitively more addictive) character metaphor. And if trends in virtual assistants and intelligent helpers – software robots – continue, then knowledge-bases (such as Wolfram Alpha or IBM Watson) will continue to come peppered with a patina of natural language, allowing us to move through data faster, with less training, and in a more human manner.

This is the trend in software robots, and it is headed towards hardware, too.

Hardware is the new software

Software robots show us what hardware robots will be expected to do in the near future. First, natural language is the de-facto interface for a range of functions from social robotics to customer service and personal healthcare companions – not only because it increases the quantity of data collected, but also because it decreases the cost of collecting it. Not only because it replaces people, but because it amplifies them.

That said, the natural language interface, or voice UI, is only one of many that robots will be expected to provide. Virtual assistants can now perform a range of functions based on the online services that are commonly integrated. Since these services can now be integrated into any connected system, we can put together a laundry list of what hardware robots will be expected to do:

• Understand voice input (respond to commands, answer questions, etc.)
• Offer personal assistance (schedule appointments, set reminders, check stocks, etc.)
• Provide mobility assistance (give directions, predict traffic, find parking, etc.)
• Interface with connected devices (TVs, garage doors, lights, surveillance cameras, etc.)
• Allow you to buy stuff (and keep track of what you’ve already bought)
• Interface with social media (send photos, read tweets, etc.)
• Detect mood (read emotions, body language, save state, etc.)
• Build predictive models (track user behavior, save user state, etc.)

Robotics will increasingly adopt a voice UI, akin to what we see in today’s personal assistants, because task completion is simpler, faster, and more effective, menus go away and personality (the UX of NLP) is a lot of fun. These software robots are testbeds and proving grounds for what users expect from tomorrow’s hardware robots; they set a bar for user experience. Personal assistants, like their distant cousins from a hardware lineage, will soon be smart enough to reply to other robots that call us on the phone, and we will want to equip our Jibo, Nest or Alitalia system with it’s own resident assistant that will answer the phone when the robot from the cable company calls.

Whether you happen to be Siri or the gal working for customer service at the cable company, your job, your life, your car and your family is being invaded by the swarms of personal assistants changing tomorrow’s robotics industry today. We have a choice: either we design hardware robots as assistants and companions (to help us accomplish tasks and keep us company), or we design them like bots (to mull the garbage, aggregate the trash, and harass one another).

The future, it seems, contains both.

Many robots are capable of picking up a part if the part is presented to the robot at a particular location. However, if the part has shifted from the expected location, the robot may no longer be able to grasp it … and if the part is sufficiently distant from its expected location, the robot could – as it attempts to grasp it – bump into it, push it further away, and jam the material handling system. This can, in turn, trigger a system fault and shut down the whole line — all because the robot does not know where the boundary between success and failure lies.

Robots function well in large scale industrial applications when reliability is designed into the system. This is accomplished by designing specialized hardware and software, extensively testing the system to identify potential failure spots, and developing contingency plans to handle failures when they occur. When a robot fails at a task, it often requires human intervention to clear the fault and restart the process, and this can be expensive. As such, robots are not typically used to execute a task until extremely high levels of reliability can be achieved — this is why customized hardware and software costs are typically only justified when the production volume is sufficiently high and the tasks are repetitive (such as in automotive assembly lines).

If we want to be able to use robots in small production batch operations or for non-repetitive tasks, however, we will need to prevent major system failures from occurring in the first place. Imagine a robot that could estimate the likelihood of completing its task before it even begins, and that calls for a human to help it figure out the problem when it foresees failure. A human operator could then provide the robot with assistance with the portion of the task that it finds challenging (for example, determining the orientation of a part, or finding a grasping strategy), leaving the robot to execute the remainder of the task itself.

In most situations, human intervention — at the right time, before the problem occurs — is far less costly than recovering from a system shutdown.

My students have been building a robot, called RoboSAM (ROBOtic Smart Assistant for Manufacturing), to demonstrate this concept in bin picking applications, which challenge a robot’s ability to perceive the desired object in the environment and to successfully pick it up and deliver it in a known pose. If RoboSAM is not sure whether it can pick the desired part from a bin containing many different parts, then it calls a remotely located human operator for help. The “human-on-call” concept, as we call it, is fundamentally different from the human-in-the-loop approach, which requires the human operator to actively monitor the manufacturing cell and take control away from the robot when the robot is about to make a mistake. “Human-on-call” requires the robot to call the human operator when it decides that it needs help. 

This model allows a single remotely situated human operator to help multiple robots on an “as need” basis. It also enables robots to be deployed on more difficult tasks. For the foreseeable future, many tasks in small and medium manufacturing companies fall into this category, making the human-on-call concept the right economic model for deploying robots in these contexts.

People often ask what humans will do when robots become more widespread. In my opinion, humans will be needed to teach robots how to do different tasks and bail robots out when they are confused. The key will be to develop technologies that allow robots to ask for help when needed. Recent work in our lab is a step in that direction.

If you like this post, you may also be interested in:

• Factory-in-a-Day: EU FP7 invests €7.9M to make robotics affordable for SMEs
• Low-cost robots like Baxter, UR5 and UR10 successfully entering small and medium enterprises (SMEs)
• Takeoff in robotics will power the next productivity surge in manufacturing
• What’s hot in robotics? Four trends to watch
• ABB launches YuMi and acquires gomTec
• Team RBO from Berlin wins Amazon Picking Challenge convincingly
• People and Robots: UC’s new multidisciplinary CITRIS initiative wants humans in the loop
• National Science Foundation and federal partners award $31.5M to advance co-robots in US

The benefits of assistive technologies are well known – reduced inefficiencies, low costs, available 24/7 and much more. But not so well known are the appropriate legal regulations that need to be negotiated in order for much assistive technology to be put to good use. A new interdisciplinary international workshop seeks to cover this terrain, bridging the gap between two very different disciplines.

The problems are multi-faceted and must be carefully considered: from technical issues inherent in cloud robotics, to security, autonomy, privacy, and legal issues, such as the regulation of replacement of organs by robotic devices. Then there are ethical issues relevant to the acceptance of assistive technology by the general public. Despite recent advances, the multidisciplinary nature of this field means there’s still a long way to go to overcome these problems, and new ones are likely to arise.

In an effort to bridge these different worlds, we conceived an Interdisciplinary International Workshop on the Legal, Ethical and Social Issues regarding Therapeutic and Educational Robotics. It will take place at The 2015 New Friends Conference in the Netherlands this autumn. Here, experts and researchers from different backgrounds are invited to debate the issues that the assistive robotic technology will raise, and propose simple solutions to mitigate challenges.

Broad in scope, the workshop aims to cover not only legal and ethical issues, but to create intelligible bridges between different disciplines: a technological problem raised by a psychologist could, for example, be solved simply by applying privacy enhancing knowledge; the same solution may also be applicable to concerns raised by a caregiver in a hospital.

We also seek to promote inter-community debate on the differing viewpoints various disciplines may hold about what appear to be – on the surface – the same topics. Privacy issues, for instance, are vital to the legal community; they may not appear relevant to roboticists but, in fact, Human Robot Interaction (HRI) is also concerned with privacy. In HRI, it’s about robots invading personal space and is related to proxemics. Another example is the psycho-social problems that loneliness and age-related depression bring. Technologies capable of socially connecting elderly people to the larger world could help mitigate these.

Philosophers, judges, lawyers, academics, psychologists, sociologists, anthropologists, cognitive scientists, engineers, care professionals, practitioners, bioethicists, eco-friendly professionals, communication and audiovisual workers, are all invited to share their thoughts, experiences and concerns.

By combining short presentations about problem areas with panel discussions that provide solutions, and by encouraging researchers from diverse fields who share interests in and concerns about robotics, we aim to cover much ground. The goals are to pave the way for assistive technology’s introduction into healthcare facilities, to reduce the workload of caregivers, and to assist in the therapy of patients.

The Interdisciplinary International Workshop on the Legal, Ethical and Social Issues regarding Therapeutic and Educational Robotics takes place at the New Friends Conference –  the first International Conference on Social Robots in Therapy and Education. It will be held in Almere, The Netherlands on 22 and 23 October 2015.

We have a tall order when it comes to dreaming up a trustworthy care robot: a robot could clean the house, find and fetch objects, and even keep seniors company. But if robots take on so many daily care tasks for the elderly, is it possible that seniors will have to interact with them too much? Is there such a thing as a socially acceptable amount of interaction with a care robot? Let us know what you think as we continue our reader polls about care robots.

Credit: eu-nited/KUKA

Employees of both companies are jointly conducting field tests to explore processes such as assembly and in- vehicle screw application. They are also developing systems concepts to make cooperation between humans and robots safe.

Direct human-robot interaction makes it possible to employ trendsetting manufacturing concepts, where the lightweight robot acts as a worker’s “third hand”. The lightweight robot was originally developed by the German Aerospace Center (Deutsches Zentrum für Luft- und Raumfahrt) for use in outer space. Its sensitive motorized grippers give it a delicate touch, which enables it to handle objects gently and perform difficult tasks precisely. The robot can be positioned and set up to optimally support workers in terms of ergonomics. As an example, the lightweight robot takes over and performs tiring tasks such as steps that involve handling items overhead. Working with and handling the robot is straightforward and intuitive, which reduces programming time and increases the efficiency of manufacturing processes. Due to their precise performance lightweight robots also contribute to enhanced quality.

This partnership is a continuation of the successful and innovation driven collaboration between the two companies. Daimler and KUKA launched their first joint pilot project at the Mercedes-Benz Untertürkheim plant in 2009. Since then, more than 500,000 rear axle gearboxes have been assembled with the support of the lightweight robot.

Credit: UCSC Bionics Lab, Raven IV

This 4-year multi-campus research project seeks to emulate the expert-apprentice relationship using human beings and robots. It focuses on developing ways in which robots can learn from human activity in order to help humans by providing extra hands, eyes and brain power as necessary, enabling multilateral manipulation from multiple vantage points. The range of applications includes manufacturing and surgery.
Multilateral manipulation systems have the potential to improve healthcare, improve American competitiveness and product quality in manufacturing, and open the door to new service robot applications in the home
Read more