Image from adarit.com/

In this episode, Audrow Nash interviews Robert Williamson, a Professor at the Australian National University, who speaks about a mathematical approach to ethics. This approach can get us started implementing robots that behave ethically. Williamson goes through his logical derivation of a mathematical formulation of ethics and then talks about the cost of fairness. In making his derivation, he relates bureaucracy to an algorithm. He wraps up by talking about how to work ethically.

Robert Williamson

Robert (Bob) Williamson is a professor in the research school of computer science at Australian National University. Until recently he was the chief scientist of DATA61, where he continues as a distinguished researcher. He served as scientific director and (briefly) CEO of NICTA, and lead its machine learning research group. His research is focussed on machine learning. He is the lead author of the ACOLA report Technology and Australia’s Future. He obtained his PhD in electrical engineering from the university of Queensland in 1990. He is a fellow of the Australian Academy of Science.

Links

• Download mp3 (14.4 MB)
• Robert Williamson’s homepage
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

The pilot projects

The three companies’ accomplished voucher work varies a lot and sets the basis to enlarge the array of industrial challenges tackled in the pilot work of ROBOTT-NET.

Linak’s pilot project will address some of the barriers that are still hindering the introduction of robots in High-Mix-Low-Volume (HMLV) production. This pilot will focus on the development of a modular assembly robot solution called MARS, that can be introduced in a wide-range of industries. You can check out Linak’s voucher project here:

Irabia’s pilot project UrbanPestPatrol aims to use robots for pest management in industrial facilities. The purpose of the project is to minimize the risk of a pest infestation and its resulting cost. You can learn about Irabia’s voucher project here:

https://www.youtube.com/watch?v=1zmHDOa1ErU

Lastly, Nissan will be looking into automated kitting for complex and diverse applications. This pilot will build a robotic test cell that enables Nissan Motor Manufacturting UK to develop automated kitting capability for its part families. Find out more about Nissan’s voucher project here:

ROBOTT-NET pilots help companies develop their voucher work through proof-of-concept level and accelerate it towards commercialisation. A pilot project will be a medium-scale research prototype installation, lasting for up to 18 months developing the robot technology and business case explored in the voucher stage and applying it to an industrial application demonstration at an end user’s site.

If you want to learn more about the pilot projects, you can check out ROBOTT-NETs website.

In this interview, Audrow speaks with Andrea Bajcsy and Dylan P. Losey about a method that allows robots to infer a human’s objective through physical interaction. They discuss their approach, the challenges of learning complex tasks, and their experience collaborating between different universities.

Some examples of people working with the more typical impedance control (left) and Bajcsy and Losey’s learning method (right).

To learn more, see this post on Robohub from the Berkeley Artificial Intelligence Research (BAIR) Lab.

Andrea Bajcsy
Andrea Bajcsy is a Ph.D. student in Electrical Engineering and Computer Sciences at the University of California Berkeley. She received her B.S. degree in Computer Science at the University of Maryland and was awarded the NSF Graduate Research Fellowship in 2016. At Berkeley, she works in the Interactive Autonomy and Collaborative Technologies Laboratory researching physical human-robot interaction.

Dylan P. Losey

Dylan P. Losey received the B.S. degree in mechanical engineering from Vanderbilt University, Nashville, TN, USA, in 2014, and the M.S. degree in mechanical engineering from Rice University, Houston, TX, USA, in 2016.

He is currently working towards the Ph.D. degree in mechanical engineering at Rice University, where he has been a member of the Mechatronics and Haptic Interfaces Laboratory since 2014.  In addition, between May and August 2017, he was a visiting scholar in the Interactive Autonomy and Collaborative Technologies Laboratory at the University of California, Berkeley.  He researches physical human-robot interaction; in particular, how robots can learn from and adapt to human corrections.

Mr. Losey received an NSF Graduate Research Fellowship in 2014, and the 2016 IEEE/ASME Transactions on Mechatronics Best Paper Award as a first author.

Links

• InterACT Lab
• MAHL Lab
• Andrea Bajcsy’s website
• Dylan’s Losey’s website
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon
• Download MP3 (14.9 MB)

In this episode, Audrow Nash speaks with Maja Matarić, a professor at the University of Southern California and the Chief Science Officer of Embodied, about socially assistive robotics. Socially assistive robotics aims to endow robots with the ability to help people through individual non-contact assistance in convalescence, rehabilitation, training, and education. For example, a robot could help a child on the autism spectrum to connect to more neurotypical children and could help to motivate a stroke victim to follow their exercise routine for rehabilitation (see the videos below). In this interview, Matarić discusses the care gap in health care, how her work leverages research in psychology to make robots engaging, and opportunities in socially assistive robotics for entrepreneurship.

A short video about how personalized robots might act as a “social bridge” between a child on the autism spectrum and a more neurotypical child.

A short video about how a robot could assist stroke victims in their recovery.

Maja Matarić

Maja Matarić is professor and Chan Soon-Shiong chair in Computer Science Department, Neuroscience Program, and the Department of Pediatrics at the University of Southern California, founding director of the USC Robotics and Autonomous Systems Center (RASC), co-director of the USC Robotics Research Lab and Vice Dean for Research in the USC Viterbi School of Engineering. She received her PhD in Computer Science and Artificial Intelligence from MIT, MS in Computer Science from MIT, and BS in Computer Science from the University of Kansas. 

Links

• Interaction Lab
• Embodied
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon
• Download MP3 (13.7 MB)

In this episode, Abate interviews Peter Harris from HighRes Biosolutions about automation in the field of drug discovery. At HighRes Biosolutions they are developing modular robotic systems that work alongside scientists to automate laboratory tasks. Because the requirements of each biomedical research laboratory are so varied, the robotic systems are specifically tailored to meet the requirements of each lab.

Peter Harris
Peter Harris is the CEO of HighRes Biosolutions. Prior to HighRes, Peter was VP and Managing Director at Axel Johnson, Inc. He spent most his career as the President & CEO of Cadence, Inc., a high technology medical device manufacturing and engineering firm enabling medical companies to bring better devices to market faster. Peter has been a Visiting Executive Lecturer at the Darden School of Business at the University of Virginia for over 10 years.

Links

• HighRes Biosolutions Website
• “Collaborative robots are redefining lab automation” by Peter Harris
• Peter Harris LinkedIn
• Download mp3 (16.6 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Audrow Nash interviews Ayanna Howard, Professor at the Georgia Institute of Technology, about her work to help children with the movement disorder cerebral palsy. Howard discusses how robots and tablet can be used to “gamify” pediatric therapy. The idea is that if therapy is fun and engaging children are more likely to do it, and thus, they are more likely to see the long-term benefits of the therapy. Howard discusses how therapy is “gamified,” how a small humanoid robot is used to coach children, and how they work with pediatricians.

Ayanna Howard

Ayanna Howard, Ph.D. is Professor and Linda J. and Mark C. Smith Endowed Chair in Bioengineering in the School of Electrical and Computer Engineering at the Georgia Institute of Technology. She also holds the position of Associate Chair for Faculty Development in ECE. She received her B.S. in Engineering from Brown University, her M.S.E.E. from the University of Southern California, and her Ph.D. in Electrical Engineering from the University of Southern California.

Her area of research is centered around the concept of humanized intelligence, the process of embedding human cognitive capability into the control path of autonomous systems. This work, which addresses issues of autonomous control as well as aspects of interaction with humans and the surrounding environment, has resulted in over 200 peer-reviewed publications in a number of projects – from scientific rover navigation in glacier environments to assistive robots for the home. To date, her unique accomplishments have been highlighted through a number of awards and articles, including highlights in USA Today, Upscale, and TIME Magazine, as well as being named a MIT Technology Review top young innovator and recognized as one of the 23 most powerful women engineers in the world by Business Insider.

In 2013, she also founded Zyrobotics, which is currently licensing technology derived from her research and has released their first suite of therapy and educational products for children with differing needs. From 1993-2005, Dr. Howard was at NASA’s Jet Propulsion Laboratory, California Institute of Technology. She has also served a term as the Associate Director of Research for the Georgia Tech Institute for Robotics and Intelligent Machines and a term as Chair of the multidisciplinary Robotics Ph.D. program at Georgia Tech.

Links

• Download mp3 (21.9 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Instead of worrying so much about robots taking away jobs, maybe we should worry more about wages being too low for robots to even get a chance. Seasonal labor for harvesting agricultural products, particularly fruits and vegetables, is dependent on human labor from a diminishing universe of willing workers.

Robots that can supplement or replace human workers in the harvesting process are being developed and tested in startups and academia, but almost all are not yet ready for prime time.

In a NY Times article written by Neil Irwin entitled Rethinking Low Productivity, productivity growth has been on a downward path since the Financial Crisis. Irwin, who writes about economic trends, asks whether the downward trend is the cause of low growth, or the result, a troubling question in the dynamics of the agriculture industry.

“Inventors and business innovators are always developing better ways to do things, but it takes a labor shortage and high wages to coax firms to deploy the investment it takes to actually put those innovations into widespread use.”

Those two phenomena are happening today in the U.S. agricultural industry. Labor rates are rising while willing laborers are diminishing — yet farmers are not yet investing in robotics. In fact, they are accommodating to the labor crisis by planting less and changing the crops they grow to be less labor intensive.

J.W. Mason, the author of a paper by the Roosevelt Institute cited as the basis for the thesis of Irwin’s article, says:

“If the labor market tightens and wages rise, that will be the impetus to get companies to consider more big-ticket innovations that generate productivity growth so that we don’t perpetuate the present conundrum in which both arguments cannot be true:

“On Mondays and Wednesdays, economists argue that wages are low because robots are taking people’s jobs.

On Tuesdays and Thursdays, it’s that we can’t have wages rise because productivity growth is low.”

This situation cannot continue without radical changes else we will become buyers of foreign fruits and vegetables paying prices set by foreign providers and dependent on them for availability and quality.

A number of factors are causing a reset in the ag industry in addition to the declining availability of farm workers: the challenges and complexities of employing and retaining farm labor; the rising cost of farm workers; changing farmlands; climate change; the growth of indoor farming; and the broader automation of the agriculture industry. All are propelling farmers to make changes in how they farm (or change downward the product mix they farm (which is often a path toward failure)). Market challenges for the sector include unclear value propositions, limited awareness of robotic systems among growers, insufficient robotic solutions, the difficulty of matching human-like dexterity with machines, fragmented technology development, and weak support.

Tractica, in their report on agricultural robotics, forecast an optimistic resolution to this temporary conundrum in the very near term and that that shipments of agricultural robots will increase significantly in the years ahead, rising from 32,000 units in 2016 to 594,000 units annually by 2024, by which time the market is expected to reach $74 billion in annual revenue. Certainly the time is ripe and, like Mason said, “Both arguments can’t be true.”

By Jeff Morgan, Trinity College Dublin

Robots have been taking our jobs since the 1960s. So why are politicians and business leaders only now becoming so worried about robots causing mass unemployment?

It comes down to the question of what a robot really is. While science fiction has often portrayed robots as androids carrying out tasks in the much the same way as humans, the reality is that robots take much more specialised forms. Traditional 20th century robots were automated machines and robotic arms building cars in factories. Commercial 21st century robots are supermarket self-checkouts, automated guided warehouse vehicles, and even burger-flipping machines in fast-food restaurants.

Ultimately, humans haven’t become completely redundant because these robots may be very efficient but they’re also kind of dumb. They do not think, they just act, in very accurate but very limited ways. Humans are still needed to work around robots, doing the jobs the machines can’t and fixing them when they get stuck. But this is all set to change thanks to a new wave of smarter, better value machines that can adapt to multiple tasks. This change will be so significant that it will create a new industrial revolution.

Industry 4.0

This era of “Industry 4.0” is being driven by the same technological advances that enable the capabilities of the smartphones in our pockets. It is a mix of low-cost and high-power computers, high-speed communication and artificial intelligence. This will produce smarter robots with better sensing and communication abilities that can adapt to different tasks, and even coordinate their work to meet demand without the input of humans.

In the manufacturing industry, where robots have arguably made the most headway of any sector, this will mean a dramatic shift from centralised to decentralised collaborative production. Traditional robots focused on single, fixed, high-speed operations and required a highly skilled human workforce to operate and maintain them. Industry 4.0 machines are flexible, collaborative and can operate more independently, which ultimately removes the need for a highly skilled workforce.

For large-scale manufacturers, Industry 4.0 means their robots will be able to sense their environment and communicate in an industrial network that can be run and monitored remotely. Each machine will produce large amounts of data that can be collectively studied using what is known as “big data” analysis. This will help identify ways to improve operating performance and production quality across the whole plant, for example by better predicting when maintenance is needed and automatically scheduling it.

For small-to-medium manufacturing businesses, Industry 4.0 will make it cheaper and easier to use robots. It will create machines that can be reconfigured to perform multiple jobs and adjusted to work on a more diverse product range and different production volumes. This sector is already beginning to benefit from reconfigurable robots designed to collaborate with human workers and analyse their own work to look for improvements, such as BAXTER, SR-TEX and CareSelect.

While these machines are getting smarter, they are still not as smart as us. Today’s industrial artificial intelligence operates at a narrow level, which gives the appearance of human intelligence exhibited by machines, but designed by humans.

What’s coming next is known as “deep learning”. Similar to big data analysis, it involves processing large quantities of data in real time to make decisions about what is the best action to take. The difference is that the machine learns from the data so it can improve its decision making. A perfect example of deep learning was demonstrated by Google’s AlphaGo software, which taught itself to beat the world’s greatest Go players.

The turning point in applying artificial intelligence to manufacturing could come with the application of special microchips called graphical processing units (GPUs). These enable deep learning to be applied to extremely large data sets at extremely fast speeds. But there is still some way to go and big industrial companies are recruiting vast numbers of scientists to further develop the technology.

Impact on industry

As Industry 4.0 technology becomes smarter and more widely available, manufacturers of any size will be able to deploy cost-effective, multipurpose and collaborative machines as standard. This will lead to industrial growth and market competitiveness, with a greater understanding of production processes leading to new high-quality products and digital services.

Exactly what impact a smarter robotic workforce with the potential to operate on its own will have on the manufacturing industry, is still widely disputed. Artificial intelligence as we know it from science fiction is still in its infancy. It could well be the 22nd century before robots really have the potential to make human labour obsolete by developing not just deep learning but true artificial understanding that mimics human thinking.

Ideally, Industry 4.0 will enable human workers to achieve more in their jobs by removing repetitive tasks and giving them better robotic tools. In theory, this would allow us humans to focus more on business development, creativity and science, which it would be much harder for any robot to do. Technology that has made humans redundant in the past has forced us to adapt, generally with more education.

But because Industry 4.0 robots will be able to operate largely on their own, we might see much greater human redundancy from manufacturing jobs without other sectors being able to create enough new work. Then we might see more political moves to protect human labour, such as taxing robots.

Again, in an ideal scenario, humans may be able to focus on doing the things that make us human, perhaps fuelled by a basic income generated from robotic work. Ultimately, it will be up to us to define whether the robotic workforce will work for us, with us, or against us.

This article was originally published on The Conversation. Read the original article.

By Christoph Salge, Marie Curie Global Fellow, University of Hertfordshire

How do you stop a robot from hurting people? Many existing robots, such as those assembling cars in factories, shut down immediately when a human comes near. But this quick fix wouldn’t work for something like a self-driving car that might have to move to avoid a collision, or a care robot that might need to catch an old person if they fall. With robots set to become our servants, companions and co-workers, we need to deal with the increasingly complex situations this will create and the ethical and safety questions this will raise.

Science fiction already envisioned this problem and has suggested various potential solutions. The most famous was author Isaac Asimov’s Three Laws of Robotics, which are designed to prevent robots harming humans. But since 2005, my colleagues and I at the University of Hertfordshire, have been working on an idea that could be an alternative.

Instead of laws to restrict robot behaviour, we think robots should be empowered to maximise the possible ways they can act so they can pick the best solution for any given scenario. As we describe in a new paper in Frontiers, this principle could form the basis of a new set of universal guidelines for robots to keep humans as safe as possible.

The Three Laws

Asimov’s Three Laws are as follows:

• A robot may not injure a human being or, through inaction, allow a human being to come to harm.
• A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
• A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

While these laws sound plausible, numerous arguments have demonstrated why they are inadequate. Asimov’s own stories are arguably a deconstruction of the laws, showing how they repeatedly fail in different situations. Most attempts to draft new guidelines follow a similar principle to create safe, compliant and robust robots.

One problem with any explicitly formulated robot guidelines is the need to translate them into a format that robots can work with. Understanding the full range of human language and the experience it represents is a very hard job for a robot. Broad behavioural goals, such as preventing harm to humans or protecting a robot’s existence, can mean different things in different contexts. Sticking to the rules might end up leaving a robot helpless to act as its creators might hope.

Our alternative concept, empowerment, stands for the opposite of helplessness. Being empowered means having the ability to affect a situation and being aware that you can. We have been developing ways to translate this social concept into a quantifiable and operational technical language. This would endow robots with the drive to keep their options open and act in a way that increases their influence on the world.

When we tried simulating how robots would use the empowerment principle in various scenarios, we found they would often act in surprisingly “natural” ways. It typically only requires them to model how the real world works but doesn’t need any specialised artificial intelligence programming designed to deal with the particular scenario.

But to keep people safe, the robots need to try to maintain or improve human empowerment as well as their own. This essentially means being protective and supportive. Opening a locked door for someone would increase their empowerment. Restraining them would result in a short-term loss of empowerment. And significantly hurting them could remove their empowerment altogether. At the same time, the robot has to try to maintain its own empowerment, for example by ensuring it has enough power to operate and it does not get stuck or damaged.

Robots could adapt to new situations

Using this general principle rather than predefined rules of behaviour would allow the robot to take account of the context and evaluate scenarios no one has previously envisaged. For example, instead of always following the rule “don’t push humans”, a robot would generally avoid pushing them but still be able to push them out of the way of a falling object. The human might still be harmed but less so than if the robot didn’t push them.

In the film I, Robot, based on several Asimov stories, robots create an oppressive state that is supposed to minimise the overall harm to humans by keeping them confined and “protected”. But our principle would avoid such a scenario because it would mean a loss of human empowerment.

While empowerment provides a new way of thinking about safe robot behaviour, we still have much work to do on scaling up its efficiency so it can easily be deployed on any robot and translate to good and safe behaviour in all respects. This poses a very difficult challenge. But we firmly believe empowerment can lead us towards a practical solution to the ongoing and highly debated problem of how to rein in robots’ behaviour, and how to keep robots -– in the most naive sense -– “ethical”.

This article was originally published on The Conversation. Read the original article.

In this episode, MeiXing Dong conducts interviews at the 2017 Midwest Speech and Language Days workshop in Chicago. She talks with Michael White of Ohio State University about question interpretation in a dialogue system; Dmitriy Dligach of Loyola University Chicago about extracting patient timelines from doctor’s notes; and Denis Newman-Griffiths of Ohio State University about connecting words and phrases to relevant medical topics.

Links

• Download mp3 (14.7 MB)
• Midwest Speech and Language Days 2017 Website
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

By Eran Klein, University of Washington and Katherine Pratt, University of Washington

In the 1995 film “Batman Forever,” the Riddler used 3-D television to secretly access viewers’ most personal thoughts in his hunt for Batman’s true identity. By 2011, the metrics company Nielsen had acquired Neurofocus and had created a “consumer neuroscience” division that uses integrated conscious and unconscious data to track customer decision-making habits. What was once a nefarious scheme in a Hollywood blockbuster seems poised to become a reality.

Recent announcements by Elon Musk and Facebook about brain-computer interface (BCI) technology are just the latest headlines in an ongoing science-fiction-becomes-reality story.

BCIs use brain signals to control objects in the outside world. They’re a potentially world-changing innovation – imagine being paralyzed but able to “reach” for something with a prosthetic arm just by thinking about it. But the revolutionary technology also raises concerns. Here at the University of Washington’s Center for Sensorimotor Neural Engineering (CSNE) we and our colleagues are researching BCI technology – and a crucial part of that includes working on issues such as neuroethics and neural security. Ethicists and engineers are working together to understand and quantify risks and develop ways to protect the public now.

Picking up on P300 signals

All BCI technology relies on being able to collect information from a brain that a device can then use or act on in some way. There are numerous places from which signals can be recorded, as well as infinite ways the data can be analyzed, so there are many possibilities for how a BCI can be used.

Some BCI researchers zero in on one particular kind of regularly occurring brain signal that alerts us to important changes in our environment. Neuroscientists call these signals “event-related potentials.” In the lab, they help us identify a reaction to a stimulus.

In particular, we capitalize on one of these specific signals, called the P300. It’s a positive peak of electricity that occurs toward the back of the head about 300 milliseconds after the stimulus is shown. The P300 alerts the rest of your brain to an “oddball” that stands out from the rest of what’s around you.

For example, you don’t stop and stare at each person’s face when you’re searching for your friend at the park. Instead, if we were recording your brain signals as you scanned the crowd, there would be a detectable P300 response when you saw someone who could be your friend. The P300 carries an unconscious message alerting you to something important that deserves attention. These signals are part of a still unknown brain pathway that aids in detection and focusing attention.

Reading your mind using P300s

P300s reliably occur any time you notice something rare or disjointed, like when you find the shirt you were looking for in your closet or your car in a parking lot. Researchers can use the P300 in an experimental setting to determine what is important or relevant to you. That’s led to the creation of devices like spellers that allow paralyzed individuals to type using their thoughts, one character at a time.

It also can be used to determine what you know, in what’s called a “guilty knowledge test.” In the lab, subjects are asked to choose an item to “steal” or hide, and are then shown many images repeatedly of both unrelated and related items. For instance, subjects choose between a watch and a necklace, and are then shown typical items from a jewelry box; a P300 appears when the subject is presented with the image of the item he took.

Everyone’s P300 is unique. In order to know what they’re looking for, researchers need “training” data. These are previously obtained brain signal recordings that researchers are confident contain P300s; they’re then used to calibrate the system. Since the test measures an unconscious neural signal that you don’t even know you have, can you fool it? Maybe, if you know that you’re being probed and what the stimuli are.

Techniques like these are still considered unreliable and unproven, and thus U.S. courts have resisted admitting P300 data as evidence.

Imagine that instead of using a P300 signal to solve the mystery of a “stolen” item in the lab, someone used this technology to extract information about what month you were born or which bank you use – without your telling them. Our research group has collected data suggesting this is possible. Just using an individual’s brain activity – specifically, their P300 response – we could determine a subject’s preferences for things like favorite coffee brand or favorite sports.

But we could do it only when subject-specific training data were available. What if we could figure out someone’s preferences without previous knowledge of their brain signal patterns? Without the need for training, users could simply put on a device and go, skipping the step of loading a personal training profile or spending time in calibration. Research on trained and untrained devices is the subject of continuing experiments at the University of Washington and elsewhere.

It’s when the technology is able to “read” someone’s mind who isn’t actively cooperating that ethical issues become particularly pressing. After all, we willingly trade bits of our privacy all the time – when we open our mouths to have conversations or use GPS devices that allow companies to collect data about us. But in these cases we consent to sharing what’s in our minds. The difference with next-generation P300 technology under development is that the protection consent gives us may get bypassed altogether.

What if it’s possible to decode what you’re thinking or planning without you even knowing? Will you feel violated? Will you feel a loss of control? Privacy implications may be wide-ranging. Maybe advertisers could know your preferred brands and send you personalized ads – which may be convenient or creepy. Or maybe malicious entities could determine where you bank and your account’s PIN – which would be alarming.

With great power comes great responsibility

The potential ability to determine individuals’ preferences and personal information using their own brain signals has spawned a number of difficult but pressing questions: Should we be able to keep our neural signals private? That is, should neural security be a human right? How do we adequately protect and store all the neural data being recorded for research, and soon for leisure? How do consumers know if any protective or anonymization measures are being made with their neural data? As of now, neural data collected for commercial uses are not subject to the same legal protections covering biomedical research or health care. Should neural data be treated differently?

These are the kinds of conundrums that are best addressed by neural engineers and ethicists working together. Putting ethicists in labs alongside engineers – as we have done at the CSNE – is one way to ensure that privacy and security risks of neurotechnology, as well as other ethically important issues, are an active part of the research process instead of an afterthought. For instance, Tim Brown, an ethicist at the CSNE, is “housed” within a neural engineering research lab, allowing him to have daily conversations with researchers about ethical concerns. He’s also easily able to interact with – and, in fact, interview – research subjects about their ethical concerns about brain research.

There are important ethical and legal lessons to be drawn about technology and privacy from other areas, such as genetics and neuromarketing. But there seems to be something important and different about reading neural data. They’re more intimately connected to the mind and who we take ourselves to be. As such, ethical issues raised by BCI demand special attention.

Working on ethics while tech’s in its infancy

As we wrestle with how to address these privacy and security issues, there are two features of current P300 technology that will buy us time.

First, most commercial devices available use dry electrodes, which rely solely on skin contact to conduct electrical signals. This technology is prone to a low signal-to-noise ratio, meaning that we can extract only relatively basic forms of information from users. The brain signals we record are known to be highly variable (even for the same person) due to things like electrode movement and the constantly changing nature of brain signals themselves. Second, electrodes are not always in ideal locations to record.

All together, this inherent lack of reliability means that BCI devices are not nearly as ubiquitous today as they may be in the future. As electrode hardware and signal processing continue to improve, it will be easier to continuously use devices like these, and make it easier to extract personal information from an unknowing individual as well. The safest advice would be to not use these devices at all.

The goal should be that the ethical standards and the technology will mature together to ensure future BCI users are confident their privacy is being protected as they use these kinds of devices. It’s a rare opportunity for scientists, engineers, ethicists and eventually regulators to work together to create even better products than were originally dreamed of in science fiction.

The device named “Spark” flew high above the man on stage with his hands waving in the direction of the flying object. In a demonstration of DJI’s newest drone, the audience marveled at the Coke can-sized device’s most compelling feature: gesture controls. Instead of a traditional remote control, this flying selfie machine follows hand movements across the sky. Gestures are the most innate language of mammals, and including robots in our primal movements means we have reached a new milestone of co-existence.

Madeline Gannon of Carnegie Mellon University is the designer of Mimus, a new gesture controlled robot featured in an art installation at The Design Museum in London, England. Gannon explained: “In developing Mimus, we found a way to use the robot’s body language as a medium for cultivating empathy between museum-goers and a piece of industrial machinery. Body language is a primitive yet fluid means of communication that can broadcast an innate understanding of the behaviors, kinematics and limitations of an unfamiliar machine.” Gannon wrote about her experiences recently in the design magazine Dezeen: “In a town like Pittsburgh, where crossing paths with a driverless car is now an everyday occurrence, there is still no way for a pedestrian to read the intentions of the vehicle…it is critical that we design more effective ways of interacting and communicating with them.”

So far, the biggest commercially deployed advances in human-robot interactions have been conversational agents by Amazon, Google and Apple. While natural language processing has broken new ground in artificial intelligence, the social science of its acceptability in our lives might be its biggest accomplishment. Japanese roboticist Masahiro Mori described the danger of making computer generated voices too indistinguishable from humans as the “uncanny valley.” Mori cautioned inventors from building robots that are too human sounding (and possibly looking) as the result elicits negative emotions best described as “creepy” and “disturbing.”

Recently, many toys have embraced conversational agents as a way of building greater bonds and increasing the longevity of play with kids. Barbie’s digital speech scientist, Brian Langner of ToyTalk, detailed his experiences with crossing into the “Uncanny Valley” as: “Jarring is the way I would put it. When the machine gets some of those things correct, people tend to expect that it will get everything correct.”

Kate Darling of MIT’s Media Lab, whose research centers on human-robot interactions, suggested that “if you get the balance right, people will like interacting with the robot, and will stop using it as a device and start using it as a social being.”

This logic inspired Israeli startup Intuition Robotics to create ElliQ—a bobbing head (eyeless) robot. The purpose of the animatronics is to break down barriers between its customer base of elderly patients and their phobias of technology. According to Intuition Robotics’ CEO, Dor Skuler, the range of motion coupled with a female voice helps create a bond between the device and its user. Don Norman, usability designer of ElliQ, said: “It looks like it has a face even though it doesn’t. That makes it feel approachable.”

Mayfield Robotics decided to add cute R2D2-like sounds to its newest robot, Kuri. Mayfield hired former Pixar designers Josh Morenstein and Nick Cronan of Branch Creative with the sole purpose of making Kuri more adorable. To accomplish this mission, Morenstein and Cronan gave Kuri eyes, but not a mouth as that would be, in their words “creepy.” Conan shares the challenges with designing the eyes: “Just by moving things a few millimeters, it went from looking like a dumb robot to a curious robot to a mean robot. It became a discussion of, how do we make something that’s always looking optimistic and open to listen to you?” Kuri has a remarkable similarity to Morenstein and Cronan’s former theatrical robot, EVA.

In the far extreme of making robots act and behave human, RealDoll has been promoting six thousand dollar sex robots. To many, RealDoll has crossed the “Uncanny Valley” of creepiness with sex dolls that look and talk like humans. In fact, there is a growing grassroots campaign to ban RealDoll’s products globally, as it endangers the very essence of human relationships. Florence Gildea writes on the organization’s blog: “The personalities and voices that doll owners project onto their dolls is pertinent for how sex robots may develop, given that sex doll companies like RealDoll are working on installing increasing AI capacities in their dolls and the expectation that owners will be able to customize their robots’ personalities.” The example given is how the doll expresses her “feelings” for her owner on Twitter:

Obviously a robot companion has no feelings, however it is a projection of the doll owners’. “To anthropomorphize their dolls to sustain the fantasy that they have feelings for the owner. The Twitter accounts seemingly manifest the dolls’ independent existence so that their dependence on their owners can seem to signify their emotional attachment, rather than it following inevitable from their status as objects. Immobility, then, can be misread as fidelity and devotion.” The implications of this behavior is that their female companion, albeit mechanical, enjoys “being dominated.” The fear that the Campaign Against Sex Robots expresses is the objectification of women (even robotic ones) reinforces problematic human sexual stereotypes.

Today, with technology at our fingertips, there is growing phenomena of preferring one-directional device relationships over complicated human encounters. MIT Social Sciences Professor Sherry Turkle writes in her essay, Close Engagements With Artificial Companionship, that “over-stressed, overworked, people claim exhaustion and overload. These days people will admit they’d rather leave a voice mail or send an email than talk face-to-face. And from there, they say: ‘I’d rather talk to the robot. Friends can be exhausting. The robot will always be there for me. And whenever I’m done, I can walk away.’”

In the coming years humans will communicate more with robots in their lives from experiences in the home to the office to their leisure time. The big question will be not the technical barriers, but the societal norms that will evolve to accept Earth’s newest species.

“What do we think a robot is?” asked robot designer Norm. “Some people think it should look like an animal or a person, and it should move around. Or it just has to be smart, sense the environment, and have motors and controllers.”

Norm’s answer, like beauty, could be in the eye of the beholder.

Today we are looking at how to program your NAO Robot for Human Interaction. Watch the video and follow the steps below to get interactive with your robot pal!

1.  Create an animation of movements
   • Right click, select “Create a new box”
   • Select “Timeline”

We want the robot to do 3 things, a five high, a hello, and a goodbye move:

• On the Timeline Edit Box
  • Name: High Five
  • Image: Click “Edit”
    • On the Edit box image
    • Click “Browse” and select a file
  • Click “OK”

Now, as you can remember, if we take it to the box, we can get access to the NAO’s Timeline. Mine now is connected to the PC. So what I’m going to do is I’m going to set some points of the NAO to do some movements. As I have said before on one of my tutorial I’m going to make it set all the joints to the whole body.

We want to make him look like he is doing a high five:

• Right click the arm
  • Click “Stiffen chain on/off”
  • Raise his arm up
• Set joints and keyframe
  • Select “Arms”
• Click “Play” on the Motion Controller

That’s a quite quick high five actually. What we want to do is to actually give him a pause when he got his arm in the air. So, the way we do that is to copy and paste the keyframe.

Grab a few already built in animation, just to show you how much it works:

• Go to Motions>Animations
  • Drag a “Hello” box to the workspace
  • Drag a “Wipe Forehead” box for a goodbye move to the workspace

We want to hear him speak, so:

• Go to Voice
  • Drag a “Speech Recognition” box to the workspace
• Drag a “Switch Case”
  • Connect the “Speech Recognition” box to the “Switch Case” box
  • On the first case, write “Hello” and connect to the “Hello” box
  • On the second case, write “High Five” and connect it to the “High Five” box
  • On the third case, write “Goodbye” and connect it to the “Wipe Forehand” box
• Set parameters of Speech Recognition
  • Word list: Hi; High Five; Goodbye;
  • Play with the threshold to see what works better
  • Click “OK”

So there we have a speech recognition, he will listen to what I am saying and then we will act a motion. Now, what we need to do is to make sure that the robot doesn’t repeat it, so you know, it doesn’t loop back. Before we do that, we are going to:

• Drag 3 “Say” box to the workspace so he can  speak back to us
  • On the “Switch case”, connect the “Hello” case on the “Say” box
    • Localized Text
      • Change the language to English(United States)
      • Write “Hi” on the textbox
  • Connect the “High Five” case on the “Say (1)” box
    • Localized Text
      • Change the language to English(United States)
      • Write “High Five” on the textbox
  • Connect the “Goodbye” case on the “Say(2)” box
    • Localized Text
      • Change the language to English(United States)
      • Write “Goodbye” on the textbox

So, here’s what we are saying, he recognizes it, says a word and does an animation, now what we want to do is we don’t want him to loop, as in he keeps hearing the same sound and maybe he starts to hear himself and repeat himself, so to stop that we need to connect the end of the motion boxes back to the “Speech Recognition” box. A simple output to the input speech.

I am going to make a little change actually, instead of “Hello”, we should put “Hi” here and I’m going to put “Hi” in there which is already done because they have to match otherwise if it doesn’t match he won’t recognize it. So if he hit “Load”

Philip: High Five!

Nao: High Five!

Philip: Goodbye!

Nao: Goodbye!

Philip: Hi!

Nao: Hello!

Brilliant! So that looks likes he’s working very well, so again if we says “Hi”, “High Five” or “Good bye” then he will greet you. If you walk in to room and you say “Hi” to a robot, obviously, he will do his animation and say a speech. Here you can use your imagination and try different things, different animations.

If you liked this article, you may also enjoy these from Robo-Phil:

• Python programming your NAO robot
• Hard at work: A review of the Laevo Exoskeleton

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Last week I had the pleasure of debating the question “does AI pose a threat to society?” with friends and colleagues Christian List, Maja Pantic and Samantha Payne. The event was organised by the British Academy and brilliantly chaired by the Royal Society’s director of science policy Claire Craig. Here follows my opening statement:

My opening statement

One Friday afternoon in 2009 I was called by a science journalist at, I recall, the Sunday Times. He asked me if I knew that there was to be a meeting of the AAAI to discuss robot ethics. I said no I don’t know of this meeting. He then asked “are you surprised they are meeting to discuss robot ethics” and my answer was no. We talked some more and agreed it was actually a rather dull story: a case of scientists behaving responsibly. I really didn’t expect the story to appear but checked the Sunday paper anyway, and there in the science section was the headline Scientists fear revolt of killer robots. (I then spent the next couple of days on the radio explaining that no, scientists do not fear a revolt of killer robots.)

So, fears of future super intelligence – robots taking over the world – are greatly exaggerated: the threat of an out-of-control super intelligence is a fantasy – interesting for a pub conversation perhaps. It’s true we should be careful and innovate responsibly, but that’s equally true for any new area of science and technology. The benefits of robotics and AI are so significant, the potential so great, that we should be optimistic rather than fearful. Of course robots and intelligent systems must be engineered to very high standards of safety for exactly the same reasons that we need our washing machines, cars and airplanes to be safe. If robots are not safe people will not trust them. To reach it’s full potential what robotics and AI needs is a dose of good old fashioned (and rather dull) safety engineering.

In 2011 I was invited to join a British Standards Institute working group on robot ethics, which drafted a new standard BS 8611 Guide to the ethical design of robots and robotic systems, published in April 2016. I believe this to be the world’s first standard on ethical robots.

Also in 2016 the very well regarded IEEE standards association – the same organization that gave us WiFi – launched a Global initiative on Ethical Considerations in AI and Autonomous Systems. The purpose of this Initiative is to ensure every technologist is educated and empowered to prioritize ethical considerations in the design and development of autonomous and intelligent systems; in a nutshell, to ensure ethics are baked in. In December we published Ethically Aligned Design: A Vision for Prioritizing Human Well Being with AI and Autonomous Systems. Within that initiative I’m also leading a new standard on transparency in autonomous systems, based on the simple principle that it should always be possible to find out why an AI or robot made a particular decision.

We need to agree ethical principles, because they are needed to underpin standards – ways of assessing and mitigating the ethical risks of robotics and AI. But standards needs teeth and in turn underpin regulation. Why do we need regulation? Think of passenger airplanes; the reason we trust them is because it’s a highly regulated industry with an amazing safety record, and robust, transparent processes of air accident investigation when things do go wrong. Take one example of a robot that we read a lot about in the news – the Driverless Car. I think there’s a strong case for a driverless car equivalent of the CAA, with a driverless car accident investigation branch. Without this it’s hard to see how driverless car technology will win public trust.

Does AI pose a threat to society? No. But we do need to worry about the down to earth questions of present day rather unintelligent AIs; the ones that are deciding our loan applications, piloting our driverless cars or controlling our central heating. Are those AIs respecting our rights, freedoms and privacy? Are they safe? When AIs make bad decisions, can we find out why? And I worry too about the wider societal and economic impacts of AI. I worry about jobs of course, but actually I think there is a bigger question: how can we ensure that the wealth created by robotics and AI is shared by all in society?

Thank you.

The featured image of this article was used to advertise the BA’s series of events on the theme Robotics, AI and Society. The reason I draw your attention to it here is that one of the many interesting questions to the panel was about the way that AI tends to be visualised in the media. This kind of human face coalescing (or perhaps emerging) from the atomic parts of the AI seems to have become a trope for AI. Is it a helpful visualisation of the human face of AI, or does it mislead to an impression that AI has human characteristics?

A few decades ago, touchscreens were impressive, yet clunky pieces of technology reserved for applications that did little more than show off that touchscreens were possible. Today’s touchscreens are commonplace and readily accepted as an easy way to interact with an operating system; they’re accurate, support multi-touch, are comfortable to use, and can even operate in multiple dimensions (pressing hard).

We may not have perfected touchscreens yet, but we’re getting close. Accordingly, engineers and researchers are already speculating about the next phase of UI development, especially for robotics control. So far, the leading candidate is gesture-based control—the use of physical gestures to relay commands.

The idea

The major limitation for touchscreens is the fact that they operate only in two dimensions; the third dimension introduced with force touch is extremely limited. Comparatively, hand gestures and physical movements can operate in three dimensions, and depending on how they’re designed, could feel more natural than swiping and tapping a smartphone screen.

Demand for three-dimensional gestural control is increasing with the onset of virtual reality (VR) and augmented reality (AR) technology; because the digital world we experience will be moving from two dimensions to three dimensions, the controls we use to manage those experiences will also need to change. With enough sophistication, these systems could provide better feedback to the users in control; rather than merely responding with visual feedback like movement or lights, users could feasibly be immersed with physical feedback like vibration or resistance.

Where we stand

Currently, one of the most advanced gestural systems is the Real-Time 3D Gesture Analysis for Natural Interaction with Smart Devices, a project headed by researchers in Linnaeus University in Sweden. However, simpler forms of gesture-based control are already available.

For example, there are projects that use Arduino to create a robot that can respond to four different simple hand gestures (plus a neutral position). Of course, iPhone technology also makes use of some simple “gestures,” such as shaking the phone to undo typing or rotating the phone to experience an immersive 360-degree view of an image or video.

The main obstacles

There are a few obstacles preventing gestures from being solidified as the next landmark in robotics control, however:

• Gesture complexity. The biggest obstacle described by researchers at Linnaeus University is the sheer complexity of physical gestures; to be effective, their recognition systems need to be able to gather and recognize thousands of tiny data points and interpret complex patterns to “understand” what movement is being attempted. Moreover, this big data-based interpretation needs to happen in real-time—especially for applications like live robotics control, or engagement in a VR-based video game. That demands not only an incredibly intelligent system, but a processor that can operate quickly.
• Accessibility. Gesture recognition systems would likely be developed to accommodate a “standard” human model, such as a “standard” right human hand. How would the system accommodate somebody whose hand was missing, or who is missing a few fingers? What about left-handed people? And people with Parkinson’s disease, or who are unable to operate their hands with precision control?
• Applications. For gestures to be accepted as a mainstream way to interact with robotics and virtual environments, robotics and virtual environments need to be available. Virtual reality (VR) technology has been advancing strongly for the past several years and is poised to take a big step forward in user adoption by 2020, but until those user adoption numbers are reliable, the demand for gestural systems is relatively low. Using hand movements and other physical gestures to control a two-dimensional screen, for example, would be largely ineffective.

On paper, gestures seem like the best form of control for the digital devices, robotics, and VR systems of the future. However, there are still many obstacles to overcome before we’re ready for large-scale adoption. Fortunately, researchers are ahead of the curve, already preparing the intelligent gesture-based recognition programs we’ll need when touchscreens become obsolete.

Robots raise all kinds of concerns. They could steal our jobs, as some experts think. And if artificial intelligence grows, it might even be tempted to enslave us, or to annihilate the whole of humanity. Robots are strange creatures, and not only for these frequently invoked reasons. We have good cause to be a little worried about these machines.

Imagine that you are visiting the Quai Branly-Jacques Chirac, a museum in Paris dedicated to anthropology and ethnology. As you walk through the collection, your curiosity leads you to a certain piece. After a while, you begin to sense a familiar presence heading towards the same objet d’art that has caught your attention.

You move slowly, and as you turn your head a strange feeling seizes you because what you seem to distinguish, still blurry in your peripheral vision, is a not-quite-human figure. Anxiety takes over.

As your head turns, and your vision become sharper, this feeling gets stronger. You realise that this is a humanoid machine, a robot called Berenson. Named after the American art critic Bernard Berenson and designed by the roboticist Philippe Gaussier (Image and Signal processing Lab) and the anthropologist Denis Vidal (Institut de recherche sur le développement), Berenson is part of an experiment underway at the Quai Branly museum since 2012.

The strangeness of the encounter with Berenson leaves you suddenly frightened, and you step back, away from the machine.

The uncanny valley

This feeling has been explored in robotics since the 1970s, when Japanese researcher Professor Masahiro Mori proposed his “uncanny valley” theory. If a robot resembles us, he suggested, we are inclined to consider its presence in the same way as we would that of a human being.

But when the machine reveals its robot nature to us, we will feel discomfort. Enter what Mori dubbed “the uncanny valley”. The robot will then be regarded as something of a zombie.

Mori’s theory cannot be systematically verified. But the feelings we experience when we meet an autonomous machine are certainly tinged with both incomprehension and curiosity.

The experiment conducted with Berenson at the Quai Branly, for example, shows that the robot’s presence can elicit paradoxical behaviour in museum goers. It underlines the deep ambiguity that characterises the relationship one can have with a robot, particularly the many communication problems they pose for humans.

If we are wary of such machines, it is mainly because it is not clear to us whether they have intentions. And, if so, what they are and how to establish a basis for the minimal understanding that is essential in any interaction. Thus, it is common to see visitors of the Quai Branly adopting social behaviour with Berenson, such as talking to it, or facing it, to find out how it perceives its environment.

In one way or another, visitors mainly try to establish contact. It appears that there is something strategic in considering the robot, even temporarily, as a person. And these social behaviours are not only observed when humans interact with machines that resembles us: it seems we make anthropomorphic projections whenever humans and robots meet.

Social interactions

An interdisciplinary team has recently been set up to explore the many dimensions revealed during these interactions. In particular, they are looking at the moments when, in our minds, we are ready to endow robots with intentions and intelligence.

This is how the PsyPhINe project was born. Based on interactions between humans and a robotic lamp, this project seeks to better understand people’s tendency to anthropomorphise machines.

Trying to communicate with a robotic lamp, 2016

After they get accustomed to the strangeness of the situation, it is not uncommon to observe that people are socially engaging with the lamp. During a game in which people are invited to play with this robot, they can be seen reacting to its movements and sometimes speaking to it, commenting on what it is doing or on the situation itself.

Mistrust often characterises the first moments of our relations with machines. Beyond their appearance, most people don’t know exactly what robots are made of, what their functions are and what their intentions might be. The robot world seems way too far from ours.

But this feeling quickly disappears. Assuming they have not already run away from the machine, people usually seek to define and maintain a frame for communication. Typically, they rely on existing communication habits, such as those used when talking to pets, for example, or with any living being whose world is to some degree different from theirs.

Ultimately, it seems, we humans are as suspicious of our technologies as we are fascinated by the possibilities they open up.

This article was originally published on The Conversation. Read the original article.

The Hyundai Medical Exoskeleton (H-MEX), composed of a high-tech back/leg brace combination device and forearm crutches, is meant to help paraplegics walk again (Photo: Hyundai)

This article was first published on the IEC e-tech website.

Advances in sensors have increased the ability of assistive robots to perform domestic handling and mobility assistance tasks as well as to analyze the environment and individuals around them so that they can carry out monitoring functions. About 4 700 elderly assistance robots were sold globally in 2015, according to the International Federation of Robotics in Frankfurt, which forecasts that sales will increase to 37 500 units between 2016 and 2019.

Robots assisting with healthcare delivery in the home not only increase their users’ autonomy but also have the potential to relieve the burgeoning demands that elderly populations place on health and care services and informal caregivers.

Meanwhile, increasingly sophisticated “companion robots” are being developed to interact with users, make social connections with them and even provide companionship and emotional support, subject to personal choice.

Several IEC Technical Committees (TCs) and Subcommittees (SCs) cooperate on the development of International Standards for the broad range of electrotechnical systems, equipment and applications used in care robots, with an emphasis on safety and interoperability.

Care robots and other assistive robotic devices form part of the wider category of Active Assisted Living (AAL) technologies. All are designed to enhance the quality of life of users and enable them to lead independent lives through the use of Information and Communication Technology (ICT).

The IEC Systems Committee on AAL (SyC AAL), which started work in 2015, has the role of promoting safety, security, privacy and cross-vendor interoperability in the use of AAL systems and services, and of fostering standardization that enables their usability and accessibility.

Human-robot interaction

Assistive robots with sensory and monitoring functions have been in use in healthcare for over a decade.

Paro, modelled on a baby seal, is a companion robot that responds to touch and can make eye contact (photo: UCI UC Irvine)

In 2004, Japanese inventor Dr Takanori Shibata created a 3 kg robot called Paro. The robot is modelled on a baby seal and makes a similar plaintive crying sound. Paro, one of the most widely studied care robots, responds to touch and can make eye contact by detecting the direction from which voices come. It also has a degree of artificial intelligence, including the ability to “learn” behaviour that elicits a positive and empathetic response in the “user” (for instance, being stroked) rather than one that evokes anger, and then to repeat the behaviour that causes the positive response. Some 5 000 Paro robots have been used in more than 30 countries, in therapeutic settings ranging from dementia care to helping earthquake survivors.

Tasks include helping carers as well as patients

Today’s assistive robots can navigate autonomously and carry out a wide range of tasks such as handling, providing assistance with mobility, lifting and bathing in order to reduce the workload of care providers. Their capabilities may include human-robot interaction skills such as face recognition, speech and gestures. These robots are equipped with multiple digital cameras for reading facial expressions and recording the near environment, and stereo microphones and speakers for monitoring and communicating with users. Some models have databases containing photo galleries, music and brain exercises for the elderly.

IEC TC 62: Electrical equipment in medical practice, and its SCs develop International Standards for the electrical equipment and systems and the software used in healthcare. Their work focuses on safety and performance, including data security and confidentiality. There is a joint working group concerned specifically with the safety of the surgical robots and assistive robots used in healthcare.

IEC TC 100: Audio, video and multimedia systems and equipment, has set up a Technical Area, TA 16: Active Assisted Living (AAL), accessibility and user interfaces, to address AAL‑specific issues related to audio, video and multimedia systems and equipment (see article Improving access to multimedia content for those with disabilities in October 2016 e-tech).

Sensors used in care robots include heart rate and blood pressure monitors; sensors to detect changes in motion, audio and scent which could indicate potentially dangerous scenarios for people living alone; range sensors for fall detection; force/torque sensors; light detection and ranging (Lidar) navigation sensors; and red, green and blue image plus depth (RGB-D) sensors for additional navigation data, enabling the robot to perceive objects and people with which it can interact. IEC SC 47E: Discrete semiconductor devices, prepares International Standards for the design, manufacture and use of sensors.

Other IEC TCs involved in standardization work for assistive robots include IEC TC 110: Electronic display devices, and IEC TC 21: Secondary cells and batteries.

As well as care robots, motorized wheelchair robots and exoskeletons offer autonomous motion assistance for the elderly and disabled. Some “robot walkers” are equipped with GPS and wireless Internet connectivity to summon help in an emergency – for instance, a fall – and to enable users’ families to monitor them remotely.

Companion robots

Companion robots are emerging as a specific category. They are designed to act as interfaces that will make it easier for the elderly to enrich their social lives and connect with their families and friends. Typically, they are powered by small electric motors and equipped with high‑definition cameras, voice and facial recognition capabilities, wireless internet connectivity, video‑calling apps, music libraries, memory games and colour-changing LEDs to indicate a device’s “mood”.

An “advanced social companion robot” undergoing trials in California this year is designed to “convey emotion” through different speech tones, sounds, lights and body language. As well as reminding users to take their medication, this robot proactively suggests activities such as reading, going for a walk, playing games or phoning friends. The device uses machine learning to tailor these suggestions to individual preferences.

The European Union and the Japanese government are jointly funding an international research project that will run from 2017 to 2020 to develop “culturally aware robots” to assist with elderly care. Dr Chris Papadopoulos of the University of Bedfordshire in the UK says that elderly users are more likely to accept robots which can “autonomously reconfigure their interactions to match the culture, customs and etiquette of the person they’re caring for”.

Designers of companion robots face issues related to privacy and care‑giving ethics, as well as many technical challenges, not least that robots are currently unable to detect the subtleties of human interactions and therefore to assess people’s moods and feelings.

Analysts see rapid adoption

Healthcare accounts for a substantial sector of the robotics market, primarily in the form of surgical robots. However, the availability of assistive robots for elderly care offering greater functionality and at cheaper prices will see this nascent market segment grow too, particularly in Japan, South Korea, Western Europe and North America.

The consultancy firm Frost & Sullivan predicted in December 2016 that the global market for care assistance and automation robots is expected to achieve rapid adoption, with a Compound Annual Growth Rate (CAGR) of 36% until 2021.

According to Japanese government data, the country’s “care‑bot” market is expected to increase 25‑fold to USD 3,7 billion by 2035.

In the EU, the market for robots and devices assisting elderly people is estimated to reach EUR 13 billion by 2016, according to the Robohub website.

The high cost of assistive robots (currently thousands of dollars each) and the challenges of building safe and certified robotic devices are among the factors restraining market growth.

Access all areas

Japan, a world leader in “eldercare robotics”, has pioneered the development of robots to lift and carry elderly patients and so alleviate the physical strain on care workers.

In addition to talking, understanding speech and navigating autonomously, the Kompaï care robot can keep track of shopping lists, plays music, and work as a videoconference system for users to talk with their doctors, for example (Photo: Robosoft)

At a February 2017 “hackathon” in Singapore on the social use of robotics for elderly healthcare, the winning team proposed an autonomous system using the Segway Robotics platform, Loomo, for transporting wheelchair‑dependent elderly residents in nursing homes and extended-care facilities. The system uses facial recognition algorithms to identify patients, then connects with their wheelchair using a custom-designed electromagnetic coupling, allowing a robot to move patients to and from activities such as meals and medical appointments. This frees up manpower, enabling caregivers to focus on more critical care and medical tasks.

The EU is funding various projects to design robots that can help with specific mobility‑related tasks. They include an intelligent robotic walker that can help someone sit in a chair or assist them as they move around the house.

In the UK, a project known as CHIRON (Care at Home using Intelligent Robotic Omni‑functional Nodes) is developing a system of modular robotic components located in multiple positions around the home which users can adapt to perform different assistive tasks. A key component is a flexible wall‑ or ceiling‑mounted robotic arm-like structure that can provide varying degrees of assistive support, depending on particular needs. The hardware of the robotic arm forms part of an integrated system designed to be connected to other devices and sensors and capable of responding to voice, visual and touch inputs.

Personal care and other assistive robots are becoming increasingly viable ways to complement healthcare technology trends such as smart monitoring systems and mobile applications, thus easing the reliance on human caregivers. The role of the IEC in developing International Standards for the sensors, motors, cameras and other components used in these devices is vital to promote the safety of new products and stimulate market growth.

As human-robot interactions become commonplace, MEPs stress that EU-wide rules are needed to guarantee a standard level of safety and security.© AP Images/European Union – EP

Robots are the technology of the future. But the current legal system is incapable of handling them. This generic statement is often the premise for considerations about the possibility of awarding rights (and liabilities) to these machines at some, less-than clearly identified, point in time. Discussing the adequacy of existing regulation in accommodating new technologies is certainly necessary, but the ontological approach is incorrect. Instead a functional approach needs to be adopted, identifying:

1. what rules can be applied to robots (as is);
2. what incentives do such rules provide;
3. whether those incentives are desirable.

The recent Resolution of the European Parliament (henceforth Resolution) has great political relevance and strategic importance in the development of a European Robotic Industry. Its considerations and conclusions will be taken into account in the current position paper.

Issues

The first issue when discussing regulation is that of definitions, for one cannot regulate something without firstly defining it. However, the term robot is technical and encompasses a wide range of applications that have very little in common. For this very reason, it is impossible to develop a unitary body of rules applicable to all kinds of robotic applications, rather different rules should apply to different classes of devices.

The major issue when discussing civil law rules on robotics is that of liability (for damages). Automation might, to some extent, challenge some of the existing paradigms; and increasing human-machine cooperation might cause different sets of existing rules to overlap, leading to uncertainty, thence increased litigation and difficulties in insuring new products.

Connected to the above is robot testing. A clear legal framework for robot testing outside the restricted environment of the laboratory is needed to assess the kind of dangers that might emerge with the use and their statistical frequency (also for insurance purposes). Similarly, standardization and the development of adequate, narrow-tailored technical standards for different kinds of robots is a major concern, both to ensure product safety and the adoption of possible alternatives to existing liability rules.

A possible non-issue when discussing rules for robotics is that of the attribution of personhood. This, if intended in an ontological way, is deprived of any reasonable grounding in both technical, philosophical and legal considerations. Instead, if understood in a purely functional way the attribution of legal personhood (like in the case for corporations) might be open for discussion (in some cases). Considering some more specific kinds of applications, in particular biorobotic devices and the issue of human enhancement, its regulation and management becomes of the greatest importance and quite likely the single most relevant bioethical issues of the nearby future, requiring ad-hoc regulation to be adopted.

Finally, privacy regulation, access to data and data use is of pivotal importance, not only for the development of a European Robotics industry but more broadly for a digital market. All the mentioned issues might fall under some – direct or indirect – competences of the EU and would certainly benefit from regulations adopted at a supranational (thence European) level.

Responses

The Resolution addresses all the above mentioned issues with consistent considerations, depicting an adequate framework for a technical – legal – debate about what narrow tailored sets of rules should be adopted at the EU level. Overall, it is of the greatest political and strategical importance for defining a modern legal system, favorable to the immersion of new technologies and the proliferation of new businesses.

More specifically:

Definitions: there is need for a definition of “robot” that needs to be inclusive. What needs to be avoided are nominalistic discussions which would inevitably emerge as soon as a regulation was adopted (should the notion of robot be too narrow). Debates about whether a robot requires to be autonomous or not, controlled or not, embodied or not are irrelevant from a legal point of view. Instead, such characteristics should allow to distinguish sub-classes of robots that might be regulated unitarily. Thence, next to a broader and all-encompassing definition of robot (that should include software and non-embodied AI), narrower definitions should be elaborated, pooling together those applications that show some relevant similarities and that can be regulated unitarily.

Liability: Human-machine cooperation will cause different sets of rules to overlap (namely product liability rules and traditional tort law principles). This will cause high levels of uncertainty and litigation, delaying innovation. With respect to compensation, it is, in many cases, sensible to separate the function of ensuring product safety from that of providing the victim with compensation. This might justify different alternative solutions to be adopted: liability exemptions for users and/or manufacturers; creation of automatic compensations funds (privately or publicly funded); compulsory insurance provisions. More broadly, the inadequacies of existing rules (in particular product liability rules) might suggest to radically replace a fault based rule with a risk-management approach (based on absolute liability rules) holding liable the party who is better placed to minimize the cost and acquire insurance (Resolution nn. 53, 55). A one-stop-shop approach might be sensible, preventing complex litigation to apportion liability among different players involved. Which solution is preferable depends on the class of applications considered, the market for such products and the possibility to address those risks through insurance (Resolution nn. 57-59).

Testing: a uniform set of rules allowing testing outside the laboratories and even in human environments should be adopted, defining clear standards (in particular with respect to safety, insurance and management of the experiment) thus reducing discretionary powers of local authorities (Resolution n. 23).

Standardization & European Robotics Agency: standards represent the most effective way to ensure high levels of product safety and provide certainty ex ante to manufacturers who conform to them (Resolution n. 22). However, the time required for the adoption of a new standard and its breadth is incompatible with the current pace of technological innovation. A European Robotic Agency, such as the one suggested by the Resolution (nn. 15-17), could have a strategic importance setting a supranational standard, that could be of use beyond European borders. Otherwise, other leading economies will attempt to do the same.

Electronic Personhood: set forth by the Resolution, this notion is purely functional and intends to facilitate the registration, insurance and management of some devices (in particular nonembodied AI) with a legal tool that is equivalent to that used for corporations (so called legal personhood), see Resolution n. 59, let. E) and F).

Human Enhancement: the use of robotics to overcome human limits might become problematic given the lack of a clear sets or rules and criteria that could help discern what kind of manipulations of the human body should be allowed. The constitutional principles of human dignity, equality, and freedom of self-determination, as understood today in the broader bioethical debate are per se insufficient, and narrower criteria ought to be adopted. The legal grounds to justify an intervention by the EU in this field are less evident than in all other matters mentioned, however they can be found in the freedom of movement of EU citizens, which would suggest, to some extent, a uniform framework. With respect to the content of such principles, human dignity ought to be understood as objective and external, limiting self-determination, and reversibility of the intervention onto the body should also be taken into consideration.

Privacy & Free Flow of Data: privacy cannot be granted simply through informed consent. Consent is hardly ever truly informed, and the very possibility to dissent is limited, should one want to use the service or device requiring the collection of personal data for its operation. On the one hand, the current EU Regulation setting forth the “-by design” principle, should be narrowed down through the adoption of specific standards, specifying what satisfies that criteria in different classes of applications (see Recommendation nn. 17-21). On the other hand, consumers should be compensated for allowing access and use of – private and anonymized – data through post-sale services, enriching after-sale duties imposed on the producer.

In this episode, Audrow Nash interviews Bradley Knox, founder of bots_alive. Knox speaks about an add-on to a Hexbug, a six-legged robotic toy, that makes the bot behave more like a character. They discuss the novel way Knox uses machine learning to create a sense character. They also discuss the limitation of technology to emulate living creatures, and how the bots_alive robot was built within these limitations.

Brad Knox

Dr. Bradley Knox is the founder of bots_alive. He researched human-robot interaction, interactive machine learning, and artificial intelligence at the MIT Media Lab and at UT Austin. At MIT, he designed and taught Interactive Machine Learning. He has won two best paper awards at major robotics and AI conferences, was awarded best dissertation from UT Austin’s Computer Science Department, and was named to IEEE’s AI’s 10 to Watch in 2013.

Links

• bots_alive Kickstarter page
• Download mp3 (16.8 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Level 3 automation, where the car handles all aspects of driving with the driver on standby, is being tested in Sweden. Image courtesy of Volvo cars

Today’s self-driving car isn’t exactly autonomous – the driver has to be able to take over in a pinch, and therein lies the roadblock researchers are trying to overcome. Automated cars are hurtling towards us at breakneck speed, with all-electric Teslas already running limited autopilot systems on roads worldwide and Google trialling its own autonomous pod cars.

However, before we can reply to emails while being driven to work, we have to have a foolproof way to determine when drivers can safely take control and when it should be left to the car.

‘Even in a limited number of tests, we have found that humans are not always performing as required,’ explained Dr Riender Happee, from Delft University of Technology in the Netherlands, who is coordinating the EU-funded HFAuto project to examine the problem and potential solutions.

‘We are close to concluding that the technology always has to be ready to resolve the situation if the driver doesn’t take back control.’

But in these car-to-human transitions, how can a computer decide whether it should hand back control?

‘Eye tracking can indicate driver state and attention,’ said Dr Happee. ‘We’re still to prove the practical usability, but if the car detects the driver is not in an adequate state, the car can stop in the safety lane instead of giving back control.’

Next level

It’s all a question of the level of automation. According to the scale of US-based standards organisation SAE International, Level 1 automation already exists in the form of automated braking and self-parking.

Level 4 & 5 automation, where you punch in the destination and sit back for a nap, is still on the horizon.

But we’ll soon reach Level 3 automation, where drivers can hand over control in situations like motorway driving and let their attention wander, as long as they can safely intervene when the car asks them to.

HFAuto’s 13 PhD students have been researching this human-machine transition challenge since 2013.

Backed with Marie Skłodowska-Curie action funding, the students have travelled Europe for secondments, to examine carmakers’ latest prototypes, and to carry out simulator and on-road tests of transition takeovers.

Alongside further trials of their transition interface, HFAuto partner Volvo has already started testing 100 highly automated Level 3 cars on Swedish public roads.

Another European research group is approaching the problem with a self-driving system that uses external sensors together with cameras inside the cab to monitor the driver’s attentiveness and actions.

Blink

‘Looking at what’s happening in the scene outside of the cars is nothing without the perspective of what’s happening inside the car,’ explained Dr Oihana Otaegui, head of the Vicomtech-IK4 applied research centre in San Sebastián, Spain.

She coordinates the work as part of the EU-funded VI-DAS project. The idea is to avoid high-risk transitions by monitoring factors like a driver’s gaze, blinking frequency and head pose — and combining this with real-time on-road factors to calculate how much time a driver needs to take the wheel.

Its self-driving system uses external cameras as affordable sensors, collecting data for the underlying artificial intelligence system, which tries to understand road situations like a human would.

VI-DAS is also studying real accidents to discern challenging situations where humans fail and using this to help train the system to detect and avoid such situations.

The group aims to have its first interface prototype working by September, with iterated prototypes appearing at the end of 2018 and 2019.

Dr Otaegui says the system could have potential security sector uses given its focus on creating artificial intelligence perception in any given environment, and hopes it could lead to fully automated driving.

‘It could even go down the path of Levels 4 and 5, depending on how well we can teach our system to react — and it will indeed be improving all the time we are working on this automation.’

The question of transitions is so important because it has an impact on liability – who is responsible in the case of an accident.

It’s clear that Level 2 drivers can be held liable if they cause a fender bender, while carmakers will take the rap once Level 4 is deployed. However, with Level 3 transitions, liability remains a burning question.

HFAuto’s Dr Happee believes the solution lies in specialist insurance options that will emerge.

‘Insurance solutions are expected (to emerge) where a car can be bought with risk insurance covering your own errors, and those which can be blamed on carmakers,’ he said.

Yet it goes further than that. Should a car choose to hit pedestrians in the road, or swerve into the path of an oncoming lorry, killing its occupants?

‘One thing coming out of our discussions is that no one would buy a car which will sacrifice its owner for the lives of others,’ said Dr Happee. ‘So it comes down to making these as safe as possible.’

The five levels of automation:

1. Driver Assistance: the car can either steer or regulate speed on its own.
2. Partial Automation: the vehicle can handle both steering and speed selection on its own in specific controlled situations, such as on a motorway.
3. Conditional Automation: the vehicle can be instructed to handle all aspects of driving, but the driver needs to be on standby to intervene if needed.
4. High Automation: the vehicle can be instructed to handle all aspects of driving, even if the driver is not available to intervene.
5. Level 5 – Full Automation: the vehicle handles all aspects of driving, all the time.

The feedback system enables human operators to correct the robot’s choice in real-time – Jason Dorfman, MIT CSAIL

For robots to do what we want, they need to understand us. Too often, this means having to meet them halfway: teaching them the intricacies of human language, for example, or giving them explicit commands for very specific tasks. But what if we could develop robots that were a more natural extension of us and that could actually do whatever we are thinking?

A team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and Boston University is working on this problem, creating a feedback system that lets people correct robot mistakes instantly with nothing more than their brains.

Using data from an electroencephalography (EEG) monitor that records brain activity, the system can detect if a person notices an error as a robot performs an object-sorting task. The team’s novel machine-learning algorithms enable the system to classify brain waves in the space of 10 to 30 milliseconds.

While the system currently handles relatively simple binary-choice activities, the paper’s senior author says that the work suggests that we could one day control robots in much more intuitive ways.

“Imagine being able to instantaneously tell a robot to do a certain action, without needing to type a command, push a button or even say a word,” says CSAIL director Daniela Rus. “A streamlined approach like that would improve our abilities to supervise factory robots, driverless cars and other technologies we haven’t even invented yet.”

In the current study the team used a humanoid robot named “Baxter” from Rethink Robotics, the company led by former CSAIL director and iRobot co-founder Rodney Brooks.

The paper presenting the work was written by BU PhD candidate Andres F. Salazar-Gomez, CSAIL PhD candidate Joseph DelPreto, and CSAIL research scientist Stephanie Gil under the supervision of Rus and BU professor Frank H. Guenther.  The paper was recently accepted to the IEEE International Conference on Robotics and Automation (ICRA) taking place in Singapore this May.

Past work in EEG-controlled robotics has required training humans to “think” in a prescribed way that computers can recognize. For example, an operator might have to look at one of two bright light displays, each of which corresponds to a different task for the robot to execute.

The downside to this method is that the training process and the act of modulating one’s thoughts can be taxing, particularly for people who supervise tasks in navigation or construction that require intense concentration.

Rus’ team wanted to make the experience more natural. To do that, they focused on brain signals called “error-related potentials” (ErrPs), which are generated whenever our brains notice a mistake. As the robot indicates which choice it plans to make, the system uses ErrPs to determine if the human agrees with the decision.

The team uses EEG brain signals to detect if the person notices a mistake – Jason Dorfman, MIT CSAIL

“As you watch the robot, all you have to do is mentally agree or disagree with what it is doing,” says Rus. “You don’t have to train yourself to think in a certain way –  the machine adapts to you, and not the other way around.”

ErrP signals are extremely faint, which means that the system has to be fine-tuned enough to both classify the signal and incorporate it into the feedback loop for the human operator.

In addition to monitoring the initial ErrPs, the team also sought to detect “secondary errors” that occur when the system doesn’t notice the human’s original correction.

“If the robot’s not sure about its decision, it can trigger a human response to get a more accurate answer,” says Gil. “These signals can dramatically improve accuracy, creating a continuous dialogue between human and robot in communicating their choices.”

The feedback system enables human operators to correct the robot’s choice in real-time – Jason Dorfman, MIT CSAIL2

While the system cannot yet recognize secondary errors in real time, Gil expects the model to be able to improve to upwards of 90 percent accuracy once it can.

In addition, since ErrP signals have been shown to be proportional to how egregious the robot’s mistake is, the team believes that future systems could extend to more complex multiple-choice tasks.

Salazar-Gomez notes that the system could even be useful for people who can’t communicate verbally: a task like spelling could be accomplished via a series of several discrete binary choices, which he likens to an advanced form of the blinking that allowed stroke victim Jean-Dominique Bauby to write his memoir “The Diving Bell and the Butterfly.”

“This work brings us closer to developing effective tools for brain-controlled robots and prostheses,” says Wolfram Burgard, a professor of computer science at the University of Freiburg who was not involved in the research. “Given how difficult it can be to translate human language into a meaningful signal for robots, work in this area could have a truly profound impact on the future of human-robot collaboration.”

Read the paper here.

The project was funded in part by Boeing and the National Science Foundation.

If you enjoyed this article from CSAIL, you might also be interested in:

• MIT CSAIL’s 6-foot-tall NASA humanoid robot has landed
• Researchers add a splash of human intuition to planning algorithms
• Wearable AI that can detect the tone of a conversation
• Ingestible robots, glasses-free 3-D, and computers that explain themselves
• Design, simulate and build a custom drone

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Artificial intelligence (AI) already plays a major role in human economies and societies, and it will play an even bigger role in the coming years. To ponder the future of AI is thus to acknowledge that the future is AI.

This will be partly owing to advances in “deep learning,” which uses multi-layer neural networks that were first theorized in the 1980s. With today’s greater computing power and storage, deep learning is now a practical possibility, and a deep-learning application gained worldwide attention in 2016 by beating the world champion in Go. Commercial enterprises and governments alike hope to adapt the technology to find useful patterns in “Big Data” of all kinds.

In 2011, IBM’s Watson marked another AI watershed, by beating two previous champions in Jeopardy!, a game that combines general knowledge with lateral thinking. And yet another significant development is the emerging “Internet of Things,” which will continue to grow as more gadgets, home appliances, wearable devices, and publicly-sited sensors become connected and begin to broadcast messages around the clock. Big Brother won’t be watching you, but a trillion little brothers might be.

Image: CB Insights

Science fiction has already explored the theme of robot rights, such as the film Bicentennial Man. Image: Columbia Pictures

Science fiction likes to depict robots as autonomous machines, capable of making their own decisions and often expressing their own personalities. Yet we also tend to think of robots as property, and as lacking the kind of rights that we reserve for people.

But if a machine can think, decide and act on its own volition, if it can be harmed or held responsible for its actions, should we stop treating it like property and start treating it more like a person with rights?

What if a robot achieves true self-awareness? Should it have equal rights with us and the same protection under the law, or at least something similar?

These are some of the issues being discussed by the European Parliament’s Committee on Legal Affairs. Last year it released a draft report and motion calling for a set of civil law rules on robotics regulating their manufacture, use, autonomy and impact upon society.

Of the legal solutions proposed, perhaps most interesting was the suggestion of creating a legal status of “electronic persons” for the most sophisticated robots.

Approaching personhood

The report acknowledged that improvements in the autonomous and cognitive abilities of robots makes them more than simple tools, and makes ordinary rules on liability, such as contractual and tort liability, insufficient for handling them.

For example, the current EU directive on liability for harm by robots only covers foreseeable damage caused by manufacturing defects. In these cases, the manufacturer is responsible. However, when robots are able to learn and adapt to their environment in unpredictable ways, it’s harder for a manufacturer to foresee problems that could cause harm.

The report also questions about whether or not sufficiently sophisticated robots should be regarded as natural persons, legal persons (like corporations), animals or objects. Rather than lumping them into an existing category, it proposes that a new category of “electronic person” is more appropriate.

The report does not advocate immediate legislative action, though. Instead it proposes that legislation be updated if robots become more complex; if and when they develop more behavioural sophistication. If this occurs, one recommendation is to reduce the liability of “creators” proportional to the autonomy of the robot, and that a compulsory “no-fault” liability insurance could cover the shortfall.

But why go so far as to create a new category of “electronic persons”? After all, computers still have a long way to go before they match human intelligence, if they ever do.

But it can be agreed that robots – or more precisely the software that controls them – is becoming increasingly complex. Autonomous (or “emergent”) machines are becoming more common. There are ongoing discussions about the legal liability for autonomous vehicles, or whether we might be able to sue robotic surgeons.

These are not complicated problems as long as liability rests with the manufacturers. But what if manufacturers cannot be easily identified, such as if open source software is used by autonomous vehicles? Whom do you sue when there are millions of “creators” all over the world?

Artificial intelligence is also starting to live up to its moniker. Alan Turing, the father of modern computing, proposed a test in which a computer is considered “intelligent” if it fools humans into believing that the computer is human by its responses to questions. Already there are machines that are getting close to passing this test.

There are also other incredible successes, such as the computer that creates soundtracks to videos that are indistinguishable from natural sounds, the robot that can beat CAPTCHA, one that can create handwriting indistinguishable from human handwriting and the AI that recently beat some of the world’s best poker players.

Robots may eventually match human cognitive abilities and they are becoming increasingly human-like, including the ability to “feel” pain.

If this progress continues, it may not be long before self-aware robots are not just a product of fantastic speculation.

The EU report is among the first to formally consider these issues, but other countries are also engaging. Peking University’s Yueh-Hsuan Weng writes that Japan and South Korea expect us to live in a human-robot coexistence by 2030. Japan’s Ministry of Economy, Trade and Industry has created a series of robot guidelines addressing business and safety issues for next generation robots.

Electronic persons

If we did give robots some kind of legal status, what would it be? If they behaved like humans we could treat them like legal subjects rather than legal objects, or at least something in between. Legal subjects have rights and duties, and this gives them legal “personhood”. They do not have to be physical persons; a corporation is not a physical person but is recognised as a legal subject. Legal objects, on the other hand, do not have rights or duties although they may have economic value.

Assigning rights and duties to an inanimate object or software program independent of their creators may seem strange. However, with corporations we already see extensive rights and obligations given to fictitious legal entities.

Perhaps the approach to robots could be similar to that of corporations? The robot (or software program), if sufficiently sophisticated or if satisfying certain requirements, could be given similar rights to a corporation. This would allow it to earn money, pay taxes, own assets and sue or be sued independently of its creators. Its creators could, like directors of corporations, have rights or duties to the robot and to others with whom the robot interacts.

Robots would still have to be partly treated as legal objects since, unlike corporations, they may have physical bodies. The “electronic person” could thus be a combination of both a legal subject and a legal object.

The European Parliament will vote on the resolution this month. Regardless of the result, reconsidering robots and the law is inevitable and will require complex legal, computer science and insurance research.

This article was originally published on The Conversation. Read the original article.

An Air Force airman at the control module of an MQ-9 Reaper. Credit: John Bainter/USAF

Machine ethics offers an alternative solution for artificial intelligence (AI) safety governance. In order to mitigate risks in human-robot interactions, robots will have to comply with humanity’s ethical and legal norms, once they’ve merged into our daily life with highly autonomous capability. In terms of technical challenges, there are still many open questions in machine ethics. For example, what is deontic logic and how can it be used for improving AI safety? How do we fashion the knowledge representation for ethical robots? What about the adaptiveness of the ethical governance? These are all significant questions for us to investigate.

In this interview, we invite Prof. Ronald C. Arkin to share his insights on robot ethics, with a focus on its technical aspects.

Date: February 1st 2017

Interviewee: Prof. Ronald C. Arkin, Regents’ Professor and Associate Dean in the College of Computing at the Georgia Institute of Technology and Executive Committee Member of IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous Systems

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan).

Ron Arkin

Q1 WENG: Thank you for agreeing to an interview with us. Can you tell us a little about your background?

ARKIN: I have been a roboticist for 35 years, so it’s hard to tell [only] a little, but I will do my best. Currently I am a Regents’ Professor and Associate Dean in the College of Computing at the Georgia Institute of Technology. I got my Ph.D. from the University of Massachusetts in 1987, and have been working on robotics since that time. I actually have degrees in chemistry as well—a bachelor’s and a master’s in organic chemistry. So, I have a rather diverse background and followed a non-traditional path compared to most computer scientists.

Q2 WENG: What is your definition of “Ethical Robots”?

ARKIN: That’s an interesting question! I have been asked that one before. I would say an ethical robot is a robotic system that is able to comply with the ethical and social norms of human beings—in some cases in accordance with international law, in other cases in accordance with national or local law. In addition, also being able to comport with humans in a natural manner, obey, and adhere to the same principles as an ethical person.

Q3 WENG: Could you please tell us why you are particularly interested in “Embedded Ethics”? Why is this concept so important?

ARKIN: That’s why you need to read the preface of my book [Governing Lethal Behaviour in Autonomous Robots]. It’s a rather long story. But let me try to give you a short version. Again, it dates back to a meeting—the first international symposium in Roboethics in 2004—where I was exposed to international conventions, e.g., the Geneva convention. It also led me to understand the Vatican’s position on how they viewed appropriate ethical interactions. Perhaps even more important than that one experience was the fact that we roboticists that have been in the field for some time finally started to have success, and we were seeing our technology move out of the laboratory community and into the real world. As such, it wasn’t hard to see how this could go in a variety of different directions.

Some robotics technology could benefit humanity, some not so much. At that point, one has to ask, “who is responsible for this?” So I assumed that, in part, it was myself. I believe that I have to take the responsibility for the technology that I helped to create. The issue is that we, as roboticists, had actually succeeded in many ways in terms of creating technology that we only envisioned when we first started that is going to have potentially profound impact on our society and the world as a whole. As such, we should be proactive in the ways in which we move forward [in order] to get a better understanding of the consequences of the work, and the social implications of the technology we are designing.

https://www.youtube.com/watch?v=jOJwkvihYtc

Q4 WENG: The IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous Systems has published their report “Ethically Aligned Design” in December 2016. As a member of the IEEE Global Initiative, could you please tell us about your role?

ARKIN: I serve a variety of roles. First, I am on the executive committee of that particular initiative, so I work at a high-level with about 30 other people or so. We have committees that large. I also co-chair the committee in Affective Computing. Joanna Bryson is my co-chair on that.

Affective Computing is concerned with the issues surrounding the illusion of emotions we can potentially create in these artifacts, and the ways human beings can bond to them. We are exploring a variety of different things, including robot intimacy, or the notion of applying robotic nudges and moving people in certain directions subtly, and in so doing to change the way people act and behave using these technologies, while trying to understand what may or may not be appropriate guidelines based on ethical considerations.

It’s about 15-20 people in that committee. We are one of the newer committees and we only have a couple of pages in the initial design report. But actually, in the June meeting in Austin, for the next release of the document 2.0, we will have as much as the other committees do. Finally, I am also a member of the humanitarian committee. I think it is called economic and humanitarian. I don’t know the exact title, but I am interested in the humanitarian side, and potential uses of robotics for humanitarian applications.

Q5 WENG: What is “Deontic Logic”? Why does it matter to machine ethics?

ARKIN: It may or may not matter. It’s just one approach to a particular problem. Deontic logic is a logic of prohibitions, obligations and permissions. It is one type of framework that is more transparent in order to get a better understanding of the way in which systems operate in the real world. Then it explains its reasoning to a human being, as compared to say deep learning or other forms of neural networks which may indeed provide good results but provide no explanatory capability to humans after the decision has been made.

The notion of prohibitions basically tells the robot some things it cannot do. The notion of permissions describe the things it is potentially allowed to do. And the notion of obligations says things that the robot must do under specific circumstances. It provides a nice framework of being able to explore morality. Others have already explored this, like Bringsjord at RPI who has applied deontic logic in this framework. We are certainly not the first in that particular space. Much of the work we have done has dealt with first-order predicate logic as a basis. It doesn’t have to be deep because it is so explicit in many cases.

Q6 WENG: Your hybrid embedded ethics framework aims for governing lethal behavior of military robots. Can we apply the same technical framework to service robots? For example, personal care robots might have some non-lethal, but still risky autonomous behaviors that need to be regulated by machine ethics.

ARKIN: I believe so. Indeed the whole point of the architecture is trying to design the systems to make them generalizable and not ad hoc for particular sets of circumstances. We have yet to look at service robotics in a traditional sense, but we have already developed something we refer to as the intervening Ethical Governor, which we are using for health care applications—particularly for assisting the management of two human beings in a particular relationship, like a patient and their care giver.

A robot hands a medication bottle to a person. Photo credit: Keith Bujak. Source: Georgia Tech News Center

We are doing work for the National Science Foundation, joint with Tufts University, looking at how in early stage Parkinson’s there is a condition called the facial masking. The loss of muscle movements in the face of the patient can produce stigmatization, and the patient-care giver relationship deteriorates due to the loss of non-verbal communication. We just had a paper accepted at ICRA describing the architecture that we have and how it has been modified from the military version. We generalized it from the military constraint-based version to a rule-based version.

But also, instead of using the international humanitarian law, we are using occupational therapy guides as a basis for generating the responses when two human beings in the relationship step outside of acceptable bounds. So the acceptable bounds now are not defined by international humanitarian law, but they are defined by occupational therapy—such as when a patient or a care giver start to yell at each other, or one starts to leave the room. The hard part is recognizing when dignity is being abridged, particularly in the patient. So we model the moral emotions of empathy, shame and embarrassment. We tried to ensure that the empathy level in the care giver remains within acceptable bounds and that the patient shame and embarrassment doesn’t rise to unacceptable levels. So the intervening Ethical Governor intervenes when a really inappropriate event occurs. It then provides strong corrective action by the robot to encourage people to get along better. The moral adaptor component which we used in the ethical architecture—initially with the guilt model—has now been extended to other moral emotions like shame, embarrassment, guilt and empathy [in order to] try and provide more subtle non-verbal queues [to help the] robot steer the relationship in the correct way before it get out of hand.

Q7 WENG: Regarding your research on machine ethics, what kind of indicators you use to evaluate the final outputs of your project?

ARKIN: In the first one, we were designing a prototype to prove the concept of the architecture. We never make the claim that this is the best way to do it. It is just one way to do it. Then we demonstrate that particular capability through the use of simulations and other aspects. The system was not intended to be fielded, but hopefully others could take those core ideas and potentially put them to use in the real world if they so choose. For the intervening governor, we’re running human subject tests right now—not with Parkinson’s patients yet, but with the elderly population, 50 or older. We’re trying to get a better understanding of the correct sets of rules, and to correct the interventions in a normative fashion, so we can ultimately move on in the final year of the grant to work with actual Parkinson’s patients. Again, using standard HRI technics, such as recording the kinds of reactions they have and parsing the data from the sessions and using pre- and post- evaluation questionnaires.

Q8 WENG: Let’s talk about your proposed machine ethics framework. You had mentioned that it includes four main parts: (1) Ethical Governor (2) Ethical Behavior Control (3) Ethical Adaptor (4) Responsibility Advisor. First of all, what is the major difference between Ethical Governor and Ethical Behavior Control? Is it the difference between deliberative and reactive control, or something else?

ARKIN: The Ethical Governor is basically a standalone component to the architecture which is a bolt on component at the end of the existing system. In the case of military scenario: asking for permission—should it fire or not. This is an intervening stage which either grants or denies permission through the validation of permission-to-fire variable. So it is straightforward thing to be able to do. The Behavior Ethical Control is a different kind of thing as it works purely with the reactive component. What it tries to do is, for each of the individual behaviors themselves, to respond whether or not to engage a target. This is prior to dealing with the Ethical Governor making the decision; trying to make the decision correctly before it gets to the Ethical Governor, where individual behaviors are given ethical guidelines to try and ensure that the behavior is correct itself. Each of the behaviors is provided with appropriate ethical constraints. So the difference is: one is located within the reactive component in each of the individual behaviors that makes the activity of the architecture, and the Ethical Governor is at the end of the architectural chain.

Q9 WENG: What is the importance of Ethical Adaptor? What kinds of machine learning skill that you chose to improve its adaptiveness?

ARKIN: Machine learning is something we haven’t generally explored in this particular context. Especially the military context, we kind of engineer out the adaptiveness and machine learning for fear that the robot would be granted the authority to start to generalize the targets in the field, which we feel is inappropriate. Learning after action is okay. but that’s a different story. The importance of the Ethical Adaptor, and its cognitive models of guilt that we use, is the fact that even the actions the agent takes may inadvertently end up unethical. This is referred to as a “moral luck”. It could be good luck or bad luck. But if it is bad luck, you want to be able to ensure proactively that the bad action doesn’t occur again. That’s actually the functions of cognitive guilt for people as well to reduce the likelihood of reoccurrence of a behavior that resulted in bad actions. By giving this to a robot we hope that if it makes a mistake it will not repeat it. The probability of that behavior reoccurring would be reduced.

The best example is the case of battle damage assessment for a military scenario. For example, if a robot does its assessment and it looks like a released weapon wouldn’t create too much collateral damage, within acceptable limits, then it will release the weapon. But after it does it, the battle damage action is actually more severe then, it initially appeared to be. Under those circumstances, we want the robot in the field to reduce the probability of releasing the weapon again under similar circumstances. That’s, in essence, the analogue of guilt we use, and eventually, it will get to the point that the robot will not engage targets at all. Again, the software is updated to make sure the use is consistent with Geneva convention.

Q10 WENG: The Laws of War (LOW) and Rules of Engagement (ROE) are written in the form of natural language. How did you transfer these norms into machine readable formats?

ARKIN: It’s important to keep it in mind that we do not try and capture all the Rules of Engagement and all the Laws of War. We always advocate that the system should be used in very narrow circumstances and very narrow situations, such as building clearing operations or counter-sniper operations. It’s impossible to capture all human morality and all the Laws of War at this particular point of time. So, the question is what are the relevant Laws of War and Rules of Engagement for these particular narrow situations that the robot will find itself in.

Let’s take a room clearing operation. We try and do this using these constraint-based approaches. The Laws of War basically say that if you see a civilian, you should not shoot them; if you see a surrendered person, you should not shoot them. For aerial vehicles, if you are outside the kill zone (this is the simulations we did) which is designated by the Rules of Engagement, you cannot fire your weapon—explain simply why you cannot do that. If you release the weapon a toward a legitimate target but it would result in damage of cultural property, like a mosque or church or some similar circumstance, that is illegal as well. You are forbidden from doing those sort of things.

So, the initial work we did was more for aerial circumstances. We also computed the collateral damage using a battle damage assessment strategy to determine whether if it is within acceptable limits for a particular target and to restrain the weapon for proportionality. To be able to do this you have to be able to understand the military necessity of the target, which is assigned by a human being. You have to be able to understand the proportionality of the weapon. In other words: Which weapon is sufficient to take out the target but to do no more damage of the surroundings than is absolutely necessary? You have to ensure that you don’t destroy cultural property as well. Basically, adhere to all necessary principles, such as discrimination which is to be able to tell apart combatants and non-combatants. We also state we should restrict the use of these particular systems for high-intensity combat operations, or inter-state warfare, and not the kind of circumstances you find in modern counter- insurgency operations. They should be used with great care and great caution in very narrow circumstances where the predictability of their performance is far more likely to be good.

Q11 WENG: Why did you choose case-based reasoning (CBR) but not rule-based reasoning (RBR) when you designed Responsibility Advisor?
ARKIN: Historically we used case-based reasoning a lot for a variety of different reasons. Part of the reason is to gain information from previous experts. One of the examples in the past was to work with snipers in the laboratory. We can work with experts. It can be legal experts, it can be medical experts, it can be military experts. Again, the primary reason is to be able to readily use the experience of warfighters to guide us in operations.

Q12 WENG: Finally, your proposal “Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture” was published in 2007. There have been great technological improvements in artificial intelligence in the past decade. Are there any recent updates from your robot ethics research, or anything that you want to share with us?

ARKIN: The project ended around 2010, so there is additional information. As I say, I would encourage those who are really interested to take a look at the book entitled “Governing Lethal Behavior in Autonomous Robots” . That represents the final statement and outcome for that particular project and includes the significant technical details describing that work. Recent advances include the application in other domains, such as the caregiver-patient relationship [discussed above] using the same underlying principles, including both the Moral Adaptor and Ethical Governor in a slightly different. If we were successful, it may also be useful for assisting other human-human relationships, such as a parent and a teenager who may be at odds with one another or even in marriage consulting. We hope to use robots for the benefit of mankind, whether it’s to make sure that warfare is conducted within internationally agreed upon laws, or whether it’s to ensure that people behave appropriately toward each other

If you enjoyed this article, you may also enjoy:

• How do we regulate robo-morality?
• One being for two origins: A new perspective on roboethics
• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• On the ethics of research in robotics, with Raja Chatila
• A European perspective on robot law: Interview with Mady Delvaux-Stehres
• Open Roboethics initiative delivers statement to United Nations CCW

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

TERESA project. Photo credit: Claire Cocano

TERESA is a 3-year research project funded by the European Union and carried out by six institutions from four European countries. Its goal is to develop a new socially intelligent semi-autonomous telepresence system.

A telepresence system consists of an internet-connected mobile robot equipped with a video screen, speakers, and microphones.  A remotely located pilot sitting at a computer can control the robot and use it to interact with people at the robot’s location.

Telepresence systems offer a number of advantages over traditional teleconferencing solutions such as Skype. The robot acts as an avatar for the pilot, allowing the pilot to appear much more present to people around the robot. In addition, because a telepresence robot is mobile, it enables spontaneous interaction, such as mingling at a cocktail party, that would not be possible with Skype.

TERESA project. Photo credit: Claire Cocano

However, existing telepresence systems suffer from a key limitation. They require the pilot to manually execute many behaviours that would happen automatically and effortlessly if the pilot was physically present.  These include navigating through groups of people in a socially appropriate way, maintaining a socially appropriate distance and orientation to others, and maintaining socially appropriate eye contact.

The need to manually execute these low-level behaviours at the same time as selecting high-level behaviours, such as whom to talk to and what to say, can afflict the pilot with cognitive overload, which in turns leads to poor quality social interaction.

In the TERESA project, we have created a new semi-autonomous telepresence system that automates low-level decision making, thereby freeing the pilot to focus on the social interaction itself. TERESA can autonomously navigate through groups of people in a socially intelligent way and approach a specific person selected by the pilot. During conversation, TERESA can also maintain a socially appropriate distance from those around the robot and set the height, orientation, and tilt of its video screen so as to facilitate that conversation.

TERESA is not just intelligent but socially intelligent, thanks to advances in multi-modal perception, machine learning, and robotics developed in the project by an interdisciplinary team of computer scientists, engineers, sociologists, and experts in human-robot interaction.

Working with an elderly day centre in France, we have repeatedly validated TERESA with dozens of elderly subjects. Our results show that semi-autonomous telepresence systems can play a key role in improving the quality of remote interaction. Although TERESA is just a proof of concept, it illustrates the potential of social intelligence to transform telepresence systems and other robots as well.

If you enjoyed this article, you may want to read more about telepresence robotics:

• Telepresence robots help chronically ill kids maintain social, academic ties at school
• Telepresence has come a long way, where do we go from here?
• Personalizing telepresence in service robots
• ORIGIBOT: An affordable telepresence robotic test platform with arm and gripper
• Telepresence, or tele-existence? 4 devices seeking to break down barriers to remote presence

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

bots_alive Gotcha

Modern character AI could be better.

I’m unaware of any NPCs or electronic toy characters that can sustain an illusion of life over more than an hour. They suffer from predictability, simplicity, and inauthentic verbal and nonverbal behavior.

This post is about a new form of character AI. Much like motion capture for scripted animation, this new technique may revolutionize how interactive characters are created, through observation of authentic human-generated behavior.

I’ll explain via a product on which we’re pioneering this technique.

Bots_alive! A robot that plays and finds its way

We launched bots_alive last week on Kickstarter (campaign here). Here’s a 1-minute teaser video with mostly scripted motion.

The product is a smartphone kit that gives simple but lifelike autonomy to a popular RC toy, the Hexbug® Spider.

That video shows the product as we envision it at shipping time. Here you can see how it acts now, with this new character AI technique driving its core functionality already.

Training data to build a character

The character AI for these robots is created in a new way. Here’s how it works.

A human teleoperates the robot through many different scenarios. The teleoperator acts as an improvising puppeteer; he or she follows heuristics yet also empathizes with the robot and authentically responds to what happens, whether it be with curiosity, fear, jubilation, or another emotion.

For bots_alive, the teleoperator watches a screen display of what the system sees and pushes buttons to send commands of forward, back, left, right, forward-right, forward-left, back-right, or back-left. We also consider not pushing a button to be a no-action command.

From these teleoperation sessions, we gather training data containing (a) teleoperation commands and (b) information about the context in which each command was given. A category of machine learning called supervised learning is applied to create a model of the puppeteer, which effectively answers the question: In context X, what is the probability that the teleoperator would have given command Y?

Creating a behavioral model

All of that happens during development. In the end user’s hands, the robot is autonomously controlled by the model. The model of the puppeteer is the character AI.

Let’s assume that the contextual information in the training data is sufficient to justify why the teleoperator chooses one action over another. Then the data contains the spontaneity, uncertainty, and social authenticity* expressed by the human puppeteer. And the autonomous robot retains these qualities if the machine learning is effective.

Why go to the trouble of using machine learning?

We expect this process to create character AI that feels more organic and alive than traditional character AI.

Traditional methods consist of imagining and writing rules for behavior or a finite state machine. It’s much more of an abstract exercise than immersing someone in a situation as the puppeteer.

If you play video games, imagine how much you sometimes empathize with the character you’re controlling. You may even forget the separation between the two of you, experiencing the world as that character. Consider whether your control in these moments is different, both at a high level and in tiny movements, than what the character would do if you had simply written down a set of rules for it to act by.

In that difference lies the promise of this method for developing character AI.

In psychology research methodology, there’s a broad consensus that if you want to know what someone would do in a situation, you don’t ask them what they would do. You put them in the situation and observe. Traditional character AI development is like asking the human developer or designer. Our method is instead analogous to observing a human immersed in the experience of the character. That’s why we expect it to create more authentic characters.

Our method is an application of what’s called learning from demonstration.** For times the teleoperation needs to include human interaction with the character, we keep the teleoperation secret to keep the human interaction partner from changing his or her behavior from what would be done with an autonomous character. We coined this secret version learning from the wizard (or LfW) when it was a research project at the MIT Media Lab. The name combines learning from demonstration with the Wizard of Oz experimental paradigm.

Training as interactive machine learning

In our development, I act as the teleoperator and machine learning programmer. The training process is not simply a bunch of demonstration. Rather, it’s an iterative process of:

• demonstrate
• apply machine learning on the demonstration data set
• observe behavior from the learned model
• create more demonstrations in those contexts the robot isn’t acting satisfactorily
• apply machine learning, and so on.

A general representation of interactive machine learning. From Amershi et al., 2015. Power to the People: The Role of Humans in Interactive Machine Learning. In AI Magazine.

Throughout these iterations, the teleoperator and algorithm designer also reflect on what contextual information still needs to be encoded to improve learning, determine what context cannot be encoded and therefore should be ignored by the teleoperator, and find fun and more delightful behavior than originally puppeteered.

How can we know that it’s better?

We can’t. Not with 100% confidence. But we have already seen that it can make compelling character AI.

We saw it in our study at MIT. In a relatively large randomized experiment, kids interacted either with a human-teleoperated robot or one with autonomous behavior learned from previous teleoperation. Compared to a no-robot condition, the teleoperated robot and the autonomous robot programmed by machine learning elicited similar behavior from their human interaction partners. Curiously, when children were asked whether the robot was human-controlled or autonomous, approximately half in each condition thought it was human-controlled. If you’re familiar with the Turing Test, the popularly known test of the effectiveness of an artificial intelligence, you might recognize that the MIT study constitutes passing a certain narrow and social Turing Test.

Compelling character AI was achieved in another such Turing test, using a technique similar to learning from demonstration, in Unreal Tournament 2004. BotPrize is a yearly academic competition for the bot that best passes as human. The 2012 winner was a bot by Jacob Schrum, Igor Karpov, and Risto Miikkulainen. It combined algorithmic optimization—which can result in markedly unnatural behavior that still achieves the specified goal— of its combat effectiveness with selective replay of recorded human behavior. Twenty-five human judges played alongside the Unreal-playing bots, repeatedly judging whether the other characters were human or artificial during combat. The bot by Schrum and colleagues was judged human more than 51.9% of the time. That percentage was just below the two human characters who were judged most human, at 53.3% and 52.2%. The algorithm is described here.

Lastly, we see compelling character AI now, in our playtesting of bots_alive.

The limitations of this approach are (1) sensing and encoding contextual information and (2) applying machine learning effectively. Both academia and industry are intensely researching how to improve those two limitations. Major strides have been made in the past 10 years, most notably through deep learning, and so it seems likely that further improvement is yet to come.

Our ability to write down behavioral rules that encode authentic behavior with an illusion of life? If there’s a reason why that will improve meaningfully, I don’t see it.

Pepper, Jibo and Milo make up the first generation of social robots, leading what promises to be a cohort with diverse capabilities and future applications. But what are social robots and what should they be able to do? This article gives an overview of theories that can help us understand social robotics better.

I most like the definition which describes social robots as robots for which social interaction plays a key role. So social skills should be needed by the robot to enable them to perform a function. A survey of socially interactive robots defines some key characteristics that of this type of robot. A social robot should show emotions, have capabilities to converse on an advanced level, understand the mental models of their social partners, form social relationships, make use of natural communication cues, show personality and learn social capabilities.

Understanding Social Robots offers another interesting perspective of what a social robot is: Social robot = robot + social interface

In this definition, the robot has its own purpose outside of the social aspect. Examples of this can be care robots, cleaning robots in our homes, service desk robots at an airport or mall information desk or chef robots in a cafeteria. The social interface is simply a kind of familiar protocol which makes it easy for us to communicate effectively with the robot. Social cues can give us insight into the intention of a robot, for example shifting gaze towards a mop gives a clue that the robot is about to change activity, although they might not have eyes in the classical sense.

These indicators of social capability can be as useful as actual social ability and drivers in the robot. As studies show, children are able to project social capabilities onto simple inanimate objects like a calculator. A puppet becomes an animated social partner during play. In the same way, robots only have to have the appearance of sociability to be effective communicators. An Ethical Evaluation of Human–Robot Relationships confirms this idea. We have a need to belong, which causes us to form emotional connections to artificial beings and to search for meaning in these relationships.

How should social robots look?

Masahiro Mori defined the Uncanny Valley theory in 1970, in a paper on the subject. He describes the effects of robot appearance and robot movement on our affinity to the robot. In general, we seem to prefer robots to look more like humans and less like robots. There is a certain point at which robots look both human-like and robot-like, and it becomes confusing for us to categorise them. This is the Uncanny Valley – where robot appearance looks very human but also looks a bit ‘wrong’, which makes us uncomfortable. If robot appearance gets past that point, and looks more human, likeability goes up dramatically.

In Navigating a social world with robot partners: A quantitative cartography of the Uncanny Valley (2) we learn that there is a similar effect between robot appearance and trustworthiness of a robot. Robots that showed more positive emotions were also more likeable. So it seems like more human looking robots would lead to more trust and likeability.

Up to this point, we assume that social robots should be humanoid or robotic. But what other forms can robots take? The robot should at least have a face to give it an identity and make it an individual. Further, with a face, you can indicate attention and imitate the social partner to improve communication. Most non-verbal cues are relayed through the face and creates an expectation of how to engage with the robot.

The appearance of a robot can help set people’s expectations of what they should be capable of, and limit those expectations to some focused functions which can be more easily achieved. For example, a bartender robot can be expected to be able to have a good conversation and serve drinks, take payment, but probably it’s ok if it can only speak one language, as it only has to fit the context it’s in.

In Why Every Robot at CES Looks Alike, we learn that Jibo’s oversized, round head is designed to mimic the proportions of a young animal or human to make it more endearing. It has one eye to prevent it from triggering the Uncanny Valley effect by looking too robotic and human at the same time. Also, appearing too human-like creates the impression that the robot will respond like a human, while they are not yet capable.

Another interesting example is of Robin, a Nao robot being used to teach children with diabetes how to manage their illness (6). The explanation given to the children is that Robin is a toddler. The children use this role to explain any imperfections in Robin’s speech capabilities.

Different levels of social interaction for robots

A survey of socially interactive robots contains some useful concepts in defining levels of social behaviour in robots:

• Socially evocative: Do not show any social capabilities but rely on human tendency to project social capabilities.
• Social interface: Mimic social norms, without actually being driven by them.
• Socially receptive: Understand social input enough to learn by imitation but do not see social contact.
• Sociable: Have social drivers and seek social contact.
• Socially situated: Can function in a social environment and can distinguish between social and non-social entities.
• Socially embedded: Are aware of social norms and patterns.
• Socially intelligent: Show human levels of social understanding and awareness based on models of human cognition.

Clearly, social behaviour is nuanced and complex. But to come back to the previous points, social robots can still make themselves effective without reaching the highest levels of social accomplishment.

Effect of social robots on us

To close, de Graaf poses a thought-provoking question (4):

How will we share our world with these new social technologies and how will a future robot society change who we are, how we act and interact—not only with robots but also with each other?

It seems that we will first and foremost shape robots by our own human social patterns and needs. But we cannot help but be changed as individuals and a society when we finally add a more sophisticated layer of robotic social partners in the future.

If you enjoyed this article, you may also enjoy:

• A Pepper-warm reception at Decos
• Robots at your service: Empowering healthy aging at European Robotics Week, 2016
• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• When robots become art: Interview with Yi-Wei Keng
• Hello Pepper: Getting started to program robots on Android
• Survey: Evaluate ethics of health related privacy with care robots

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The population of the scenic ski-resort Davos, nestled in the Swiss Alps, swelled by nearly +3,000 people between the 17^th and 20^th of January. World leaders, academics, business tycoons, press and interlopers of all varieties were drawn to the 2017 World Economic Forum (WEF) Annual Meeting. The WEF is the foremost creative force for engaging the world’s top leaders in collaborative activities to shape the global, regional and industry agendas for the coming year and beyond. Perhaps unsurprisingly given recent geopolitical events, the theme of this year’s forum was Responsive and Responsible Leadership.

With the onset of the fourth industrial revolution, increasingly discontented segments of society not experiencing congruous economic and social progress are in danger of existential uncertainty and exclusion. Responsive and Responsible Leadership entails inclusive development and equitable growth, both nationally and globally. It also involves working rapidly to close generational divides by exercising shared stewardship of those systems that are critical to our prosperity.

In the end, leaders from all walks of life at the Annual Meeting 2017 must be ready to react credibly and responsibly to societal and global concerns that have been neglected for too long.”

Developing last year’s theme—“The fourth industrial revolution”—this year’s luminaries posited questions, among many others, concerning incipient robotics and artificial intelligence technologies set to have a pronounced impact on the global economy and global consciousness alike. What can we learn from the first wave of AI? How can the humanitarian sector benefit from big data algorithms? How will drone technology change the face of warfare? Can AI and computational tech help foster responsive and responsible leadership? What are the downsides of technology in the fourth industrial revolution?

Enjoy a selection of tech-themed videos below.

And a bit about global science including big data, open source science and education.

Image: Wheat Genetics and Germplasm Improvement

I’ve long believed that Augmented Reality (AR) and robotics are closely related. Both model their environments to some degree. Robotics uses that model to guide the behavior of a machine, whereas AR uses it to provide an enhanced sensory experience to a human.

The exact nature of that enhanced experience is bounded only by available sensory, computational, and display (audio, haptic, …) hardware, and by how the data gathered can be usefully transformed into overlays that augment the natural perception of the human user. What is useful is a function of both the content of those overlays and the latency, how much lag time is introduced by the computations involved in generating the overlays. Faster computational hardware can produce more detailed overlays with the same latency or the same overlays with lower latency than slower hardware.

One important application for AR is making it safer and easier for a human to work in collaboration with robotic hardware. For example, a robot might provide the path it intends to follow and the 3D space through which it intends to pass, and that information might be converted in an AR display into highlighting of anything occupying that space. Or perhaps a machine wants to direct the attention of its human counterpart to some particular element of the environment, say one specific plant. That too could be highlighted in the display.

While these examples only scratch the surface of what is possible, they do serve to illustrate that the content of the AR overlays need not be generated entirely from data gathered by sensors attached to the display itself, but can be provided by other sources, including but not limited to other nearby devices. Those sources might include aerial or satellite imagery and information from databases. In the farming context, they might include 3D soil maps produced from core samples.

Examples of overlays that might be useful for a farmer include thermal imagery, current soil moisture content, soil surface porosity and water absorption capacity, exaggerated vertical relief and what to expect in the way of runoff and resulting erosion for various precipitation scenarios, highlighting all plants of a particular species, all plants exhibiting nutrient deficiencies or other trauma, highlighting bare soil (no mulch or plant cover), the presence, activity, and impact of various types of animals. This list could go on and on.

Machines may be better at doing particular manipulations of data, finding correlations, and even at answering well-specified questions, but they’re not so good at asking meaningful questions, much less at thinking outside the box. For this reason, the combination of human and machine is more powerful than either alone.

It’s still very early days in AR, and there’s a great deal of room for improvement. One development that is likely to occur sooner rather than later is voice operation, enabling hands-free control of the AR experience, including which overlays are active and how they are combined. With voice control, a farmer should be able to walk through a field, say what he wants to see, and make modifications to the plan controlling the robotic machinery that actually operates the farm, or issuing commands for execution by the first available machine. For most, this will be a more intimate and far richer connection to their land than what they currently experience.

If you enjoyed this article, you may also want to read:

• Robohub roundtable: Robotic bee swarms from Black Mirror – what’s hype, what’s real?
• Drought and desertification: How robots might help
• Rising need for nursery, indoor and vertical farming
• Keeping a robotic eye on pollution
• Swarms of precision agriculture robots could help put food on the table
• Top 10 technologies in precision agriculture

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

We’re beginning to see more and more jobs being performed by machines, even creative tasks like writing music or painting can now be carried out by a computer.

But how and when will machines be able to explain themselves? Should we be worrying about an artificial intelligence taking over our world or are there bigger and more imminent challenges that advances in machine learning are presenting here and now?

Join Professor Brian Cox, the Royal Society Professor of Public Engagement, as he brings together experts on AI and machine learning to discuss key issues that will shape our future. Panelists include:

• Professor Jon Crowcroft FRS, Marconi Professor of Networked Systems at the University of Cambridge
• Professor Joanna Bryson, Reader in AI Ethics, University of Bath and Affiliate, Center for Information Technology Policy, Princeton University
• Professor Sabine Hauert, Lecturer in Robotics at the University of Bristol, and President and Co-Founder of Robohub.org
• Professor Murray Shanahan, Professor of Cognitive Robotics at Imperial College London
• Professor Peter Donnelly FRS FMedSci, Director, Wellcome Trust Centre for Human Genetics, and Professor of Statistical Science, University of Oxford

https://www.youtube.com/watch?v=IunNpGGt3H8

You might also enjoy the following articles about AI and AI/robotics policy:

• MIT Media Lab to participate in $27 million initiative on AI ethics and governance
• How artificial intelligence is changing our Christmas shop
• White House report: Artificial intelligence, automation, and the economy
• White House launches public workshops on AI issues
• White House releases reports focusing on opportunities and challenges of AI

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Image: Eunike Nugroho

What might the first western explorer have thought upon encountering whales, flamingos and iguanas? What would have they thought of these strange near-fantastical creatures?

What is it like for 21st-century people to encounter robots for the first time? Are they friend or foe? Where do they come from and what controls them? If we saw one for the first time tomorrow, what would we tell our friends?

This is the thinking behind Robot Life Survey – an alternative history series from design company After the flood that visualises Boston Dynamic’s Big Dog, Prox Dynamics’ Black Hornet Nano, and Harvard University’s Kilobot Swarm as if witnessed by an early explorer. What is our relationship with these strange, futuristic creations?

Max Gadney, After the flood director, comments: “In this series, mechanical intelligence is discovered for the first time by a mystery character and recorded for posterity and science. This explorer records their wonder, alluding to a wider picture of and cast of players. It was a fantastic project to work on at After the flood, combining the passion for art and complex mechanics that underpins much of what we do.”

This series of the Survey is available in three prints – each presenting a record of the explorer’s experiences along with snatches of field notes. A larger book based on Robot Life Survey and its explorer will follow in 2017. This will visualise more robots, notes and thinking around this new world of physical, connected intelligence.

The voice of the explorer is provided by Lloyd Shepherd, author of historical thrillers The English Monster, The Poisoned Island and Savage Magic published by Simon and Schuster. Lloyd has spent many years studying the age of enlightenment, and with a background in the tech industry he is the best person to breathe life into the world of Robot Life Survey. The art itself is by Indonesia-based artist named Eunike Nugroho, who specialises in botanical and animal life illustrations and paintings.

Enjoy the “Explorer Notes” by Lloyd Shepherd below:

Big Dog

Image: Eunike Nugroho

“Of all the creatures I’d encountered on my voyages, this was the one that most disturbed me. It stepped silently through the trees, its queasily familiar legs quick-stepping through the snow, as it sought to self-correct its balance at each contact with the ground. On its back was an enormous hessian sack, tied to its body with rope, the property, no doubt, of some Siberian tribesman who had captured this thing and adapted its programming for his own banal ends. It would have come up to above my waist, had I not been sitting in a tree watching it pass, and it was the biggest single machine I had yet seen. But this is not what disturbed me about it. What gave me pause was the memory of Lovelace’s wolf-hound, a great grey beast of enormous gentleness which had plodded this changing Earth with the exact same care as this silent machine. The dog must have been long-dead by that time, but my heart softened at the memory of it, as I watched this machine make its considered way through those deep, silent, freezing woods.”

Nano UAV

Image: Eunike Nugroho

The noise made by one of these tiny machines must be almost inaudible, but the little swarm we encountered down in the valley made an angry sound in its buzzing aggregate. I had seen plans for machines such as these in Lovelace’s study, and had marvelled at how the elegant lines had been squeezed and folded into such tiny mechanisms. Each was barely five centimetres in length, and they must be light indeed to hover in the air so effortlessly, though it is some time since anyone has caught one. Their yellow-and-silver sides reflected the harsh Afghan sun, and as they moved away from us their buzz mellowed, putting me in mind of English summer gardens, though no bee I ever saw cast a shadow such as these machines did on the orange rock, and no singular purpose guides a bee’s flight in the way it seemed to hold these little things in check, as they flew away from us, in inscrutable formation, for a destination at which I could only guess.

Kilobot Swarm

Image: Eunike Nugroho

There were too many of the little machines to count, but there may have been as many as a thousand. The herd moved like oil gliding down a steel ramp, slowly and with its only apparent purpose being adjacency, a need to hold itself together, to cohere into a shape that was a circle, then a triangle, then a square, a puddle of technology which could not make up its mind. Each of the little machines was five centimetres across, and moved on little feet which tapped on the concrete, making a sound like light rain. All each of them knew of the world was the location of its neighbours, and a simple task held in the code of no more than a dozen of them. So was this a single organism, a slime-mold made of plastic and silicon, its intention rippling across tiny invisible networks? Was each device’s awareness of its neighbours the beginnings of a kind of consciousness? And if Lovelace were standing with me now, watching these little machines ripple across the ground, would those machines detect a presence, and stop, and turn to their Creator? No answer came, and on the mechanistic puddle slid, aware only of itself.

If you enjoyed this article, you might also be interested in:

• When robots become art: Interview with Yi-Wei Keng
• SOCIAL ROBOTICS JAPAN: Robot Town Sagami’s strange, yet surprisingly hopeful, anime promo
• Why watching Westworld’s robots should make us question ourselves
• The new Westworld: Humanizing the un-human, or dehumanizing humankind?
• Ex Machina: When Turing meets Bechdel test
• President Obama discusses artificial intelligence with Media Lab Director Joi Ito

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

A robot hands a medication bottle to a person. Photo credit: Keith Bujak. Source: Georgia Tech News Center

The Ethical, Legal and Social Aspects of Social Robots in Healthcare and Education Workshop (also called ELS Workshop) was held in Yokohama the 14^th Nov 2016 during the JSAI-isAI Conference. The workshop was twinned with another workshop in the New Friends Conference in Barcelona the 2^nd Nov 2016.

The workshop aimed at addressing the concerns that the use of social robots in therapy end education pose in the legal, ethical and societal (ELS) domain in a constructive and proactive manner. The workshop undertook ELS challenges that arise from the introduction of robots in therapy and education. A booklet is available to anyone who wishes to find out more, and can be found here.

The ELS Workshop first dealt with aspects of human dignity and the questions of how social robots might endanger an individual’s dignity, for example, by implying an overly simplistic model of human agency or by discriminating against certain demographics. Secondly, the workshop revolved around the topic of privacy. Third, the topic of liability was addressed.

These three overarching ELS issues – dignity, privacy, and liability – were discussed in an open discussion workshop format. The workshop focussed on defining the benefits, conflicts, and possible solutions to the conflicts that such new technologies pose. Researchers from all disciplines, as well as practitioners, actively engaged in the discussions – in particular, those participants who submitted an abstract for the workshop.

Here follows a summery of the issues raised by, and findings of the workshop.

Challenges and issues of social robots in therapy and education

• Public participation: Users are not taken into account during the design process of the robot and that is why “user-centered” design is, at the current stage, not a true statement.
• Public awareness: Connected with the user-centered idea, citizens opinion is not taken into account during the policy-making process. At the same time, however, it seems that people do not read the government website and are not very engaged with the government. Some participants mentioned that in Korea individual privacy is not perceived to have much value and that the government does not promote public awareness concerning it.
• Public security: It was explained in the workshop that in Korea there are lots of security issues in the country, and a case concerning a data leak was explained. If the proliferation of robots in the marketplace is to increase, it will be crucial to address all the security issues, especially if elderly and disabled people are involved.
• Privacy postmortem: Should informed consent include privacy after the death of the person?
• Legal management: The case of the Henna Hotel was introduced as an example of how robotic technology is already in the market. While the industry has already taken steps towards the introduction of robotic technology to the market, there has been no legal accompaniment, especially in the case of educational and therapeutic robots.
• Standards: Due to the lack of this legal management, it is not very clear what standards apply to educational and therapeutic robots. They simply do not exist in the industry.
• Legal uncertainty: Unless there is a problem, it seems that the Law is not providing any pro-active measure towards how to ensure a correct design/use of robot technology.
• Immigrant discrimination: Japan is investing in robotic technology instead of hiring foreigners due to language difficulties as well as cultural differences. Elderly prefer to be treated by Japanese caregivers or by robot technology rather than by immigrants.
• Employment of robot technology: The immigrant discriminatory scenario is aggravated by the low wages Japanese caregivers receive.
• Dehumanizing practice: In Japan, it is normally the daughter or the daughter-in-law that takes care of the mother/grandmother. Dehumanizing the practice of care could bring more free time to the caregiver (that is in this case a relative) but could bring about the decrease of human-human interaction from the care-receiver side.
• Multiple legislations: There is an urgent need to address the ELS issues if this robotic technology is going to be shipped all over the world. Compliance with each national legislation will bring about many conflicts that challenge the insertion of these robots to the market.
• Multiple robots: Each robot is different and the context where they are inserted too. Therefore, it is difficult to standardize case-by-case scenarios.
• Substitution of humans: Some of the participants wondered why humans need to surrender to the robot taking-over scenario. and who is in charge of deciding this. This brought about the following issue: technology is not people-driven.
• Robot obligations: If academia is talking about the possibility of providing robot rights, this highlights the importance of also deciding obligations.
• Unexpected consequences: Some of the robots that are entering the market have not been tested in real contexts and unexpected consequences can easily arise.
• Agency: connected with the unexpected consequences, it is not very clear whether the agency comes from the context, or if the agency comes from the designers. Who is entitled to decide the agency of their robot?
• New agency: The literature has been comparing robots to animals, corporations and other things… should the robot be considered a new kind of agent?
• Religion: Agency is perceived differently depending on the religion. Some comments on how inanimate things have spirit were highlighted, as well as how Hinduism also welcomes low-agency objects to be important, e.g. a book cannot be touched by the feet. What is not clear is whether this is a problem of religion or the individual.
• Human behavior: An experiment in Japan found resulted in children beating a robot in a mall. This experiment was explained in the workshop in order to avoid the replica of the behavior in the human-robot interaction.
• Education: Not a lot of schools have access to robotic technology, although it seems to be important to educate the population on the correct use of robotics technology.
• New law: do robots challenge the legal system in a different manner than other agents do?

Addressing the Challenges / Finding Solutions

• Legal certainty: it is important to have knowledge about domestic laws. A concept similar to “regulation by design” should be implemented.
• Regulatory prohibition: Depending on the use of robots we should ban the producing of robots to avoid unfortunate scenarios. The majority agreed that military robots should be banned.
• Public participation: the creation of a random jury, as we have in the legal system, could help on the participation of the general public on the decision-making process, as well as on the design of the robot technology. Some information and materials could be given to the citizens and they could provide their feedback.
• Employment of technology: increasing the salary of the workers could make the caregiver profession more attractive and increase efficiency. Working less hours can also increase efficiency although it is not clear how it could be modeled with care professions.
• Unexpected consequences: If we could predict and model an agent’s behavior then we could avoid unexpected consequences. Building robot strategies within the robot itself could also be of help.
• Education: it seems very difficult to know how to teach creativity in Japan, where the system is based on memorization. Helping this could help the acceptance of robotic technology as well as promoting the active participation on the design process of the robots.
• Transparency: The robot should be able to show or explain how it is using personal data, in order to promote transparency and user awareness. The use of black boxes could provide more information on the robot usage. It is not clear, however, what extent the government should have access to it.
• Regulated-by-design: This hybrid regulatory model should be embedded into the system to avoid any violations with domestic and international laws.
• Pro-activity: The system of Tokku in Japan should be spread all over the world in order to have Testing Zones for regulatory purposes.
• Regulatory model: we need a hybrid model that can cope with robotic technologies as well as with other technologies, as the delivery of legal counseling is changing rapidly – from having divorce apps to chatbots that deal with parking tickets.
• Regulatory model: connected to this idea, each project that involves robotic technology could be designed in a way that knowledge can be extracted from it for policy purposes.
• Case-by-case: the regulatory model should be created in a way that can cope with the particular needs of every robot and every context where this is inserted, as not all the robots possess the same degree of intelligence or the same capabilities.
• Robot Agency: the creation of an agency that deals with robots as it occurs in Japan could help having better management of all the ELS issues.
• Open international discussions: there is the need to open the debate to more people and to more countries.
• Replacing humans: robots should be conceived as helpers of the humans not a replacement of them.
• Clarity on the capabilities classification could bring about intelligibility on what is/should be permitted and what should not.
• This clarity relates also to the vision and expectations of the different countries on robotic technology, e.g. international future robotic trends.
• Decision-making of the robot: in the case of an accident, everyone could be assigned different values/importance in order the robot to choose the best target. This could raise discrimination issues at the same time.
• Harmonization: There is the need to harmonize what impact means, what is risk, and what are the legal issues concerning this type of robotic technology. There is also the need to have a common understanding.
• Alternative solutions: if the technology allows, the robot should be privacy friendly using other technologies, e.g. removing the cameras in the case of drones; or using vibrations of the floor to know that the patient has fallen.
• Privacy: a system similar to the noise that cameras in Korea make when they take a picture could be adopted in order to spread awareness among the users. The problem with this solution is that there are available apps that allow users to take a picture without making noise.
• Ethics could be addressed by common sense rules, although common sense varies between countries and individuals.
• The creation of a data bank for scientific innovations could help avoiding the non authorized uses of data.
• An international expert committee could be created to draft group reports

Main Issues (Recurring Topics)

• Regulatory model: There are no clear rules on the use of robotic technology in therapy and education. Law is not pragmatic.
• All the parties want to avoid responsibility of harm occurrence
• Growing need of the importance of ELS issues when the human-robot interaction involves vulnerable parts of the society.
• Solutions: each solution can bring about negative consequences. Therefore, an analysis of all the pros and cons will have to be made in order to mitigate any risk posed by the solutions
• The future of work (involving the promotions of co-working spaces)
• Third uses of data (involving privacy post mortem)
• The need for an international set of rules and basic principles.
• Robots will change the way we conceive this world.

Future Steps

• The legal system needs to
  • be disrupted
  • be pro-active
  • be dynamic
  • be more accessible
• There is the need to run more ELS workshops in different countries with different target participants: industry, laymen, legal scholars, etc.
• A great effort has to be done in order to change how people learn as everything is converging in this world. The term people in this case involves any person from any age.
• There is the need to share the knowledge on robotic technology among legal scholars.
• A clear classification on the capabilities of robots (educational, therapeutic) could be of help.
• There is the need to speak with main organizations, e.g. WTO, about this topic.
• We do need an international legal framework that can cope with all these ELS issues in the use of robotic technology, especially if they work with vulnerable parts of the society.
• There should be placed the idea that robotic technology are a supplement of the human’s work

Sophisticated household robots are only just starting to show up in our lives, but all the building blocks for a veritable “Cambrian explosion” of robotics are there, as Gill Pratt described it when he was running the recent DARPA Robotics Challenge. The service robotics industry is emerging, and we will soon be seeing robots of all shapes and sizes making their first forays into our everyday lives.

This is borne out by the recent explosion in robotics and AI funding, which saw robotics investments increase  exponentially over the last five years. While approximately $1 billion was invested in robotics between 2009 and 2014, roughly the same amount was invested in 2015 alone and 2016 is on track to double the total investment again.

According to the CBInsights October 2016 report into “The State of Enterprise: Robotics”, these new investments are largely clustered in autonomous vehicles and the service robotics industry, with strong growth noted in enterprise focused robotics companies.

“Over 50 enterprise-focused robotics companies have raised $259M across 63 deals this year (as of 10/5/2016). These companies are building robots for industrial automation, manufacturing, warehouse automation, and restaurant services, among other tasks.

At the current run-rate, deals to enterprise robotics startups are projected to cross 80 this year, beating last year’s record of 73.

Figure from CB Insights blog “Robotics Startups Funding”

As few as five years ago, the robotics landscape looked simple. While drone and autonomous vehicle technologies were beginning to emerge and cause some excitement, most of the robotics industry was industrial, and the remainder of the landscape was comparatively arid and boring: a few hospital robots, some robot vacuum cleaners, and a few toys and educational robots.

The emergence of the service robotics industry as a serious economic force has taken many people by surprise. It has also made it more difficult to make sense of the robotics landscape, particularly because the new service robotics businesses impact so many markets and applications, making categorization and statistics difficult to obtain, and trends hard to analyze.

Source: Yahoo Finance.

The impact of the internet on the retail sector is a marked example. The demise of physical retail has been hyped since the first dot com bubble, and perhaps this hype has lulled us into a false sense of security. But the huge decline in value of most major US retailers over the past decade is simply shocking. At the same time, robotics offers a potential solution to the pending economic disaster that is threatening the bricks and mortar retail industry. Early signs indicate that robotics is bringing manufacturing back to the US, enabling small batch, just-in-time production in a range of areas from automotive to electronics to biomedical.

Now it seems service robotics solutions in the retail and consumer spaces are offering new ways for brick and mortar companies to likewise maintain their competitive edge. And helping to shape these solutions are a number of Silicon Valley startups featured in this report: Catalia Health, Cleverpet, Bossa Nova, Fetch Robotics, Savioke, Marble, Dispatch, Dishcraft, Momentum Machines, Eatsa, Fellow Robots and Simbe. Indeed, Silicon Valley is at the epicenter of emerging service robotics industry.

Since 2010 Silicon Valley Robotics has been tracking early stage robotics startups and supporting them as they grow. Many of these companies are creating fundamentally new interactions and products, and as they do so, are leveraging the latest business model concepts, including various twists on “Robots as a Service” and “cloud robotics”. With so much new ground being broken, we saw a need to share their stories so that the industry as a whole can learn and grow, too.

Our first report on “Service Robotics Case Studies in Silicon Valley 2015” looked at enterprise robotics companies like Fetch Robotics, Adept, Fellow Robots, and Savioke, who occupied the space between backroom logistics and front-of-house customer service.

The new “Service Robotics Case Studies in Silicon Valley 2016” takes a deeper dive to explore how robotics companies are driving changes in how we interact — not just with technology — but with our pets, our health care providers, and our retail experience.

1. CleverPet (B2C) >> changing how people interact with their pets and delivering data-driven interactions at the far end of the spectrum of the new behavioral robotics.
2. Catalia Health (B2B2C) >> changing how people interact with their medication and life choices, but also how healthcare providers interact with patients >> intelligence built in by experts; allowing for smarter, data-driven interactions.
3. Simbe (B2B) >> allowing retail managers to interact more efficiently with their stock

A clear theme is that each of these robotics startups can unlock data-driven insights with potentially enormous additional business upside by leveraging the ubiquitous connectivity that is being made possible by cloud computing, and more specifically, cloud robotics. Emerging cloud robotics technology enables far more than the simple ability to continuously update and upgrade a physical product, or even to augment the intelligence of that product; it allows robots to become active mobile big data collectors.

Another important theme is how the RaaS or “Robots as a Service” model is maturing, with a range of transactions and applications (e.g. commission-based vs rental-based etc.) to complement the roll out of cloud robotics. Robots are no longer being marketed as simple products for sale, but as sophisticated tools for gaining insights and creating value and driving customer change.

Our second set of case studies and commentaries looks at how accelerators and investors are supporting the drive towards RaaS and finding business models that map robotics technologies into existing business structures, in areas ranging from health to consumer, hospitality to retail.

1. HAX >> helping startups focus on business models via programs like HAX Boost
2. Lemnos Labs >> seeing opportunities for RaaS, working with startups on developing their business plan
3. Comet Labs >> analyzing how intelligent machines can reimagine retail from logistics to customer experience

Every case study and commentary in this report underscores the fundamental importance (and reliance) of human-robot interaction to the service robotics business model — a trend that is sure define the industry well into the future. With so many ‘humans in the loop’ it will become critical that we design robots and businesses that set a high bar for safety, privacy, and ethics. For this reason, we are pleased to announce that Silicon Valley Robotics has just launched a “Good Robot Design Council” with industry expert advisers in an effort to develop design guidelines for the robotics industry that can help fill the ethical gap between standards and laws.

To that same end, our next service robotics reports will focus on self-driving vehicles, agriculture, and other ‘niche’ verticals, with an emphasis on the increasing importance of human-robot interaction, and a look at which aspects of design make good robots and successful businesses.

If you enjoyed this article, you may also enjoy:

• Why are some social robots selling well, whilst others are stalled or failing?
• Robots may save the global economy
• The rise of China’s medical robots sector
• Robots at your service: Empowering healthy aging at European Robotics Week, 2016
• Robot co-workers in the operating room

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In April 2016, British Standards Institution (BSI) published the world’s first ethical standard for the design, production, sale, and usage of social robots. “BS 8611: 2016 Robots and robotic devices” gives guidelines to identify ethical hazards and reduce associated risks to acceptable levels. BS 8611 could be crucial in determining the nature of emerging human-robot relationships. However, the publication also raises many new questions. Are there any conflicts with ISO 13482 when incorporating BS 8611 into the design of safer robots? Should we consider regulating robot ethics? If so, how?

In this interview, we invite Prof. Joanna Bryson to share her insights on how to regulate future human-robot relationships.

Date: December 9^th 2016.

Interviewee: Prof. Joanna J. Bryson, Reader at Department of Computer Science, University of Bath, and affiliate of the Center for Information Technology and Policy, Princeton University.

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan).

Prof. Joanna Bryson. Image: University of Bath

Q1 WENG: Thank you for agreeing to an interview with us. Can you tell us a little about your background?

BRYSON: My primary academic interest is comparative intelligence.  I was particularly interested in animal behavior as a student, and took a liberal arts degree from Chicago with a major in the Behavioural Sciences.  I now have two degrees in Artificial Intelligence (including my PhD at MIT) and another masters degree in Psychology (both masters are from Edinburgh.)  I’m currently an Associate Professor at Bath where I founded the Intelligent Systems group, and organized Artificial Models of Natural Intelligence, which is basically my own laboratory.

During my PhD I was working on a humanoid robot project and was astonished that people thought they had moral obligations to the robot, even though it did not function at all.  It was just a bunch of motors welded together in the shape of a human.  I began publishing about this problem, thinking that people didn’t know enough about AI, but in recent years I’ve realised they don’t understand enough about ethics.  They are confused about what is really the root of ethical obligation, and they do not understand the nature of artefacts.  They do not understand that we can construct both AI and ethics so that we either are or aren’t obliged to AI.  I have become well-known for arguing that we are ethically obliged to construct AI we are not ethically obliged to.

 Q2 WENG: What is the meaning of “ethical hazards”? Except deception and anthropomorphism, what are other ethical hazards in BS 8611: 2016?

BRYSON: I have to admit I haven’t memorised that document and don’t have time to go back to read it now.  Maybe you are talking about a moral hazard?  A moral hazard is something likely to make you do the wrong thing.  In particular, I have argued that attributing ethical status to AI is a moral hazard, because it allows us to escape the responsibilities we ourselves have as the manufacturers, builders and operators of AI.  This is particularly obvious in the case of military weapons – we have enough trouble with distributing blame between soldiers, their commanders, and the government that commissions them.  Making weapons themselves “sinks” for guilt is not honest or beneficial.  But there are also moral hazards associated with owning “friends” and “lovers” – if children relate mostly with things they own, how will they grow to be able to be equal partners with their spouse or coworkers, or members of a democracy?

Q3 WENG: Some members from Bristol Robotics Laboratory are particularly interested in “embedded intelligence”? Why is this concept important? Does it have any impact on BS 8611: 2016?

BRYSON:  I’m sorry, I don’t think it matters particularly whether AI is embedded in an artefact or “channelled” through the cloud, except obviously there are privacy issues when we connect AI onto the Internet or into any shared digital memory.  In fact, one of the ways I recommend ensuring that we have no ethical obligation to robots or other AI is to ensure there is never a unique copy: that it should be backed up to remote sites at all times.

Q4 WENG: Prof. John Holland said, “..an internal model allows a system to look ahead to the future consequences of current actions, without actually committing itself to those actions”. Can you tell us about the concept of “internal model”? Does a passive dynamic robot have its own internal model as well?

BRYSON:  Ha, that’s a great question!  The idea of an internal model is just a representation that allows, as Prof Holland says, a means for planning around and anticipating the world.  Rodney Brooks famously recommended that AI should not have a model, should use the world as it’s own best model, but in fact that is not practical. There’s not enough information in a sensory signal to disambiguate a world sufficiently to create animal-like intelligence, we need prior expectations to interpret the sensory input.  Still, Brooks was a genius of AI and the field moved forward much faster when we realised we could use minimalist representations; just those necessary for action.  We realised that humans “flush” our memory quickly if sensory information contradicts it, and operate on partial knowledge all the time.  Previously we though AI had to have perfect knowledge so it could make perfect plans, but now we understand that that isn’t tractable.

So does a passive dynamic have an internal model? No.  It is a model of an animal walking system, but to some extent it embodies expectations about for example gravity, but by and large it can do no planning or control. It arrives only where it goes and never knows that it has gotten there or been anywhere else, so certainly cannot anticipate anything.  But its design anticipates particular regularities about the world, so its designers had models that are partly embodied in the walker.

Q5 WENG: I conducted an empirical case study on humanoid robots and regulation with researchers from Waseda University. We found that inside “Tokku” Special Zones there were strong demands for regulating robotics in road traffic, privacy, tele-communication, safety, tax, highway traffic, etc… However, it is not clear that whether robot ethics should be regulated. Do you have any comments on this?

BRYSON: Unfortunately I also haven’t had time to read your study yet, but I can talk a bit about regulation.  There are two important things to recognise.  First, government should not be thought of as an imposition, particularly in a democracy, but even in an autocracy government only functions to the extent it is tolerated.  Government and regulation is the way we citizens coordinate our desires and vision.  Many people in technology seem to think of regulation only as limits on their freedom, and indeed even the word compromise can be used in both a negative and a positive way.  But technologists need to understand that when people come together and work together, they often create something of mutual benefit, public goods that are greater than the summed contribution of those who are cooperating.  Regulations can include those that mandate investment of government resources (our taxes) in innovation.  They can be liberating as well as constraining. Coming to agreements with fellow citizens about essential matters like privacy and the distribution of wealth that AI has huge impact on is essential, and yes, agreements require compromise, but it is through these that we create peace, stability, and affluence.

Having said all that, what I get from surveys of citizens is not information about what regulations would really work.  Governance and regulation are extremely complex and have been studied for centuries; it is as crazy to suggest that ordinary citizens construct regulations as that they construct nuclear weapons or that random people be drafted to represent a country in the World Cup or the Olympics.  We need to respect expertise as well as the desires of ordinary citizens.  What the fears and goals of citizens tell us is how easy or hard it would be to implement particular regulations, how likely they are to be accepted and respected.  Most people on the planet believe that their ancestors still have concerns with their own behaviour, but I wouldn’t want to build policy relying on dead ancestors as actors.  But it does make sense to know people think this if you are governing.

 Q6 WENG: What’s your definition of “ethical robots”? Should we teach them right from wrong?

BRYSON:  I think it makes more sense to program than to teach. We have enough trouble teaching citizens and governments to do good.  Most actions have ethical consequences and there is no problem in building AI to recognise and reason about these, but I would never recommend holding the AI as morally responsible, even though I think AI will necessarily take actions with moral consequences.  As the British EPSRC Principles of Robotics make clear, the responsibility for commercial products lies either in their manufacturer (if they fail to meet an advertised specification) or in their owner / operator (if they are improperly used.)  There are ethical implementations of AI, for example protecting the privacy of the individual, or limiting the damage an individual can do with an autonomous vehicle, but the AI itself should not be designated the responsible party in any circumstance.

 Q7 WENG: Do you believe that Asimov’s Three Laws of Robotics will be guiding principles to ensure the safety and security of human-robot interactions?

BRYSON:  Asimov’s laws certainly impact how people think about AI ethics; the first three (of five) Principles of Robotics reflect (and correct) Asimov’s laws.  But we are working hard to make people understand that these laws are by no means necessary or sufficient for AI ethics.  First of all, Asimov’s laws are intractable. It’s computationally impossible to implement them as described in the books, we can never know all the consequences of our actions.  Second, the books themselves show repeatedly that the laws are insufficient to create truly moral behaviour, that’s the fun of the short stories.  And third, Asimov like most science fiction writers uses AI as a stand-in for humans, for examining what it is to be human. The laws describe the robot as the responsible party, and that’s as I said earlier a mistake.

 Q8 WENG: Do you agree that robots should not be legal persons? Why?

BRYSON:  I don’t know for sure whether people shall make them legal persons, but I am certain that they should not, and in the long run will not be.  A legal person is something that is subject to human justice, and I do not believe it is possible in a well-designed system to create the kind of suffering that is intrinsic to human punishment.  There is no necessity for robots or AI to know shame, pain, regret for the passage of time.  These things are completely intrinsic to being human, we do not consider people with deficits in these concerns to be mentally healthy.  This is because we have evolved to be social organisms and there is no way to extract these properties from our intelligence.  But well-designed AI is modular, anything that is deliberately added by a designer can be removed.  So that’s one reason it would be incoherent for AI to be a legal person.  Another reason is that there is no discrete unit of AI or robots like there is for humans.  We know what it is to be a human being, though we are sometimes a little challenged by people who are brain dead or conjoined twins.  But AI cannot be neatly counted like humans, and therefore the idea of taxing a robot like a person is ludicrous.  And, again, it is a moral hazard.  The companies who decide to employ robotics should be the ones who have legal and economic liability for whatever their robots do.  We should not let them wiggle out of their responsibility because of our love for science fiction or our deep desire to construct perfect, obedient, cyber “children”.

Q9 WENG: What is the definition of “well-designed robots” in your opinion?

BRYSON: A robot is well-designed if the company that produces it is willing to take full responsibility for its actions, and further if governments are willing to license it as safe for human society.  That second requirement is necessary because some corporations have more resources than many countries, so may be willing to accept risks that individual citizens would prefer not to be subject to.

Q10 WENG: Finally, what will be the biggest challenge to developing a human-robot co-existence Society in the following decade?

BRYSON:  We still have many challenges.  I think we will solve dexterity and physical safety before we solve cyber security.  I don’t think we have a full grasp yet about what impact we are already having on human society and human autonomy.  It’s difficult to measure this because so many things are changing at once; we also have the problems of climate change, of massive increases in human numbers and access to communication.  We are currently in a period of grotesque income inequality and political polarisation – some people blame these on AI, but in fact this happened only a century ago, and probably contributed to the two world wars and the great depression. So getting on top of understanding how we are influencing each other and compromising our health with our planets’ is essential to answering this question.

If you enjoyed this article, you may also also enjoy:

• One being for two origins: A new perspective on roboethics
• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• On the ethics of research in robotics, with Raja Chatila
• A European perspective on robot law: Interview with Mady Delvaux-Stehres
• Open Roboethics initiative delivers statement to United Nations CCW

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The European Robotics Week in 2016 took place from 18 to 22 November in several countries including The Netherlands, Austria, Lithuania, Norway, Portugal, Serbia and many more. This event has been occurring since 2011 to spread public awareness of robotics applications and brings together industry, researchers and policymakers. The central event this year was held in Amsterdam. I attended one of the 5 days of activities in the Maritime Museum. The theme was ‘Robots at your service – empowering healthy aging’ which encompassed a variety of activities arranged over the 5 day duration, including debates, open sessions where you could network while interacting with different kinds of robots, workshops for children and a 2 day hackathon. I attended the robot expo and two of the debate sessions which I will summarise below.

Robot Exhibition

Credits: Visual Outcasts

Although it’s clearly still early days for general consumer robotics in terms of the price-value ratio, there are ever more options available for enthusiasts and for very specific applications. The exhibition had a good selection, including these lovely bots:

• Inmoov open source 3d printed robot
• KUKA Robotics human safe industrial robot with fancy torque sensors
• Tiago Pal, ROS-based mobile manipulator
• Paro the therapeutic seal robot
• Pleo the dinosaur pet
• Softbank’s Pepper and Nao robots, Pepper being hosted by Decos, which uses Pepper as a receptionist
• Meccano’s Meccanoid G15 robot
• Robokind’s Milo robot, for teaching children with autism

Panel Discussion: Roboethics

This discussion was about ethics in robotics but it touched a really wide variety of aspects around this, including some philosophy. The speakers were of a very high standard and from a wide variety of backgrounds which gave the discussion its great breadth.

Here are some highlights:

Robots in care: good or bad?

• How should robots in care evolve?

• Robots should be applied to care because the need for care increases as populations in developed countries age, while the labour force interested in care shrinks. But are robots really the answer to this sensitive problem?
• The distinction was made that a wide range of activities qualify as ‘care’, ranging from assisting people to care for themselves, to caring for them, to providing psychological and emotional support in times of depression, distress or loneliness. Are robots suitable for all of these kinds of activities or only a spectrum? Before you judge, consider this example given, of a Nao robot used in interacting with children with diabetes, in the PAL project (Personal Assistant for health Lifestyle). The robot interacts with the children to educate them about their illness and help them track it. The children confide in the robot more easily than adults and hospital attendance in children with diabetes goes up. This is an example of using robots to build a relationship and put people at ease – something that the robot does more easily in this case than a human. When is a robot more trustworthy than a human? When is the human touch really needed?
• Should we want robots to be more human when they are used for care? In some cases we do, when there is a need to soothe and connect, to comfort. But in other cases a more impersonal and less present robot might be desirable. For instance, if you needed help going to the toilet or rising from bed for the rest of your life, would you need a lot of human interaction around that, or prefer it to blend seamlessly into your life to enable you to be as independent as possible?

Robo ethics and liability.

• Responsibility vs liability of robots for their misdeeds

• This question always comes up in a discussion about robot ethics – people feel uncomfortable with the idea of a robot’s accountability for crimes. For instance, if an autonomous car runs over a pedestrian, who would be responsible for that, the car, owner or manufacturer?
• A good point was raised by philosophy professor Vincent Muller to take this argument further. If a child throws a stone through a window, she is responsible for the action but the parent is liable for damages. In the same way, a robot can be responsible for doing something wrong, but another entity, like the owner, might be liable for damages caused.
• When discussing that a robot can be held liable for a crime, we imply it can understand that it’s actions were wrong and did it anyway. But robots as yet have no understanding of what they are doing, thus the conclusion was that a robot cannot be meaningfully convicted of a crime.

Panel Discussion: Our Robotic Future

In this next session, a general discussion on the future of robotics ensued, followed by each speaker giving their wish and hope for this future. There was a strong Euro-centric flavour to this discussion, which gave a fascinating insight into the European search for identity in this time of change – in the robotics and AI revolution, who are we and what do we stand for? How will we respond to the threats and oppourtunities? How can we lead and usher in a good outcome? The panel itself fell on the optimistic side of the debate, looking forward to positive outcomes.

Robotics Reasearch.

• The debate started off underlining the need to share information and research so that we can progress quickly with these high potential technologies.
• A distinction was made between American and European methods of research in AI and robotics

• American research is funded by defence budgets and not shared openly
• In the US, research is done by large corporations like Facebook and Google, and is often funded by DARPA.
• In Europe, research is funded by the European Commission, and often is performed through startups which means less brute strength, but more agility to bring research to fruition. However because these startups are so small, they can be reluctant to share their intellectual property which might be their only business case.

Accepting the robot revolution

• There is a lot of hype about robots and AI in the media which stirs people’s imaginations and fears – how can we usher in all the benefits of the robot revolution?

• Part of the fear is that people don’t like to accept that the last distinction we have of being the smartest on Earth could be lost.
• There is a growing economic divide caused by this technological revolution
• We should not create a system that creates advantage for only a select group, but aim for an inclusive society that allows all to benefit from abundance of robotics.
• People can be less afraid of robots if the value they add is made clear, for example a robot surgeon which is more accurate than a human surgeon can be viewed positively instead of as a threat.
• Technological revolutions are fuelled by an available workforce which can pick up the skills needed to usher them in. For example, in the industrial revolution, in which agricultural workers could be retrained to work in factories, and later a large labour force was available to fuel the IT revolution which, again, would have put many out of work by automating manual tasks. There is a concern that with STEM graduates on the decline, we will lack the skilled resources to build momentum for AI and Robotics.
• Investors in robotics and AI are discouraged by the growing stigma around these technologies.
• Communication policies on this topic are designed around getting people to understand the science, but this thinking must shift. The end users who currently fear and lack understanding must become the center of communication in the future – AI and robotics must enable them and they should have the tools to judge and decide on it’s fate.

Conclusion

This is certainly an exciting time to be alive, as there is so much to still determine and discover in the growth of the AI and Robotics industries and disciplines. Such an event also highlights how far we are from this reality – 10 year horizons were discussed for AI and Robotics to become commodities in our homes. It is very encouraging to see the clear thinking and good intentions that go into making these technologies mainstream. In the coming months I’d like to dig into the topics of ethics, regulation, EU funding, and what the future of AI and robotics could bring.

Click here to visit Thosha Moodley’s blog.

If you enjoyed this article, you may also enjoy:

• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• Survey: Evaluate ethics of health related privacy with care robots
• Care-O-bot 4 celebrates its premiere as a shopping assistant
• Robot companions are coming into our homes — how human should they be?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The concept One Being for Two Origins, from the teachings of Buddha, considers the unification of technology, nature, and human beings. It emerges from the theory of the three aspects: The values of the “Good”, the “Bad”, and “Muki” (Avyakata). The theory of One Being for Two Origins follows the theory of the three aspects and explains a mechanism for how values, such as the good and the bad, emerge in reality using the three aspects. The theory has a contradictory structure, hence it can be difficult for those with a scientific background to fully comprehend it. In this interview, Nagoya University’s Associate Professor Hiroko Kamide shares with us her insights on robot ethics, with a focus on One Being for Two Origins.

Date: December 5^th 2016

Interviewee: Prof. Hiroko Kamide, Designated Associate Professor, Nagoya University, Japan

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan)

Q1 WENG: Thank you for agreeing to an interview with us. Can you tell us a little about yourself and your background?

KAMIDE: My background is social psychology. During my doctoral course, I studied close relationships and well-being. After receiving my PhD, I started researching human-robot interaction and collaborated with engineers to help realize so-called “human-friendly robots”. We established psychological methods to evaluate humanoids, and subsequently developed human-friendly behaviors for robots. I still collaborate with engineers to consider the adaptive way of introducing advanced technology, such as robots and self-driving cars, from an interdisciplinary approach.

 Q2 WENG: Currently you are also in charge of a special interest group on robots and philosophy at The Robotics of Japan (RSJ). Can you tell us something about your RSJ Special Interest Group on Philosophy in Robotics?

KAMIDE: In the RSJ group, we discuss the approach to  technology from an interdisciplinary standpoint. The members are from psychology, philosophy, law, sociology, and engineering. Psychological evaluation, like technological advancement, is important to realize robots that your average person can accept in their daily-life. This collaborative work between psychology and engineering is crucial to developing technologies that can satisfy a human’s needs and expectations. But sometimes we experience terrible events that we cannot control: nuclear accidents, wrongful use of robots, etc. Therefore we started to investigate what the appropriate approach is when implementing and dealing with the cutting-edge technology. It’s a the responsibility of researchers to convey information to ordinary users about how we should deal with technology in order to maintain harmony between technology, nature, and humans.

Q3 WENG: What is the theory One Being for Two Origins? Why is it important to the unification of humanity and nature?

KAMIDE: The theory of One Being for Two Origins has a contradictory structure, and it’s difficult for those with a scientific background to fully understand. At the first step, we begin with a theory of values—the theory of the three aspects—as an introduction to the theory of One Being for Two Origins. The theory of three aspects explains the mechanism of how values, such as good and bad, emerge from the three concepts of the good, the bad, and the “Muki”. Then the theory of One Being for Two Origins follows the theory of three aspects as the fundamental mechanism of order. Through the theory of One Being for Two Origins, we become able to recognize the truth and biases behind language, and as such can avoid conflicts between technology, nature and human-beings.

Q4 WENG: Your paper “One Being for Two Origins – A necessary awakening for the future of robotics” has been presented at the 2016 IEEE ARSO Workshop in Shanghai, China. The co-author is Professor Masahiro Mori, who is a world famous roboticist because of his “uncanny valley” theory. How come you decided to choose this topic? Does the concept One Being for Two Origins have any relationship with “uncanny valley”?

KAMIDE: His theory of uncanny valley has a very thoughtful and groundbreaking perspective, therefore many researchers in the world still refer to his theory. As for my impression, he developed a very clear sense of perspectives that leads us to understand the theory of uncanny valley. His method of observing the truth of the world is in line with the teachings of the Buddha.

Q5 WENG: Regarding regulation for robotics, you pointed out that there is widespread misunderstanding from regulators. What is the misunderstanding? What kind of legal crisis will materialize if they don’t change their way of thinking? How can your proposed theory help solve this legal crisis?

KAMIDE: As the theory of three aspects explains, the value of good and bad are just derived from Muki, that transcends such values and is a true quality of the technology. Useful technology is positive. On the other hand, it would be bad if technology harms people. Technology is fundamentally neither good nor bad. It’s the manner in which technology is applied and operated by humans that is either good or bad. It is humans themselves that attribute these aspects to the technology.

Regulations are attempts to eliminate the bad aspects of technology. However, it is not correct to assume that only the good remains when the bad is eliminated, because technologies themselves are the Muki. And if bad did not exist, good would not exist either, as one is parasitic on the other. To eliminate the bad is to interfere in the working of the Muki in producing these values. The same occurs robots.

Our mindset determines how we approaches the Muki. If we are not mentally prepared, Muki will transform into bad. If we are mentally prepared, we can treat the Muki as good. Bad can transform into good then, of course, good can also transform into bad. And the more good qualities a robot possesses, the stronger its bad qualities will be, following the transformation. Sharp scalpels save lives, but sharp daggers kill people. This is because the working of the Muki is powerful.

In all cases, the mindset of humans is crucial. So, the responsibility of us in robotics is to ensure that technology is not used in bad ways. We must both establish methods of using robots, and work to refine users’ mindsets.

Q6 WENG: Robotics is like a double-edged sword, it can either help or kill people. Robotic lethal weapons present a dilemma. Should they be banned or not? Do you have any comment on this?

KAMIDE: The problem is not in robotics but in ourselves—we who raise such dualistic questions. As I mentioned, the technology of robotics doesn’t have any values. Robotics is not double edged sword at all; our minds are double edged. People easily attribute these values to the physical entity but it is not true. We have to try and observe the fact that we see not what we see, but what we want to see.

Q7 WENG: Do you agree with using dualism as the meta-principle to teach robots right from wrong? If not, do you think it is possible to adopt your theory into creating ethical robots’?

KAMIDE: The “right” and “wrong” is always in the eye of the beholder, and there are no absolute standards. As long as humans accept that they are subject to subjective mechanisms of perceiving the world, it won’t be troublesome to develop ethical robots. But if humans see the “right” and “wrong” as existing independently, objective, it will be definitely problematic.

Q8 WENG: What are major challenges for implementing One Being for Two Origins into robot ethics?

KAMIDE: Clarifying the importance of applying wisdom, not just knowledge. In this theory, knowledge refers to the ability to divide the phenomena into detailed classifications in order to understand the difference between things. On the other hand, wisdom is necessary to discover the meaning in the different things. Knowledge is developed through education, usually, but wisdom is not taught explicitly. Therefore, we are accustomed to dualistic thinking based on knowledge, that divides the world into its constituent parts, but we have difficulty uniting the parts using wisdom, in a way of monism. This is one of reason that we often raise dualistic questions, and eventually develop conflicts.

Q9 WENG: Professor Shigeo Hirose gave an insight called “Morality Engineering” in his short review “A Dialog on Robots” Vol. 7, No. 4, JRSJ. Do you think that One Being for Two Origins might be helpful in implementing his proposal of “Morality Engineering”?

KAMIDE: His article is very interesting. Opinions from both sides who are optimistic or pessimistic to a robotized future society are quite realistic and thoughtful. Recently many philosophers have started to think about morality in robotics. His “Morality Engineering” suggests implementing “rational egoism”, and I agree with the importance of this approach. One Being for Two Origins is the methodology to transcendent and unify opposite aspects of the phenomena to see the truth. So it is helpful to consider how “rational egoism” truly functions in the interactions between humans and robots.

Click here to read Hiroko Kamide’s paper “One being for two origins — A necessary awakening for the future of robotics”.

If you enjoyed this article, you may also also enjoy:

• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• On the ethics of research in robotics, with Raja Chatila
• A European perspective on robot law: Interview with Mady Delvaux-Stehres
• Open Roboethics initiative delivers statement to United Nations CCW

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Hiroshi Ota and Minako Inoue with 2 Robovie R3 robots in Oriza Hirata’s “I, Worker”

How can robotics help to enhance the development of the modern arts? Japan’s famous playwright, stage director Oriza Hirata and leading roboticist Hiroshi Ishiguro launched the “Robot Theater Project” at Osaka University to explore the boundary between human-robot interactions through robot theater. Their work includes renditions of Anton Chekhov’s “Three Sisters”, Franz Kafka’s “The Metamorphosis”, and their own play “I, Worker”. Their work has spread internationally to Paris, New York, Toronto and Taipei.

For this interview, we would like to invite their collaboration partner Yi-Wei Keng, director of Taipei Arts Festival, to share his insights on the intersection of robotics and the arts.

Interviewee: Mr. Yi-Wei Keng, Artistic Director of Taipei Arts Festival (TAF)

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA Initiative and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan)

Interviewer: Ms. Diane Chiang, Guest Editor at ROBOLAW.ASIA Initiative

WENG: Thank you for agreeing to an interview. Can you tell us a little about yourself and your background?

KENG: I have diverse backgrounds. I studied civic engineering in my first year at university and changed my focus to philosophy in my second year. After graduation and military service, I went to Prague to study non-verbal theater. I dedicated myself to disabled theater when I came back from Europe at the beginning of 2000s’. This multi-discipline experience helped me to think of arts in broader sense.

WENG: Can you introduce Taipei Arts Festival to our readers? What are the collaborations between Taipei Arts Festival and Robot Theatre project in recent years?

KENG: Taipei Arts Festival is the city’s festival, and was established in 1998. TAF gets its entire budget from the Taipei city government. I became the artistic director in 2012 and invited “Three Sisters, Android version” by Oriza Hirata and Hiroshi Ishiguro in 2013. Professor Hiroshi Ishiguro also visited TAF and gave public talks about his robotics work. Then in 2015 we invited their “La Metamorphose version Androide” to TAF.

WENG: What caused you to collaborate with Mr. Oriza Hirata? Did you face any challenges in implementing the piece into Taipei Arts Festival?

KENG: Well, it was really an accident. My wife read an article about the Robot Theatre Project in Monthly Art Magazine Bijutsu Techo. She told me about this and I did some research. I thought it was an interesting project, then I found Seinendan through my connections in Japan. Oriza Hirata gave a very positive response about our invitation. Everything went well during the festival, that is why we decided to invite Seinendan again in 2015.

WENG: Taiwan’s choreographer Yi Huang had used his KUKA robot companion to design his “Huang Yi & KUKA” robot choreography. Can you introduce more examples of robot choreography to us?

KENG: For example, French company Cie 111 made the “Sans Objet” in 2009, it was also a dance show with 2 dancers and an industrial robot, like Yi Huang’s KUKA project in 2013. French choreographer Blanca Li made the “Robot” in 2013 where the NAO Robot by Aldebaran Robotics to dance with real dancer.

WENG: In Japan, an art theater in Tokyo performed a theatrical rendition of Osamu Tezuka’s “Black Jack”. The character “Pinoko” was played by a robot created by Waseda University’s Takanishi Laboratory. The arms, legs, and body of the Takanishi Laboratory developed robot are detachable and equipped with 25 motors, so the robot can vividly resemble the Black Jack character. With your experience and expertise in performance art, what are your thoughts on this?

KENG: What surprises people most about robots or puppets is that we know they are objects but they act like they have soul inside. But if there were many robots in our life, then robots on stage would not be such a surprise. What matters is when the robot looks like a human being, then robot theater loses its meaning too because people can’t tell the difference from appearance. There are more and more animation films, but there is a psychological need to see real humans in films. That is what happens in animation film, we don’t want to see the animation, which shoots like the real movie, we want to experience the pleasure of difference and surprise.

CIANG: Is there a correlation between robotic theater and anthropology? If so, what is it? Can you share your insights from a puppetry aesthetics perspective?

KENG: As I say, if a robot looks like the real people on stage, then it loses its attraction for the audience. It is the same for puppet theater: the movement and design should remain dumpy and eccentric. In the future, androids in everyday life may confuse people if we can’t tell the difference between robots and human beings. We might kill someone which we declare as a robot, but in fact is a human being. Why is robot theater important? Because we could use theater to show the fictional situation which may happen in the future and discuss its meaning and possibility through story.

CIANG: In the 2016 Taipei Film Festival, there is a movie featuring a robot called “Sayonara”. This movie is an extension of Hirata and Ishiguro’s Robot Theater Project. They use an android as an actress in the movie. To human actors, what is the difference between 3D robotic theater and a 2D movie when they play with the android?

KENG: When you see the robot in a theater you confront the reality, and the statement from story becomes stronger than in the movie. There is a life event in theater but film is an illusion made by projecting lights on a screen.

CIANG: Can you tell us something about ethics debates in robot theater? Why do ethics matter to robot theater?

KENG: The main issue is the role of actors in theater. In 1908, theorist Gordon Craig declared that the Über-Marionette should replace the real actor in the future. But acting is not only imitation but also a creative process. Ethics about robot theater can help us to understand more about the essence of theater, robots and humanity.

CIANG: Do you think that it is possible to improve the performance of robots in society by borrowing experiences from performance theory?

KENG: If a robot wants to become human, they must look like a human and act like a human. So “act” has something to do with acting. According to the theory of mirror neuron, the audience can react with the acting from the stage, because their body acts the same way when similar situations happen, so the audience reacts. Acting technique divides from inside-out like Stanislavky system and outside-in like the Delsarte method. The performance theory of François Delsatre might help better than the inside-out technique since robots don’t act by feeling.

WENG: Researchers at the University of Oxford found that 47% of U.S. jobs are at risk of AI and automation in the next two decades. Do you agree that in the field of arts and drama, many future job opportunities will be replaced by robots? Or do you rather believe that new technology will create more opportunities for humans?

KENG: There are many puppet theaters in different cultures, and there exist theaters with real actors in these cultures too. There is no contradiction between having a robot actor and real actor. The emergence of robot actors could create a revolution in acting theory. The problem is not technology, but how we treat them.

If you enjoyed this article, you might also be interested in:

• A European perspective on robot law: Interview with Mady Delavaux
• Art meets robotics in Patrick Tresset’s OPLINE Prize entry
• Robots Podcast #171: Robotics in theatre, film and television, with Grant Imahara and Richard McKenna
• Flying robots perform 100th show on Broadway, using new localization technology and algorithms
• RoboThespian gets its own YouTube channel

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Paul, a member of the Care-O-bot 4 robot family, has been greeting customers in Saturn-Markt Ingolstadt since the end of October 2016 and directing them towards their desired products. Image: Saturn.

In January 2015, Fraunhofer IPA presented a prototype of the “Care-O-bot 4” service robot. The charming helper is now proving its worth in the real world. “Paul” the robot has been greeting customers in Saturn-Markt Ingolstadt since the end of October 2016 and directing them towards their desired products.

Care-O-bot 4, alias Paul, approaches Saturn customers and welcomes them to the store. If they ask him about a certain product, he accompanies the customer to the department and points them in the direction of the relevant shelf. As he indulges in small talk about the weather, or another subject, Paul turns out to be a most charming contact partner. However, he prefers to leave actual customer service to his human colleagues. Paul is able to call another member of staff for support via “Voice over IP”. Before Paul bids a fond farewell and returns to the store entrance, he still has time to ask for feedback. In this way, he discovers whether customers appreciated the interaction or not. Martin Wild, Chief Digital Officer at Media-Saturn-Holding: “With Paul, we are offering our customers the opportunity to get to know one of the most advanced robots in the world.”

Image: Fraunhofer IPA, Photographer: Rainer Bez (2015)

Modular construction facilitates a wide range of application areas

The interactive robot prototype was originally developed by Fraunhofer IPA as an assistant robot to actively support humans in areas such as the household, hotels, care homes and hospitals. IPA scientists worked together with the design studio Phoenix Design and Schunk over a period of three years to complete the fourth generation of the robot.  While the predecessors of Care-O-bot 4 were used primarily in the development of technological fundamentals from 1989 onwards, the latest version is a modular product family offering elementary commercial solutions for the first time. Fraunhofer IPA is now working with its spin-off company “Unity Robotics” to ensure the solution is suitable for use in various commercial application areas. Saturn is the first customer to make use of the robot platform in real-life scenarios. The project will initially run for six months, with practical findings from customer interactions and employees directly incorporated into the further development of robot technology.

Care-O-bot 4 providing assistance in the kitchen. Image: Phoenix Design 2015

For the assignment with Saturn, the Stuttgart-based scientists supplemented Care-O-bot 4’s software and hardware with additional features. Ulrich Reiser, IPA project manager, explains: “For example, we specified Paul’s navigation, dialog communication and facial expressions. Semvox and Phoenix Design helped implement these changes.” Laser scanners at shin level help with Paul’s orientation. Paul’s front camera and the “Shore” software developed by Fraunhofer IIS allow him to ascertain the mood of the person he is dealing with and express their emotional state. Microphones and cameras for speech and gesture recognition respectively allow Paul to communicate with those around him.

Care-O-bot 4 open for research and industry

Care-O-bot 4 is not only making a positive impression as a retail assistant. With its modular construction and open software interface, the system can be used in many different application areas. It could, for example, conceivably be used as a mobile information kiosk in shops and airports, for pick-up and delivery services in homes and offices, for security applications or as an attraction in a robotics museum. “We retain an interest in ensuring that as many scientists and companies as possible are able to work with the system in order to expand its potential areas of application”, Reiser explains. Within the framework of the Care-O-bot 4 research platform, experts from around the world can further develop the software and hardware components. Together with the spin-off company Unity Robotics, IPA experts are intending to adapt applications to the needs of companies, making Care-O-bot 4 available as a commercial product.

If you enjoyed this article, you might also be interested in:

• Robots Podcast #221: Ori Systems: Robotic Furniture, with Hasier Larrea
• Designing our home service robot from the ground up
• Robot companions are coming into our homes — how human should they be?
• Global and iRobot floor cleaning market
• Robots that may help you in your silver age

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Photo credit: Robert Shields

Will your next doctor be an app? A cost-cutting NHS wants more patients to act as “self-carers,” with some technologized assistance. A series of flowcharts and phone trees might tell parents whose children have chicken pox how best to care for them—no visits to surgeries required. Or a mole-checking app might tell a worrywart when a given skin discoloration looks harmless, and when to go to a dermatologist, by comparing it to thousands of images in a database.

Cost-cutters in the legal field also promise an algorithmically cheapened future. Tax software simplifies the process of filing by walking the filer through a series of questions. Documents that might have taken human attorneys months to read, can be scanned for keywords in a matter of seconds. Predictive policing promises to deploy force with surgical precision.

All these initiatives have some promise, and may make health care and legal advice more accessible. But they are also prone to errors, biases, and predictable malfunctions. Last year, the US Federal Trade Commission settled lawsuits against firms who claimed their software could aid in the detection of skin cancer, by evaluating photographs of the user’s moles. The FTC argued that there was not sufficient evidence to support such claims. The companies are now prohibited from making any “health or disease claims” about the impact of the apps on the health of users unless they provide “reliable scientific evidence” grounded in clinical tests. If algorithms designed merely to inform patients aren’t ready for prime time, why presume diagnostic robots are imminent?

Legal automation has also faced some serious critiques lately. The University of North Carolina legal scholar Dana Remus has questioned the value and legitimacy of the “predictive coding” now deployed in many discovery proceedings. She and co-author Frank S. Levy (of MIT) raise serious questions about more advanced applications of legal automation as well. The future cannot be completely anticipated in contracts, nor can difficult judgment calls be perfectly encoded into the oft-reductionist formulae of data processing. Errant divorce software may have caused thousands of errors in the UK lately, just as US software systems have disrupted or derailed proper dispositions of benefits applications.

Moreover, several types of opacity impede public understanding of algorithmic ranking and rating processes in even more familiar contexts, like credit scoring or search rankings. Consumers do not understand all the implications of the US credit scoring process, and things are about to get worse as “alternative” or “fringe” data moves into the lending mix for some startups. If the consequences of being late on a bill are not readily apparent to consumers, how can they hope to grasp new scoring systems that draw on their social media postings, location data, and hundreds of other data points? At the level of companies, many firms do not feel that Google, Facebook, and Amazon are playing a fair game in their algorithmic rankings of websites, ads, and products. These concerns, too, are stymied by widespread secrecy of both algorithms and the data fed into them.

In response, legal scholars have focused on remediable legal secrecy (curbing trade secrets and improving monitoring by watchdogs) and complexity (forbidding certain contractual arrangements when they become so complicated that regulators or citizens cannot understand their impact). I have recommended certain forms of transparency for software—for example, permitting experts to inspect code at suspect firms, and communications between managers and technical staff. The recent Volkswagen scandal served as yet another confirmation of the need for regulators to understand code.

But there is a larger lesson in these failures of algorithmic ordering. Rather than trying to replace the professions with robots and software, we should instead ask how professional expertise can better guide the implementation of algorithmic decision-making procedures. Ideally, doctors using software in medical settings should be able to inspect the inputs (data) that go into them, restrict the contexts in which they are used, and demand outputs that avoid disparate impacts. The same goes for attorneys, and other professionals now deploying algorithmic arrangements of information. We will be looking at “The Promise and Limits of Algorithmic Accountability in the Professions” at Yale Law School this Spring, and welcome further interventions to clarify the complementarity between professional and computational expertise.

This post was originally published on the website of Nesta.

HBO’s latest offering (on SkyAtlantic here in the UK) is an update of Michael Crichton‘s 1973 film Westworld; this time brought to us as a ten-part television series by sci-fi re-booter extraordinaire J.J. Abrams and screenwriter Jonathan Nolan (yes, brother and frequent collaborator of Christopher). The new Westworld comes with much hope (for HBO as a potential ‘new Game of Thrones’) and hype, understandably with the talent behind it having given us so much terrific science-fiction of late, including Star Wars: The Force Awakens and Interstellar.

As with Channel 4’s series Humans, broadcast last June (news on forthcoming series 2 here), Dreaming Robot’s offered a live Twitter commentary while the show was being broadcast in the UK, and I’ll take some time afterwards to write some reflective pieces on what we see in the show. (The Twitter ‘Superguide’ from @ShefRobotics can be seen here; my own @DreamingRobots Superguide can be seen here.)

Unsurprisingly, many of the themes in the Westworld reboot could also be seen in Humans. It seems, for example, that both shows express a certain anxiety, that as our machines become more human we humans seem to become less human, or humane. But this isn’t a new idea original to either show – this anxiety as been around as long as robots themselves, from the very invention of the term robot in the 1920s. And if we trace the history of robots in science fiction, we see a history of monsters that reflect this same fear, time and again, in slightly different contexts. Because the robot – which, remember, was invented in the popular imagination long before they were built in labs – is above all else exactly that, a perfect way of expressing this fear.

So, what are we looking at in this new, improved Westworld? (because frankly, the original film lacked the depth of even the first hour of this series, being just a very traditional Frankenstein narrative and rough draft for Crichton’s Jurassic Park, made 20 years later). First – as this nifty graphic on the right illustrates – we do see the robots in Westworld becoming more human. The programme starts with a voice asking the humanoid (very humanoid) robot, Dolores (Evan Rachel Wood), ‘Have you ever questioned the nature of your reality?’ The questioner is exploring whether Dolores has become sentient, that is, aware of her own existence. The echo here with Descartes is clear. We all know about cogito, ergo sum – I think, therefore I am. But Descartes proposition isn’t just about thinking, it is about doubt.  He begins with the proposition that the act of doubting itself means that we cannot doubt, at least, our existence . So we would better understand Descartes’s proposition as dubito, ergo cogito, ergo sum: I doubt, therefore I think, therefore I am. If Dolores is found to be questioning the nature of her reality then that would be evidence for self-awareness and being, according to the Cartesian model.

The robots in Westworld are depicted as falling in love, maintaining strong family bonds, appreciating beauty, and consider the world in philosophical, reflective contexts. How much of this is merely programming and how much exceeds the limits imposed upon them by their human masters is the key question that the show teases its audience with, I suspect, though certainly, it occupies much of our attention in the first few hours. But if these moments – described as ‘reveries’ – are in any way genuine moments of creativity, then this is another category, beyond the notion of the cogito, that we might say the robots are becoming ‘alive’.

For many thinkers in the second-half of the twentieth century (for example, the post-Freudian psychoanalyst D. W. Winnicott), it is only in such moments of creativity, or enjoying genuine moments of spontaneous being, that we truly discover the self and come alive. (The Freudian and post-Freudian influences on the narrative become even more apparent in subsequent episodes.) As response to late industrial capitalism (and the shock of fascism to the human self-conception as a rational animal), this idea emerged of human beings coming ‘alive’ only when we are not acting compliantly, that is, when we are acting spontaneously, or creatively, and not according to the laws or dictates (or ‘programming’) of another individual, organisation or group consciousness. We see this trend not only in a post-Freudian psychotherapy (e.g. Eric Fromm and the Frankfurt School, R. D. Laing) and other philosophical writings but also in the popular post-war subculture media, including advertising – the sort satirised in the ad for Westworld that opens the original film.

Other perspectives give us a glimpse into the robot’s moments of becoming human. Peter Abernathy, looking at a picture that offers a peek at the world outside, says: “I have a question, a question you’re not supposed to ask.” This is an allusion to Adam and Eve and the fruit of forbidden knowledge when humankind became self-aware of the difference between right and wrong. (Peter, after this, is consumed with rage at how he and his daughter have been treated.) And like Walter, the robot that goes on a psychotic killing spree, pouring milk over his victims, Peter is determined to ‘go off script’ and reclaim for himself a degree of agency and self-determination, acting according to his own, new-found consciousness instead of according to what others have programmed for him.

MEANWHILE, the human beings (‘newcomers’) in Westworld seem less ‘humane’ than their robot counterparts. The newcomers are shown to be sadistic, misogynist, psychopathic in the indulgence of their fantasies. One could argue that this behaviour is morally justifiable in the unreal world designed solely for the benefit of paying customers – that a ‘rape’, for example, in Westworld isn’t really ‘rape’ if it is done to a robot (who, by definition, cannot give nor deny consent), but this clearly is not how the audience is being invited to see these actions.

That human beings are becoming more like machines is an anxiety for which there is a long history of evidence, and even pre-dates the cultural invention of robots in the 1920s. We can see this anxiety in the Romantic unease with the consequences of Enlightenment that gave birth to the new, rational man, and of the industrial revolution, that was turning humans into nothing more than cogs in the steam-powered machines that so transformed the economy. This has been addressed in the Gothic tale of Frankenstein, still the basis for so many narratives involving robots, including the original Westworld film and, more recently, even our most contemporary stories such as Ex_Machina, and, to a lesser extent, in this manifestation of Westworld (which will be subject of a future post). (I have written and spoken on this theme myself many times, for example here and here).

So in Westworld, we meet Dr Ford – the mad scientist who creates the machines that will, inevitably, be freed upon the world. Dr Ford immediately reminds us of another Ford, the man whose name is synonymous with the assembly lines and a mode of production in the late industrial revolution that has done so much to dehumanise modern workforces. We see these methods of production and workers in the iconic film, which was contemporary with Henry Ford’s factories, Metropolis. (Though, as we shall see, this Ford is rather more complex…).

This dehumanisation of human beings in the industrial age has been looked at in robot films since Metropolis #Westworld pic.twitter.com/NkoBH7mlD2

— Dreaming Robots Blog (@DreamingRobots) October 4, 2016

This fear reflects, too, that as post-Enlightenment humans become more rational, they become more like machines, acting in predictable, programmed ways, having lost the spontaneity and creativity of an earlier age. The humans of Westworld are exaggerations of the humans of our ‘Western’ world of rationalism, science and alienation. We don’t have to agree with this Romantic notion, that rationalism and science are negative forces in our world to accept that there is a great deal of anxiety about how rationalism and science are transforming individual human beings and our societies.

Rational dehumanisation is something personified in the actions of the corporation, which has replaced the mad-scientist as the frequent villain of the sci-fi Frankenstein-robot-twist (more to come on this in a future post) and we see hints in Episode 1 of what is to follow in Westworld, along the lines of film’s such as 2013’s The Machine, where the slightly misguided and naive actions of a scientist are only made monstrous when appropriated by a thoroughly evil, inhumane military-industrial complex.

This theme to address so succinctly in Ridley Scott’s Blade Runner, a major influence on the new Westworld, where the Tyrell Corporation boasts that their replicants are More Human Than Human. And in Blade Runner, too, we see humanoid robots behaving more humanely that the people that ruthlessly, rationally hunt down the machines. It is unclear, however, from the Tyrell slogan whether the robots are more human than the human because the technology has become so sophisticated, or because humans have fallen so low.

On Westworld as a whole, it is too early to tell if it will maintain its initial promise and be as monumentally successful as Game of Thrones, or as iconic as Blade Runner. But already this first episode has given us more to think about than the original, and undoubtedly the success and failure of the programme will be instructive.

If you liked this post, you may also want to read:

• Robots can successfully imitate human motions in the operating room
• The dark side of ethical robots
• Law-abiding robots? What should the legal status of robots be?
• Beyond Asimov: how to plan for ethical robots

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In this episode, Audrow Nash interviews several researchers presenting their work at the Robotics Science and Systems (RSS) 2016 conference in Ann Arbor, Michigan.

Audrow speaks with Gangyuan Jing from Cornell University about modular robotics; Rico Jonschkowski from Technical University of Berlin about lessons from last year’s Amazon picking challenge, which his team won; Matthew Gombolay from MIT about software to help nurses coordinate; and Dorsa Sadigh from UC Berkeley about developing human-like behavior for autonomous cars.

Links:

• Download mp3 (9.7 MB)
• Link to RSS 2016 homepage
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

In this episode, Audrow Nash interviews Hugh Herr, Director of the Biomechatronics Group at MIT. Herr talks about the accident that led to the amputation of both of his legs below the knee and how this shaped his rock climbing and academic career. Herr also discusses orthoses and exoskeletons developed by his research group, as well as the future of bionic technology.

Transcript below.

Among topics discussed is the lower-body exoskeleton below. This exoskeleton is the first lower-body exoskeleton to decreases the user’s energy expenditure when walking (measured by oxygen consumption). It works by assisting the calf muscle and using the body’s joints, rather than heavy mechanical joints. Hugh Herr says, “Subjects noticed that their legs felt heavier and awkward when they took the exoskeleton off.”

Below is a TED talk Hugh Herr gave in 2014. In this talk, Herr describes disability as a failure of technology, he discusses the design of his own prostheses and his lab’s research, and there is a dance performance by Christian Lightner and Adrianne Haslet-Davis, who lost her lower left leg in the Boston Marathon bombings.

Hugh Herr

His research program seeks to advance technologies that promise to accelerate the merging of body and machine, including device architectures that resemble the body’s musculoskeletal design, actuator technologies that behave like muscle, and control methodologies that exploit principles of biological movement. His methods encompass a diverse set of scientific and technological disciplines, from the science of biomechanics and biological movement control to the design of biomedical devices for the treatment of human physical disability.

His research accomplishments in science and technology have already made a significant impact on physically challenged people. The Transfemoral Quasipassive Knee Prosthesis has been commercialized by Össur Inc., and is now benefiting amputees throughhttp://biomech.media.mit.edu/people-hugh-herr/out the world.  In 2006, he founded the company iWalk Inc. to commercialize the Powered Ankle-Foot Prosthesis and other bionic leg devices.  Professor Herr’s work impacts a number of academic communities. He has given numerous invited and plenary lectures at international conferences and colloquia, including the IVth World Congress of Biomechanics, the International Conference on Advanced Prosthetics, the National Assembly of Physical Medicine and Rehabilitation, World Economic Forum, Google Zeitgeist, Digital Life Design, and the TEDMED Conference. He is Associate Editor for the Journal of NeuroEngineering and Rehabilitation, and has served as a reviewer for the Journal of Experimental Biology, the International Journal of Robotics Research, IEEE Transactions on Biomedical Engineering, and the Proceedings of the Royal Society: Biological Sciences. He has been invited to participate in joint funding proposals from other universities and corporations, and has served on research review panels including the National Institute of Health, the National Institute on Disability and Rehabilitation, and the Department of Veterans Affairs. In 2007, He was presented with the 13th Annual Heinz Award for Technology, the Economy and Employment. His work has been featured by various national and international media, including Scientific American Frontiers, Technology Review, National Geographic, the History Channel, and CNN.

Links:

• Download mp3 (18.3 MB)
• Link to the Biomechatronics Group’s webpage
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

The transcript is edited for clarity.

Hugh Herr:
My name is Hugh Herr and I’m a professor at MIT. I co-direct the Center for Extreme Bionics. I also am the founder of a company called BionX Medical Technologies.

Audrow Nash:
Can you tell me a bit about how you have lost your legs from the knee down?

Hugh Herr:
I was an avid mountain climber. I started mountain climbing when I was seven years old. When I was in my early teenage years I was considered a child prodigy in climbing. When I was seventeen I was out with my climbing partner at the time. Our goal was to climb Mt. Washington in the winter by ascending Huntington’s Ravine, which is about an eight hundred foot, very steep ice wall. We ascended very quickly the eight hundred foot wall, and then decided to continue towards the summit. The weather conditions got worse and worse. Even though we went just five minutes above the head-wall of Huntington Ravine, we were not able to retrace our tracks and we descended the mountain.

By the time we got to tree line it was clear that we had gone off course, but we were stuck at that point. We really didn’t have the rational choice of retracing our tracks and going towards the summit. At that point, wind speeds were so high that one could not even stand. We were forced to go down this ravine system, it’s called the Great Gulf Region. It’s the wilderness side of Mt. Washington. It was there that we spent several days in most extreme bushwhacking that one can imagine. The average depth of snow was to the waist. Sometimes it was to the chest. We were just simply trapped in a white maze.

We dug in when we were not able to move any more out of exhaustion by creating kind of these snow caves. During the day we would walk. We would probably make two miles of progress in a complete kind of marathon effort in the deep snow. We made it within a few miles of the roadway and then couldn’t walk anymore because of severe frostbite. We were later discovered, or found by a person out snowshoeing for the day, and we were plucked from the mountain via helicopter and then treated for severe frostbite and hypothermia.

Audrow Nash:
Then, so from there because of the severe frostbite you had both of your legs amputated from the knee down?

Hugh Herr:
That’s right. After a few months of effort by my medical team to “save” my biological limbs, they were amputated. We were on the mountain late January. My legs were amputated mid-March.

Audrow Nash:
Afterwards, you wanted to return to rock climbing. You were told otherwise, though.

Hugh Herr:
Yeah. I didn’t know what I would be able to do with my new body, and my father said, “If you want to climb, you should climb.” I really had no clear examples of what life would be using prostheses. My limbs are amputated. I went through about a month or two of healing, and then I was fitted with my first pair of artificial limbs. I was shocked at their lack of technological sophistication. They were actually made of plaster of Paris and it was recommended that I not walk without canes or crutches because the limbs might actually break. They were kind of these crazy trainer limbs. The first weekend I went home from the rehabilitation center they didn’t allow me to take my legs because they knew what I was capable of. The next weekend they were stupid enough to allow me to take my legs and I went climbing with my brother, Tony.

Audrow Nash:
How did you begin to adjust your prosthetic limbs for climbing and various purposes?

Hugh Herr:
I quickly abandoned the notion that the limbs needed to look human, and I quickly focused on function. I thought to myself what would be optimal designs for the vertical world of rock and ice climbing. What emerged was a series of different feet for rock and ice surfaces, so feet that could stand on small rock edges the width of a coin, feet that could wedge into rock fissures even where the human foot could not penetrate, feet that could penetrate ice walls. I made my legs exceedingly lightweight to increase my strength to weight ratio. Through design, I was able to etch out a few advantages. There were disadvantages, but I gained enough advantages that it was only twelve months after my limbs were amputated that I was climbing better with artificial limbs than I’d ever achieved with normal biological limbs before the accident.

Audrow Nash:
How did you transition into academia from this, and how did you know that that is the way you should pursue, or continue?

Hugh Herr:
To be honest, the men in my family, my father and brothers and grandfathers, were in the business of construction. Given how uncomfortable artificial limbs were at the time, I couldn’t imagine being on a construction site for the rest of my life. I also couldn’t imagine being on the construction site for the rest of my life. I decided to go to college, which I’d never plan to do. My intent was to be the best climber in the world before the accident. I went to college and I signed up for very basic math courses and computer science courses. I developed an extraordinary passion for the topics and couldn’t stop studying. I just absolutely loved the material. It’s interesting because science and a passion for science and math actually replaced somewhat my passion for climbing mountains.

Audrow Nash:
Then you began to apply what you were learning to your own prostheses with what you were learning? When did you start doing that?

Hugh Herr:
I did have the goal of continuing to design prostheses. The experience of designing new climbing legs and succeeding so wildly beyond everyone’s expectation, including my own, was very inspirational for me. I realized the power of technology to heal, to rehabilitate, even to extend human capability beyond innate capabilities. That was also a clear motivation in my acquisition of knowledge and going back to school. My first patent was obtained, issued very close to the time when I was graduating from undergraduate school in physics. It was on the mechanical interface between the residuum and the artificial limb. The idea was to use fluid bladders in certain configurations. I later proposed to my future MIT advisor that I would build this interface and have actuation pumps and pressure sensing and continually modulate the pressure field around the residuum.

Audrow Nash:
On the spot where the prosthetic limb interfaces with the residual limb is what you’re talking about?

Hugh Herr:
Yes.

Audrow Nash:
So you would use some sort of fluid to make a vacuum so that it would stay very well attached?

Hugh Herr:
Yes, a vacuum offering suspension, but also offering loading support.

Audrow Nash:
Loading support.

Hugh Herr:
A pressure field that would be very comfortable around the end of the limb, enabling a person to walk without discomfort.

Audrow Nash:
Can you tell me a bit about the prostheses that you’re wearing now?

Hugh Herr:
I’m wearing two bionic legs. Both legs have three microprocessors and twelve sensors. The microprocessors are brains, if you will. The devices control a muscle-like actuator that moves and stiffens and powers my synthetic ankles.

Audrow Nash:
What sensors are on board?

Hugh Herr:
The sensors measure position, speed, acceleration, force and temperature, as well as joint position and speed.

Audrow Nash:
What sensing are you doing, contact with the ground?

Hugh Herr:
No. The device has tendon-like series springs, and also a parallel elasticity. The torque sensing is of the torque that the series elastic actuator sees and the parallel spring sees.

Audrow Nash:
I see.

Hugh Herr:
Either one can produce net torque on a joint.

Audrow Nash:
Can you describe how they look a little bit?

Hugh Herr:
They look interesting. There’s two black batteries that sit on top, and there’s black anodized metal shields coming around the ankle. Then inside those shields are all the electronics and then the motor system. Then attached to that is a synthetic foot made of again, black carbon composite.

Audrow Nash:
What kind of grip on the bottom?

Hugh Herr:
I wear the carbon foot, and glued to that carbon foot is a typical rubber that you’d find on the bottom of a shoe.

Audrow Nash:
What kind of battery life?

Hugh Herr:
We get a few thousand fast walking steps. For me and my lifestyle, that’s fine to get through the day. If one wants to walk say, ten thousand steps, one would just take a spare battery or two. The batteries are modular. You just snap them in like a power tool.

Audrow Nash:
I see. How does this compare to other prostheses?

Hugh Herr:
All other foot/ankle prostheses in the world are human powered, meaning the energy of the human attached to them drives the movement. The legs that I’m wearing are bionic. They inject mechanical energy into a person’s stride in a way that’s similar to what the muscles used to do that were lost on amputation.

Audrow Nash:
Can you talk a bit about how they are fit to your residual limbs?

Hugh Herr:
At MIT I have a team of people thinking about fit. What is the nature of comfort? What are the mathematics of comfort? What is the science of comfort? We build mathematical descriptions of the tissues of the ends of the residuum and theories of how the shape of the structure that comes around the residuum and the stiffness of that shape to optimize comfort. Basically, the end of my limb goes into a cup-like device that supports my weight, and so the shape of that cup is critically important to comfort. We mathematically derive that shape, and then we 3D print the structure.

Audrow Nash:
You’re identifying where there are hard and soft spots in the residual limb so that you can match them or match them with their complement, so soft to hard, hard to soft so that it’s more comfortable?

Hugh Herr:
Yeah. Then what’s very complex is the shape, the equilibrium shape of the synthetic skin, if you will. That’s based on tissue, how compliant the tissues are, that shape. Mathematically we understand the compliance of the tissues in all the regions.

Audrow Nash:
What do you mean, compliance of the tissue?

Hugh Herr:
How soft or how stiff they are.

Audrow Nash:
Okay. What about stretch of say, the skin? Does that apply as well?

Hugh Herr:
In a different way. I wear a second skin silicone liner, so the design of that second skin relates to the skin’s strain field, or the amount of stretch in the skin.

Audrow Nash:
I see. Are they attached by this vacuum principle or how would you attach them to the residual limb?

Hugh Herr:
There’s a number of ways to attach. You can have a pin and a lock at the base of the limb. You can have suction. You can have a sleeve that spans from the biological leg across the socket. There’s a number of ways to hold it on.

Audrow Nash:
What have been some challenges in designing this?

Hugh Herr:
The interface. If you ask a thousand persons with limb amputation what’s the number one problem they want solved, probably all thousand would say the mechanical interface, the mechanical attachment. Why? Because it’s uncomfortable. It’s a very, very important problem. It’s a very complex problem because no one understands what comfort is for any device, or there’d be a shoe or a prosthesis. We’re developing that science, and it’s very exciting. We want to be able to produce sockets that are comfortable. We want to produce them fast at low cost.

Audrow Nash:
What are a means of doing this? You 3D print the part. How do you model the residual limb?

Hugh Herr:
We take an MRI image of the limb, and that tells us where the shape of the bones and where the skin is and where the muscles are. We then use robotic palpation tools to compress the tissues and measure displacement speeds and forces, which tell us the fundamental constants of the tissues. We then build a continuum mechanical model. If you have a mathematical description of the biological segment, you can then apply pressure fields. The model will tell you how the tissues deform. Then we define particular tissue deformations that are healthy and comfortable.

Audrow Nash:
Is the goal is to make this cheap and replicable for the general person?

Hugh Herr:
Yeah. My vision of the future is that each human individual will have a digital representation of their body. When they need any type of a bionic device, that digital body will be used to compute an optimal personalized bionic device for the human.

Audrow Nash:
This can refer to clothing as well.

Hugh Herr:
Correct. Clothing, shoes, bras, bike seats, bionic limbs, exoskeletons, neural implants that go inside the body. Everything will be personalized.

Audrow Nash:
Everything is optimized, and this relies heavily on 3D printing or additive manufacturing?

Hugh Herr:
3D printing is a tool. It’s not the dominant science. That is to say, one could build these interfaces with molding processes and not using digital fabrication. Digital fabrication is sometimes nice because it reduces the fabrication frequency, but it’s not necessary. It’s not the secret sauce.

Audrow Nash:
What’s the role of industry and academia in producing these prostheses?

Hugh Herr:
As stated, I’m a professor. The typical model of a professor in the US is they receive some type of grant monies and they conduct research with a team comprising both staff and students. There are inventions. Patents are filed. Patents are owned by the university. The university then licenses the intellectual property to founders of a company. Those founders can be the professors and students that invented, and often that’s the model, the most successful model of translation. In my case that’s exactly what happened. There was patents generated. I’m the inventor. My students are the inventors. A company’s established and then there’s a contractual relationship between the company and the university, licensing the IP within a certain domain of use. That’s what was done with my company, BionX Medical Technologies.

Audrow Nash:
What role does the company play in developing the technology?

Hugh Herr:
At university we do science. We do research that’s publishing. We test hypotheses. We do things that have never been done before, never been tested before. At the company, basically crude prototypes that we build at the university are commercialized. What that means is getting them light enough, most important strong enough, durable enough, manufacturable, at the right price point. Then all the sales and clinical services necessary to distribute that product globally.

Audrow Nash:
What does it look like when someone puts on the BiOM ankle for the first time?

Hugh Herr:
The design in the ankle is biomimetic, inspired by nature, by the human body. The algorithms that are running on these small computers on the ankles control the motor as if the motor were made of muscles and tendon and the whole structure was bone and whatnot. The whole thing moves as if it was made of flesh and bone, even though it’s made of synthetics. Because of that, when it’s fit to a human, the human is used to those dynamics, so there’s little to no training required. Often in minutes the person is saying things like they have their life back, they have their leg back, and they’re either crying out of happiness or laughing out of happiness. It’s fun to go to these fittings, actually.

Audrow Nash:
I’d like to move into talking about exoskeletons. First, would you define what an exoskeleton is and then tell us how it relates to your work with prostheses.

Hugh Herr:
My definition of an exoskeleton is a device that attaches to the body, intimately to the body, that augments physicality. What I mean by augments, it enables a human being to do something that’s beyond natural capability for an innate, healthy body. The word orthoses is a medical term for a robot that attaches to the body that enables a person with a disability or some condition to move more naturally. Exoskeletons are wearable robots that augment.

Audrow Nash:
Can you tell me a bit about metabolic cost as a parameter for designing exoskeletons?

Hugh Herr:
One goal in exoskeletal design is to augment a human by pedal locomotion. An excellent metric of evaluation is the amount of food energy a person requires when using the exoskeleton versus not using the exoskeleton. Why? My view is that if the exoskeleton increases energy levels of the human, the human won’t want to use it. It won’t have sufficient value. It’s a very important metric of evaluation.

Audrow Nash:
Basically they’ll discard it and not use it if the metabolic cost is higher than the task, or higher than it would be to accomplish otherwise?

Hugh Herr:
You know, unless they want to get exercise. If they want to actually be augmented and enhance physicality, no they won’t use it.

Audrow Nash:
How do you measure this?

Hugh Herr:
Typically for aerobic exercise it’s measured by monitoring how much oxygen a person uses when they breathe and how much carbon dioxide is expelled. With those two rates one can compute the calories burned per unit of time.

Audrow Nash:
Would you describe the exoskeleton used to decrease metabolic costs for walking?

Hugh Herr:
The first exoskeleton that was envisioned and published in the world that I’m aware of was in the nineteenth century, over a hundred years ago. It was a Russian inventor called Jagen. His dream was to augment the Russian Army, Russian soldiers. It wasn’t until 2014 that someone succeeded in building an exoskeleton that augments walking and running. My group was the first to accomplish that goal. It’s an exoskeleton that spans the foot and comes up about mid-calf, just below the knee. Fundamentally it’s an artificial calf muscle. The calf muscle is the most important muscle in bi-pedal walking. It supplies about eighty percent of the power to walk. It’s where humans are most inefficient, so the exoskeleton adds an artificial calf muscle and injects energy into the gait like a calf muscle to reduce the metabolic cost of the biological calf.

Audrow Nash:
What does it look like and what kind of actuators are you using?

Hugh Herr:
The actuators are electric based. It looks very interesting. It’s like, this crazy structure, this crazy shoe that goes higher than is normal and has these springy struts that go up on the side of the leg, and then a motor that sits on kind of a shin guard that pulls on that strut and powers movement.

Audrow Nash:
Yes, and so it’s using some sort of tether and you’re pulling that, and that’s how you flex the calf to augment walking?

Hugh Herr:
Flex the ankle, yes.

Audrow Nash:
You’re using the body as its own joint for this. Correct?

Hugh Herr:
It’s very important that exoskeletons are very, very, very light weight. Whenever you add weight to the legs, metabolic cost increases. In an effort to reduce the weight or the mass of the device, we actually use the body’s joints as the bearing instead of putting a synthetic bearing in the device.

Audrow Nash:
What kind of sensing is done with the device?

Hugh Herr:
All the electronics are up high by design so you can walk through a deep, deep puddle of water and not affect the system. We measure positions and speeds and accelerations, essentially. Then from those data we can compute torques, forces, given a model of the structure.

Audrow Nash:
You can infer where the person is in their gait and then determine if you should pull to assist the gap?

Hugh Herr:
Essentially.

Audrow Nash:
Why did you choose to revolve around the ankle?

Hugh Herr:
Again because the calf muscle is the most important muscle in walking.

Audrow Nash:
Are there other advantages to it as well, being that there’s not so much squishy …

Hugh Herr:
Yeah. The shin bone is where the leg is most stiff, most rigid, and it’s a tremendous plate of bone to apply forces on from an exoskeleton.

Audrow Nash:
Can you only use this for walking? Could someone use it for running or other activities?

Hugh Herr:
The device is very versatile. It’s powered, it has smart computation, so yes you can walk, run up and down hills, whatever you want to do. It’s as versatile as the human leg itself.

Audrow Nash:
What kind of efficiency gains are we getting?

Hugh Herr:
There’s a number of peer-reviewed manuscripts. Again, we were the first in 2014 to augment the human in a peer review publication. That first publication, the human subjects were wearing a twenty-three kilogram backpack and we augmented by eight percent. We later conducted a separate study on unloaded humans, just regular walking, and augmented by an average ten percent. The variation is huge, so you have N subjects. Some subjects are as high as almost thirty percent, some are low, and everything in between.

Audrow Nash:
What do you believe creates that variation?

Hugh Herr:
It’s a really interesting question. No one really knows, but everyone in the field has observed this phenomenon, that when you put a single exoskeletal design on N people, you get huge variation. My belief is you take ten people and you could distribute them in terms of athletic capability. Some people have a very smart spinal cord and they can exploit any tool or device very, very quickly. Others are clueless. That’s a possibility where training may be effective to mitigate that variance.

Audrow Nash:
Now, people that use it, what do they say it feels like?

Hugh Herr:
The human body is extraordinary, so a person begins to use it, they very quickly … Initially they feel the energy that it provides, but very quickly the human body gets used to it. Then you take it off and you kind of stumble, and your own biological normal legs feel heavy, awkward and slow. This tells us that wearing bionic structures will just be commonplace in the future, and innate, normal bodies will just be boring.

Audrow Nash:
Now, what kind of upper limit do you think we can get on efficiency with this device?

Hugh Herr:
With the foot/ankle device it’s not known. I would say at least twenty-five percent, but it may be even higher, which is very, very exciting.

Audrow Nash:
Then will you benefit, you think, from augmenting other muscles or other movements?

Hugh Herr:
We don’t know. What’s intriguing is our recent paper on the device. What’s intriguing is when you add exoskeletal torque and power at the ankle, not only do you reduce the power requirements of the biological ankle, but also you reduce the power requirements of the knee and hip. Even though the exoskeleton doesn’t span the knee and hip we nonetheless, reduce muscular effort at those joints tremendously. It begs the question do you really need an exoskeleton that spans the whole leg, or is the foot/ankle sufficient?

Audrow Nash:
Why would you think that it decreases the expenditure at those muscles?

Hugh Herr:
It’s just the dynamics of the system. The body, when you give it power, it uses that power distributively across the entire leg in an optimal way. It’s a new gait. It’s a different gait … than normal.

Audrow Nash:
How does it look compared to normal?

Hugh Herr:
It looks normal to an untrained eye. It doesn’t look like a bizarre walk. It is a distinctly new gait. You give the human energy from a bionic structure and the body quickly figures out how to ultimately use that energy to minimize its own metabolic food energy. That’s what humans are really good at. We’re inherently lazy and we minimize energy really, really well. We’re terrible at detecting high stress levels and probabilities of injury. We injure our knee and we’re surprised. We injure our hip and we’re surprised. An incredible value of exoskeletons is to give the human something that they’re bad at. One class of exoskeletons in the future will be exoskeletons that enable us to do athletic performances without a threat of injury.

Audrow Nash:
How would those work?

Hugh Herr:
I don’t know, but they would have to detect high levels of stress and fatigue in biological structures and then tell the human to stop or to move in a different way. Imagine a world where our top athletes never injure. Imagine the impact to human athletic performance. Performance would just go through the roof, because often what mitigates top performance is injury and recovering from injury and not training harder because you might injure, yada, yada, yada.

Audrow Nash:
What other future applications of exoskeletons do you see?

Hugh Herr:
Another form of augmentation is maintaining one’s inherent physicality independent of age. Imagine a world when you run your best marathon time when you’re sixty, and you’ve been running marathons since the age of fifteen. Why? Because you’re a more skilled athlete. As you age, technology eliminates the impact of age related degeneration by neural implants, maybe by regenerative medicine, maybe by exoskeletons or various forms of augmentation. That’s a very palpable form of augmentation. Most people ethically would be fine with that and most people would be excited about maintaining quality of life as they age.

Audrow Nash:
What are some bottlenecks in exoskeletons? What technological bottlenecks are there?

Hugh Herr:
I’m not sure there are technological bottlenecks. I think that largely the components are sufficient. It’s really a problem with design and architecture. The human machine interaction is something that’s unknown, so putting a bionic structure on a human and adapting its control in real time in a kind of human machine optimization is critically important to the future of this area of design. Imagine putting on a device and you just starting to walk and run and it adjusts its behavior to optimize its performance with you in a collaborative effort.

Audrow Nash:
Is this one of your research interests moving forward?

Hugh Herr:
Absolutely.

Audrow Nash:
What are some of your near-term research goals?

Hugh Herr:
I want to do three things first and foremost. One is I want to advance better muscle-like actuators. Perhaps actuators that are better than biological ones, kind of the engine of BionX. I want to understand electrical interface between the peripheral human nervous system and devices, so how to talk to nerve endings, essentially. I also want to understand how to attach machines to the body mechanically in a comfortable way. Those three innovations will really define bionics in this century. If you solve all three, you’re more or less done. You’ve solved bionics, limb bionics that is.

Audrow Nash:
Thanks for being on Robots Podcast.

Hugh Herr:
Thank you.

In this episode, Audrow Nash interviews Fredrik Gustafsson, Professor in Sensor Informatics at Department of Electrical Engineering in Linköping University, about an initiative to reduce poaching in a rhino sanctuary in Ngulia, Kenya. Gustafsson discusses how he first became involved in this project, how he has worked with the rangers to develop solutions, and the future of this work.

Fredrik Gustafsson

Fredrik Gustafsson is professor in Sensor Informatics at the Department of Electrical Engineering, Linköping University, since 2005. He received the M.Sc. degree in electrical engineering 1988 and the Ph.D. degree in Automatic Control, 1992, both from Linköping University. His research interests are in stochastic signal processing, adaptive filtering and change detection, with applications to communication, vehicular, airborne, and audio systems. His work in the sensor fusion area involves design and implementation of nonlinear filtering algorithms for localization, navigation and tracking of all kind of platforms, including cars, aircraft, spacecraft, UAV’s, surface and underwater vessels, cell phones and film cameras for augmented reality. He is a co-founder of the companies NIRA Dynamics (automotive safety, including tire pressure monitoring systems found in more than 20 million, here), Softube (plug-ins for music studios and software solutions for Marshall and Fender), and Senionlab (leading manufacturer of indoor navigation solutions for sm, here).

Links:

• Download mp3 (16.9 MB)
• Project Ngulia website
• A PDF overview of Project Ngulia
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Photo source: ubcpublicaffairs/YouTube

Less than 100 years from now, robots will be friendly, useful participants in our homes and workplaces, predicts UBC mechanical engineering professor and robotics expert Elizabeth Croft. We will be living in a world of Wall-Es and Rosies, walking-and-talking avatars, smart driverless cars and automated medical assistants.

But much work remains before robots will truly be integrated into our daily lives. In this short Q&A, Croft lays out the rules for engagement between humans and robots and explains why it’s crucial to get this aspect right.

What role will robots play in our lives in the future?

Elizabeth Croft, UBC mechanical engineering professor and robotics expert. Photo: UBC

They will be everywhere, helping us at home and at work. They could make you breakfast in the morning and check on your kids. They could be your frontline staff, giving visitors directions and answering questions. They could be your physician’s assistant. Or you could have a robotic avatar that will attend a meeting for you while you’re traveling on the other side of the world.

Future robots may be self-replicating, self-growing, and self-organizing. The natural evolution of robotics is toward incorporating biology. We can now grow cells around bio-compatible structures; this opens the door for the combination of the biological systems with embedded artificial intelligence, and eventually the cyber-physical workforce of the future.

What technologies are driving the growth of robotics?

Computing power continues to grow exponentially, and ubiquitous communication is being made possible by wireless technology. Dense energy storage and new energy harvesting and conversion technologies allow machines to operate in the world, unplugged. And finally, machine learning: networked computer systems have global access to huge amounts of data that, combined with robotic embodiment, allow robots to learn about the world in ways that mimic and move beyond how people learn about their environment.

Your work at UBC focuses on human-robot interaction. Why is this important? 

As robots become more and more a part of our lives, the question becomes: what are the rules of the game? What is OK for robots to do, and what is not? Robots will have abilities that we don’t have, and we need to define what they are allowed to do with that capacity.

There are some big ethical questions to consider: how does society deal with drones that can kill, for example. But there are important day-to-day questions too. If a human and a robot are accessing the same resource – the same roadway, same tools, same power source, who yields? Does the person always get their way? What if the robot is doing something for the greater good, for example, a robotic ambulance?

Researcher with robot. Photo source: ubcpublicaffairs/YouTube

In a way, I like to think of our lab as robot kindergarten. We are teaching robots basic, building-block behaviours and ground rules for how they interact with people: how to hand over a bottle of water, how to look for things, how to take turns. Having these basic behaviours in place allows us to create human-robot interactions that are natural and fluid.

To achieve our goals, our lab welcomes researchers from different disciplines—ethics, law, machine learning, experts in human computer interaction—as well as different international cultures. Different cultures have different ideas of robots. We learn a lot from these many perspectives.

Elizabeth Croft will speak about the future of robotics at a UBC Centennial public talk on May 28. Click here for more information.

Photo courtesy We Robot 2015.

UPDATED 4 Mar: We’re sad to report that Professor Tony Dyson, who built the original Star Wars R2-D2 droid, has died. We’re reposting this excellent video of his keynote at WeRobot to highlight his contribution to the field of robotics and culture.

This article was first published on IEEE Spectrum Automaton and is republished here with permission. For the IEEE version, click here; for the unabridged version, continue reading.

Collaborative robots (also called co-bots) are robots designed to assist humans in the performance of their tasks, i.e., to work jointly on a project. Because co-bots are almost plug and play — and very low-cost — SMEs are buying and the segment is quickly growing.

There are many reasons for the emergence of collaborative robots: to handle ergonomically challenging tasks, using them alongside humans in small-spaced electronics assembly lines, using them because they are affordable, using them because they are plug and play and easily trainable, using them for their flexibility to handle short runs, and using them to offload repetitive and boring jobs.

Co-bots are also emerging because sensors and computer power have become so cheap that they are driving down the cost of robots and making them more available to businesses of all sizes. Initially driving this movement was the EU SME Robot P-P-P started in Europe in 2005 with a goal of increasing worker productivity so that labor-intensive tasks wouldn’t be off-shored. Recently the movement has been promoted by the launch of the Baxter robot and its developer and chief promoter Rodney Brooks. And by the ease-of-use factor: co-bots are easier to train and deploy than big industrial robots.

Most professional service robots are collaborative by design; hence co-bots aren’t just for SMEs or for manufacturing use. Offices, homes, labs, warehouses, farms, distribution centers, hospitals and healthcare facilities are all enabling service robots to help them do their jobs better. The market is open-ended but current uses for co-bots include machine tending, material handling, assembly tasks and packaging. They also pick and place, count and inspect. As more co-bots are deployed, more uses will be discovered.

Huge Growth Potential:

In a human-machine study conducted by an MIT professor at a BMW factory, it was shown that teams made of humans and robots collaborating efficiently can be more productive than teams made of either humans or robots alone. Also, the cooperative process reduced human idle time by 85%.

At present, sales of collaborative robots represent 5% of the overall robot market but have strong growth expectations for the future. In fact, the market acceptance of collaborative robots and also drones are expected to be significant drivers in non-industrial robotic growth. This non-industrial growth will be split and different for differing aspects of the service robotics marketplace, but as Tractica (a Colorado research firm) projects, growth will be significant, exponential, and near-term.

The collaborative robotics sector is expected to increase roughly tenfold between 2015 and 2020, reaching over $1 billion from approximately $95M in 2014. Insiders suggest more rapid growth: that collaborative lightweight robots will become the top seller in the industry in about 2 years, selling hundreds of thousands and with prices falling to the $15,000-$20,000 level. In their September 2015 report, British market research firm TechNavio forecasts the global collaborative robots market to grow at a compounded annual growth rate (CAGR) of 50.88% to 2019.

Safety:

Industrial robots are caged to keep humans safe and out of harm’s way. Service robots are meant to safely leave the cage while doing tasks for humans. Collaborative robots come in all sizes and shapes and have integrated sensors and soft and rounded surfaces for safety purposes and to reduce the risk of impact, pinching and crushing. The biggest safety feature of collaborative robots is their force-limited joints, which are designed to sense forces due to impact and quickly react.

Companies and their co-bots:

Rethink Robotics, a venture-funded Boston-based robotics firm headed by Rodney Brooks, markets a two-armed industrial robot named Baxter with a base price of $25,000 that can carry out relatively advanced functions such as removing defective items from product lines, packing finished items into boxes and performing basic quality-control inspections. Additional purchase costs include a mobile pedestal ($3,000), gripper ($1,750), and a 2-year hardware and software service contract ($7,000). With an expected useful life of three years, 40 hours of operation per week translates into an average cost of around $4/hour. Though this comes before additional expenses for electricity, maintenance and repair, it remains highly competitive with manufacturing wages in the emerging world.

Sawyer

Rethink also sells a narrower one-armed robot named Sawyer with a base price of $29,000 and similar additional costs as Baxter. Sawyer is designed primarily for machine tending, circuit board testing, and other precise, repetitive tasks, specifically those that take place in the middle of a long assembly line of electronics products. There’s a huge need for a robot that can do tasks like these, especially in Asia, and that’s the market Rethink is going after with Sawyer. “We’re moving into mass electronics manufacturing,” said Brooks in an interview connected with the receipt of a funding round for scaling up sales into China and the promotion of a distribution agreement with Shanghai Electric.

Universal Robots, a Danish startup whose one-armed, 6-axis robots have had significant success in the marketplace. At this time they are the market leader producing at an annual run-rate of 5,000+ co-bots. They recently added a third robot to their product line and now have over 200 integrators and distributors around the world and thousands of sold robots at work in SME and larger factories.

All three of their robots are identical except in size and carry. The UR3, UR5 and UR10 can carry up to 3, 5 and 10 kilograms of load (6.6, 11 and 22 pounds). Pricewise, the UR10 costs $45,000, the UR5 costs $35,000 and the new UR3 is priced at $23,000. The UR3 can handle payloads of up to 3 kg (6.6 lbs), and perform tasks that include: mounting of small objects, gluing, screwing (it can tighten applying the correct torque), operating tools, soldering and painting. It has a reach radius of 500 mm (19.7 in) allowing it to be deployed in tight spaces.

Universal has found another and perhaps bigger use of co-bots through their work with automaker BMW, which already has 7,500 industrial robots at work in their factories. BMW has been testing UR robots alongside factory workers who had been tasked with ergonomically challenging assignments. The robots were quickly trained for those tasks and performed perfectly while freeing up the worker to do even more of what he or she was doing. The results of the tests turned out so well that a BMW spokesman, who oversaw the testing, said that it was likely that BMW would soon double or even triple their number of robots by the use of these low-cost, portable, easy-to-program, safe to work alongside robots. (With that kind of endorsement and prospect for the future, it’s easy to see why Teradyne paid $350,000,000 for Universal Robots.

MRK-Systeme GmbH, a German integrator of industrial robot technology, offers two collaborative software packages which work, at present, with KUKA’s 5SI low-payload robot: SafeGuiding for moving the robot manually to teach it applications, and SafeInteraction which provides five human-robot safety functions to the robot control system.

They also integrate Kuka’s robot into their own branded KR 5 SI one-armed robot.

F&P Personal Robotics, a Swiss spin-off from the University of Zurich, offers a line of co-bots emphasizing safety and functionality. Their robots are context aware, use soft material covers, have limited and controlled force sensors, motor torque is limited by the controller, and their software uses deep learning AI to enable their robots to adapt and improve based on human feedback.

F&P recently introduced their new  P-Rob 2, an all-in-one robotic arm, with sensors and software embedded in a PC as control unit in the base of the robot.

Bosch developed the APAS line of co-bots and has deployed 50 of them, mostly for internal operations. Safety is provided by a special sensor skin and software controls. Training is done using a graphic user interface on a teach pendant and the robot can be outfitted with an optional 3D camera system.

The Bosch APAS system was the first to be certified as an assistance system, i.e., safe to work alongside with human workers.

ABB, one of the Big Four global industrial robot manufacturers, recently launched a two-armed collaborative robot capable of advanced manual tasks. Called YuMi, the robot is able, with a high degree of precision, to handle small parts assembly such as mechanical components for watches, smartphones, tablets, PCs and other consumer electronics.

The YuMi is dexterous enough to be able to thread a needle.

Early in 2015 ABB acquired gomTec, a startup producing a one-armed co-bot named Roberta. gomTec was a 25-person German company headed by Bernd Gombert, a mechatronics engineer with 100+ patents and who was the head of a DLR project which became KUKA’s LBR iiwa robot (see below). Roberta is a family of 6-axis modular co-bots using custom-made brushless motors and harmonic drives. Safety sensors and joint torque sensors can be optionally added. ABB is in the process of integrating a new intuitive programming user interface and rebranding Roberta to ABB’s brand and expects to launch in Q1 or Q2 of 2016.

Because ABB is so big and has such an extensive sales, distributor and integrator marketing force, they are expected to quickly penetrate the small parts assembly market with their lineup of plug and play one- and two-armed low-cost co-bots.

MABI Robotic is a Swiss manufacturer of industrial robots. Their slick precision Speedy 10 one-armed table-top robot is lightweight, fast, easily trained but unclear as to whether it is safe to work alongside.

FANUC Corp makes CNCs, servo motors, lasers and a full line of industrial robots. Fanuc is the biggest of the Big Four robotic providers. Their CR-35iA is their first collaborative robot and it can carry the heaviest load of all the co-bots so far announced, 77 pounds, and has full safety compliance using soft rubber skin and multiple sensors including either a vision sensor or a 3D area sensor.

The CR-35iA is too big for work in a narrow space such as on an electronics assembly line but it’s ability to carry heavy loads opens it up to other tasks better suited for a robot than a human such as inserting a spare tire into a new car’s compartment.

Smokie Robotics, a Tennessee startup, produces a modular one-armed robot configurable for 3 to 7 DoFs. Their OUR-1 robot uses open source architecture enabling easy integration into existing production systems and their OUR-2 robot will be available in Q1 2016.

KUKA Robot Group, headquartered in Augsburg, Germany, is one of the global Big Four producers of industrial robots. Their first foray into co-bots is the LBR iiwa (lighweight robot, intelligent industrial work assistant) which comes with the ability to carry 15 or 30 pounds of payload and the Sunrise controller. The robot has 7 joint torque sensors which enables the control system to detect externally applied forces. The robot has the sensitivity of a human and can quickly feed information through the controller to safely control the motions of the robot.

The KUKA LBR iiwa fully complies with the industry safety standard ISO 10218 and the new standards for human-robot collaboration ISO 12100 and ISO 13849. Also, the LBR robot has been at work in the Daimler transmission plant since it was first tested in 2009. It has been used in the processing of more than 500,000 transmissions since then. Although the robot is programmed to perform automatically, it approaches objects carefully and adjusts to changed conditions. For example, if a workpiece is not optimally positioned, the robot adjusts its motion sequence accordingly using it’s SafeOperation software to restrict or monitor workspaces and safety zones between machines or between humans and machines.

KUKA, like the other Big Four vendors, has a global marketing capability consisting of sales people, distributors, integrators and resellers. If they can get the selling price down from its present $100k+, they have the ability to become a major player in the SME market – a significant breakthrough expanding KUKA’s products into that arena.

Kawada Industries makes the Nextage two-armed industrial robot. Safety is handled by the stereo vision cameras in the head and by using low-power motors and elbow controls so that don’t jut outwards even when both arms are in operation. Over 150 of the $60,000 robots have been sold and are at work in factories in Japan, according to Discuss Japan in a May, 2015 report.

What’s Next for Co-bots?

Companies of all types and sizes are finding strategic reasons to acquire or invest in robotics and robotic ventures to add to their arsenal of products and services as the metrics of ownership and deployment become more affordable.

Universal Robots has a big head start. 2016 will likely see them continue to lead, but KUKA, ABB and FANUC will begin to make inroads and experiment with different prices. By the beginning of 2017, the competition will become even more intense as the number of co-bots sold approaches 15,000 units – $0.5 billion in sales revenue. It’s still too early in the evolution of co-bots for provider consolidation, but some systems are sure to be preferred because of their flexibility, ease of training and support base.

The SME marketplace is huge – 6 million companies worldwide and almost 70% of the world’s manufacturing. A few low-cost plug and play robotic tools can easily fit into the manufacturing process in most of these companies, which is why it is easy to imagine that co-bots could be on track to sell hundreds of thousands of units beginning as early as 2018.

A significant obstacle to robots achieving their full potential in practical applications is the difficulty in manipulating an array of diverse objects. The Humans to Robots Lab at Brown is driving change in this area by using Baxter to collect and record manipulation experiences for one million real-world objects.

Central to the research at the Humans to Robots Lab is the standardization and distribution of learned experience, where advances on one robot in the network will improve every robot. This research seeks to exponentially accelerate the advancement in capabilities of robots around the world, and establish a framework by which the utility of these types of systems advances at a pace never before seen.

To better collaborate with labs around the world, Professor Stefanie Tellex and the team at Brown use the Baxter robot, an industrial robot platform that has the capability to automatically scan and collect a database of object models, the flexibility of an open source software development kit and an affordable price that makes the platform accessible to researchers all over the world. As a result of the Rethink Robotics Video Challenge, Brown will have an additional Baxter that will accelerate this research, while also providing new opportunities for continued experimentation.

“Our goal in creating the Rethink Robotics Video Challenge was to raise awareness of the tremendous amount of unique, cutting-edge research being conducted using collaborative robots that advances our collective education. The response far exceeded our expectations, and narrowing this down to one winner was an extremely difficult task for our judging panel,” said Rodney Brooks, founder, chairman and CTO of Rethink Robotics. “Brown was ultimately chosen as the best entry because the work being conducted by the Humans to Robots Lab at Brown University is critical to helping robots become more functional in our daily lives. There is a stark contrast between a robot and human in the ability to manipulate and handle a variety of objects, and closing that gap will open up a whole new world of robotic applications.”

Educational institutions, research labs and companies from around the world submitted an abstract and video showcasing their work with Baxter in one of three categories: engineering education, engineering research, or manufacturing skills development. The submissions encompassed a wide range of work, including elementary school STEM education, assistance to the physically and visually impaired, mobility and tele-operation, advanced and distributed machine learning and cloud-based knowledge-sharing for digital manufacturing and IoT, to name a few. After a detailed vetting process, the field was narrowed to 10 finalists from the following organizations: Brown University, Cornell University, Dataspeed Inc., Glyndwr University, Idiap Research Institute, North Carolina State University, Queens University, University of Connecticut, University of Sydney and Virginia Beach City Public Schools.

Finalist entries were reviewed by a premier panel of judges, including Rodney Brooks of Rethink Robotics; Lance Ulanoff, chief correspondent and editor-at-large of Mashable; Erico Guizzo, senior editor at IEEE Spectrum; Steve Taub, senior director, advanced manufacturing at GE Ventures; and Devdutt Yellurkar, general partner at CRV.

To view the ten finalist entries, please visit www.rethinkrobotics.com/videos.

Transcript below.

In this episode, Andrew Vaziri interviews Magnus Egerstedt, Professor at Georiga Tech, about his research in swarm robotics and multi-agent systems. They discuss privacy and security concerns, as well as research into interfaces designed to enable a single operator to control large swarms of robots.

The video below shows some of the strategies used by Magnus’ lab.

Magnus Egerstedt

Magnus Egerstedt is Schlumberger Professor in the School of Electrical and Computer Engineering at the Georgia Institute of Technology, where he serves as Associate Chair for Research. He received an MSc in Engineering Physics and a PhD in Applied Mathematics from the Royal Institute of Technology, Stockholm, Sweden, and a BSc in Philosophy from Stockholm University. He was then a Postdoctoral Scholar at Harvard University. Dr. Egerstedt is the director of the Georgia Robotics and Intelligent Systems Laboratory (GRITS Lab), a Fellow of the IEEE, and a recipient of a number of research and teaching awards, including the Ragazzini Award from A2C2.
Links:

• Download mp3 (17.8 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Magnus Egerstedt’s Website

Transcript

Andrew: Welcome to the Robots Podcast. Can you introduce yourself to our listeners and tell us about your research focus?

Magnus: I’m Magnus Egerstedt and I’m a swarm robotics professor from Georgia Tech. I’m interested in finding out how to get large teams of robots together, in particular when the individual robots may not be all that talented. The question: how can they make decisions so that beautiful, effective and exciting global patterns in behaviors emerge?

Andrew: Is this limited to physical robots, or are there other systems where this can be applied?

Magnus: I’m mainly interested in robotics, but this question of distributed decision making is really something. It could go from smart invertors on the smart grid to computational units and video games, to biological systems in that order. We have cells, we have birds, and we even have human societies – they’re all distributed systems as we walk through life with limited information. And yet, we’re able to build anthills and levers and human societies.

Andrew: Swarms of robots …. distributed systems … these are very complicated things. How do you generally go about trying to model those?

Magnus: I always start by pretending that I’m a somewhat stupid robot with very limited information, and start by asking: “What would I do?  How would I think about this?” After I’ve tried to do that, step two is almost always look to nature, because nature is filled with examples of elegant solutions to our problems. Now, I’m not a biologist and I am by no means trying to mimic nature. I’m shamelessly stealing good ideas that social and behavioral biologists have had. Then, when I’m done stealing from nature, I sit down with pen and paper, work out the math, put it on the robots and find it doesn’t work, because it never works when you go from math to robots. You have to iterate on the actual robots a few times.

Andrew: When you’re working with these robots there are constraints that we don’t have in nature, in terms of the way they’re able to communicate with each other. Is it difficult to model that? What is the approach?

Magnus: When you actually start deploying large teams of robots there are some things that are just impossible to model.

Things happen that you didn’t foresee: robots bumping into each other saturating other communication channels, weird occlusions and the infrared sensors messing with each other. There are always things you cannot model. One of the great arts of swarm robotics is: “How do you go from rudimentary fables or models that are somewhat relevant to things that can be deployed on a large system?” This transition is a lot of times tricky, but it’s also deeply satisfying. Swarm robotics is exciting because in the last 10 or so years we’ve gone from spotty anecdotal understanding of how to structure these things, to large teams of robots on the ground, water, and air that can actually coordinate their actions and build interesting geometries.

We now understand certain things that we didn’t ten years ago. Now it’s common to see 25 robots coordinating actions on YouTube. Ten years ago this would have been unheard of. The basic functionality of how do we build these geometries or get these global behaviors out, we’ve kind of understood both on a practical and on a mathematical level. Now there are a huge number of issues we don’t understand. Let’s say for instance one of the robots is not playing by the right rules, or let’s say it’s broken – or even worse – it’s malicious. How do we identify this? How do we not let our robots be duped by these bad robots? There are a huge number of questions that we clearly don’t understand yet, but do we understand the basics of how to assemble these kinds of structures.

Andrew: I think that’s interesting you mentioned malicious actors potentially being in the system. I know you recently published a paper about differential privacy in multi-agent systems. What is differential privacy, and what is it good for?

Magnus: Let’s say you and I and other robots are trying to do something. But what we’re doing individually is really having our own cost of utility function that we’re trying to minimize. I may like to be where it’s light and you want to be positioned so that you’re not too close to flying robots, because you’re afraid of flying robots. We have to live in this space together, but we have our own objectives. Differential privacy is a way of hiding certain information by sprinkling noise onto what we do. For instance, I may move in a little bit of a noisy way so that you can’t figure out what I’m up to. We’re still respecting each other’s actions, but no one can infer what I am trying to do. The trick here is to insert the noise in a clever way so that we don’t end up doing erratic things. We’re achieving what we set out to do as individuals, but no one can tell what’s going on.

This idea of differential privacy came from the database literature. Let’s say we have a database of all our salaries. I add my salary to the database. If you know how many people are already in the database, and you know how it changed, then you can back out my salary. But if all of us sprinkle in a little bit of clever noise then certain things about the salaries – like what standard salary and so forth – you can still get from the database, but you cannot back out any individual pieces of information. It’s the same idea with swarm robotics. You sprinkle noise on the motions to hide your intentions. This is a flip-side from the malicious behavior where you want to be able to detect a robot isn’t playing by the rules. There are two sides of the same coin. One is to hide your intentions, and the other is you want to be able to figure out what bad robots are doing. Bad robot!

Andrew: I think a theme that we’ve all seen in our local big box stores is that everything wants to be smart these days: smart grid, smart homes. Is that something that we should maybe take with a grain of salt given the concerns of being able to peer into these systems and learn something or maybe we shouldn’t know?

Magnus: Fortunately yes, I think one thing in general when we talk about technology is it’s not going to be decoupled from us; we’re going to be users of technology. We need to – in my mind – build our systems so that it respects us as individuals and maintains our privacy. People are concerned about self-driving cars. We’re not going to wake up tomorrow and be sitting in self-driving cars. This is sneaking up on us whereby we get more autonomy in the cars, but people are engaged. People are almost always going to be in the mix when it comes to distributed robotics systems. Also distributed gadgets and the internet of things; people are going to be on the internet of things together with their things.

Andrew: Differential privacy could be one way to make the internet of things a little bit more private. You mentioned self-driving cars. I know that you’ve done research on the other side of the coin where malicious agents are actively trying to change or affect what the system does. It sounds like a plot from a spy movie. Could you tell us a little bit about controlling an enemy unmanned aerial vehicle by exploiting its own safety systems? How did that work?

Magnus: We were really interested in this question, as we’re about to see aerial drone delivery systems. Google and Amazon, if you believe the talk, are going to be delivering things by aerial drones. The question is then: “Is it possible to take over these systems and take over in such a way that the systems themselves do not know they’re being taken over?” This is not a classic cyber security question; it’s a cyber physical security question.

A lot of these systems including our aerial drones that we were playing on have safeguards to hear our reasonable physical behaviors. Basically it’s – here is the stuff that is within reason! And as long as we are within reason we’re going to be fine. Our question became: “Can we hide malicious intent inside the noise of what is considered a reasonable behavior, and yet, get the systems to do completely the wrong things.” I don’t want to say that we completely demonstrated this is the case, but we’ve certainly demonstrated examples where it is possible to hide and trick the system into thinking that it’s doing the right thing. We’re camouflaging the malicious trajectories inside reasonable and rational behavior.

Andrew: In this particular example how did you inject for that signal, what was the mechanism by which you influenced the aircraft?

Magnus: In my lab we’re not hackers. Since we designed that aircraft we had an error system or a safety system that detects that something was wrong. Then we transferred wirelessly over the network and injected new signals. If someone is really clever they could probably build something to detect this is going on within the cyber security side. That was not what we were interested in. What we were interested in was to see if it is possible to hide the malicious takeover within the cyber physical security system. This is similar to the idea behind Stocks net. Where nuclear centrifuges control systems were made to believe they were behaving right. Not because someone hacked in, but because you made the signals look plausible to the security systems.

We actually drew a lot of inspiration from … this is kind of amusing … from dragonflies. Dragonflies will move in such a way. Let’s say you’re a dragonfly; you’re a male dragonfly and I’m another male dragonfly and now we’re going to fight. I move in a way to have the sun in a straight line behind me, so now you can’t tell where I am. I go in making sure that no matter how you move I’m going to move, so that you only see the sun and then BAM! I’m going to get you. Similarly in control for aerial there’s a notion of an observable subspace which here are things that the trajectory of the system can do that’s hidden from the output. We exploited this by restricting emotion to these unobservable modes of the system.

Andrew: That’s very interesting. This adds a whole other layer. It’s not just cyber warfare, it’s not that you hacked; you changed the code that was running on the target. It’s exactly intact as it was designed by its creator. You’ve changed what it observes and what it thinks it observes. That seems like a very hard thing in general to defend against, to not only take into account the cyber aspects but also all the ways things could physically interact. Is there a principled way to make sure that systems aren’t vulnerable in this fashion?

Magnus: We have been looking into this question and at this point there isn’t a principled way but here are ways people have done it. We looked at what we actually called motion probes. The system every now and then does a somewhat strange manoeuvre, where it’s basically a signature manoeuvre. It could make a small sidestep, or the quad copter makes something that resembles a figure eight. This is unexpected; if the attacker isn’t expecting it then you can actually excite these previously unobservable modes and say, “Hey, something is actually not behaving the way it should be there!”

This is how we are thinking about it and it also goes back to this question of detecting not malicious takeovers, but malicious agents. If we are swarm robots, we’re supposed to do something. If I then make a figure eight I know what you’re supposed to do in response to that, if you are one of my people. If you don’t respond correctly I can detect that perhaps something is fishy here.

Andrew:  I’d like to switch gears a little bit and ask about something that’s high risk, but in a different sense. I recall reading, maybe four years ago in a systems magazine. There’s a little blub about you and you wrote about your research philosophy and research style. You mentioned liking to keep a few high risk, high payoff topics on the horizon. I just wanted to ask, what does that mean? What does a high risk, high payoff topic look like to you?

Magnus: The wonderful thing about being a professor is that you’re largely free to explore things you find interesting. I have to pay the bills and, more importantly, make sure my graduate students don’t stave. I need to do things that are five years out which people care about. But I also have the freedom to think about things I truly am interested in that are way out there. If they come to fruition they’re going to be important, but if they don’t chances are they don’t. I always like to have a few of these questions that I think are fundamentally important and interesting down the line on the back burner.

Every now and then I return to them, so right now I have a few. One thing that I’m really fascinated by is this question of human swarm interactions. Let’s say you’re a person with a joystick surrounded by a million robot mosquitoes and you want them to do something. What are the right ways to engage with truly large teams of things? That’s one thing that I’m fascinated by right now. Another thing that drives me absolutely nuts is that people talk now about heterogeneous multi-robot system.

Andrew: What does that mean?

Magnus: It means different kinds of robots. In my lab, we have things on the ground and things in the air. But fundamentally, why is it good to be different? Why should I have fast and slow robots together? What is it that makes that better than just having a bunch of fast robots?

To me, this is an interesting question because we can understand what kinds of robots we should deploy. If you have an earthquake and you want to send out robots to figure out what’s going on in the rubble you can say the optimal thing to do is to have nine  snakes robots, 15 aerial robots and 24 ground robots. And here’s what they should be doing. To me this is fascinating. I again look to nature like ecosystems which are highly diverse and consider, why is that? How come hummingbirds’ live next to sloths? Is there something fundamentally good about that?

Andrew: I recall from some years ago you mentioned one of these high risk areas was human swarm interaction. How has it worked out over the past few years there were moments where it seemed dire?

Magnus: First of all, human swarm interaction has become an expectable thing to spend your time on and there are other people engaged with it. I think we’ve understood things, if you have small teams of robots then an effective way of engaging with them is actually to grab hold of a leader and inject information through the leader. That works well up to maybe four or five robots, then it kind of collapses. The reason why is that people get bad at imagining what – let’s say I have ten robots around me and I grab one and I shake it. How does that propagate through the ten robots? We just lose track.

For example look at universities, militaries, big companies … we’re all hierarchically organized. I think the reason becomes we don’t know what happens if we have 250 people to manage, all at the same time. I looked at sheep herding and found you’re not controlling individual sheep. You’re pushing things at the boundary with herding dogs. Instead of controlling individuals you’re exerting forces on the outskirts of the swarm. It turns out we can do some things like that quite well. I have worked with swarming tractors, so the question becomes: What does the farmer do at the side-line of the field with an iPad? Maybe one way of engaging is not to engage with the tractors at all but to engage with the map and say, “Here are regions of interest. Now you tractors, go figure it out!”

Andrew: You did some research with a leader-follower system and in that case it was a haptic interface, could you tell us a little bit about that?

Magnus: Yeah, this goes back to this question of its very unclear for people what it means to control large teams of things. We needed some way of making this information concrete to a person.

We basically computed something that we called the manipulability, which is a measure of how easy it is to move this swarm in particular directions and this we could compute. Then we used that to generate forces. The idea is try to push the swarm in a direction in which it was hard to move it then we’re experiencing large forces. It turned out that this was fairly useful. We did user studies that people could solve tasks that were very hard to solve with only a joystick, but with a haptic joystick they could solve tasks more effectively. This goes back to the question of, what information you actually need about a large swarm. In this case it was just force. If you want them to do more elaborate things you probably need more information. But if in general you just want to move the swarm from point A to point B then we thought the findings in this study where haptic feedback serves a strong and positive purpose.

Andrew: Could you paint us a picture of what the experimental setup looked like? What were the robots like? What were the users looking at and what were they holding?

Magnus: The user was holding a phantom omni, the brand doesn’t matter. It was a haptic interface; it looks like a little pan connected to a ball. You’re moving it up, to the side, or in. You’re experiencing a force depending on what feedback that the little haptic force generating ball is giving you. That was what the user was holding. Then the users were in the lab and we had 10 robots. The robots were arranged in a particular formation. What the user could do is select which robot, he or she, wanted to use as the leader and then drag it in various directions.

You can imagine having a long and skinny formation and the leader robot is embedded inside it in the middle. If you move along the direction of the long and skinny thing then that’s going to be hard! You have to move all these things out of the way and you’re experiencing lots of forces. It has what’s called low manipulability. If you move it away from the long and skinny direction, then it’s easier to melt into a form or shape that follows the leader. The user was looking at the 10 robots, the user did not know how the robots were organized, and basically tried to get them to go from one point to another point while navigating a little bit of an obstacle course, so no robots were allowed to actually get stuck behind an obstacle.

Andrew: How did you assess if it was being effective if the person felt like it was a good interface or not?

Magnus: Yeah, this is always when you do user studies, how do you know if it’s good or not? We used three metrics. One, we just timed them. So how long does it take? Two, we looked at how much did the user have to work. We summed up the control signals that were sent to the swarm. And third, we asked them! Basically we asked: did you find this annoying? There are probably more scientific ways of asking that question, but that’s basically what it boils down to: what level of stress, engagement, or discomfort did you feel when you were doing this?

Andrew: A more advanced interface might be something you alluded to the farmer by the field with a tablet and algorithms that were suited to that. Could you tell us a little bit about that?

Magnus: The backdrop to this is rather interesting. The tractors now are really advanced pieces of machinery. They can largely drive themselves. Put in the GPS of the path and then they can, more or less, do this by themselves. Moreover, cornfields are really not that complicated they’re nice and straight. In terms of the types of environments you’re encountering they’re not that complicated. The manufacturing floor is way more complicated, or my daughter’s bedroom … way more complicated. The farm field is not that complicated. This tractor manufacturer came to me and said, “You know what we know now.” So how do we make these autonomous, which means we don’t need a driver in the tractor.

That means we can make the tractors smaller, and that’s good. Maybe because it has less environmental impact, you don’t have to go back and retail afterwards. These are all good reasons why small is better. Except that it’s not less effective. Instead of having one small tractor lets have ten small tractors. Then the question is but how do you actually engage with these? They literally came and asked, “What should the farmer do, we want to give the farmer an iPad, what should the farmer do?” That was the thing we got rather excited about.

Andrew: What did you end up making, what algorithm and what kind of user interface or display?

Magnus: We ended up going with something that’s fundamentally a good idea, which is that the farmer should actually not deal with the tractors. What the farmer should be dealing with are higher level tasks. In this case what we had is a map of the world and the farmer could outline things that were interesting, or things that were urgent. Or where are where there was lots of moisture, or not a lot of moisture. Things that basically described we used, we call density functions to describe areas of the interest. Then robots run what’s called a dynamic coverage algorithm. That means the robots disperse themselves in such a way that they cover areas of interest where we get the same amount of stuff we’re in charge of. Technically, this is called a voronoi partition. The robots computered a voronoi partition.

Then inside that partition they did their driving around. Then the farmer could do is move their finger around, and on the flight dynamically the robots reconfigured themselves. We took this and this now, which has nothing to do with tractors anymore. But we have this general way of painting event density on areas of interest and then have the robots dynamically respond to that in a distributed manner.

Andrew: Have you received any feedback from either farmers or researchers about some of these advancements?

Magnus: So I have! I did a photo shoot where I was wearing a weird baseball cap and rubber boots in the lab. Don’t make fun of me; I pretended to be a robot farmer. But seriously people are excited about it. The interesting thing about swarm robotics is that it’s up and coming and these kinds of environmental surveillance tasks may be farming or just trying to figure out what’s going on in an area. I had a NASA project where we were monitoring effects of climate change down at Antarctica. We actually never ended up going to Antarctica, unfortunately.

There, you deploy robots for fairly long periods of time and they have to just respond to what’s going on in the environment and this kind of way of painting the world at the user level with areas of interest is a very natural thing. Now if you’re flying two drones, that’s not how to do it, then you need to worry about individual aircraft. If you’re just interested in covering areas then this is a really effective way of doing it. We’ve had quite a lot of funding from the Air Force Office of Scientific Research to pursue this further because in a way it’s a very simple way of describing missions that robots can immediately execute.

Andrew: In closing, I wanted to ask you a philosophical question. Would it be fair to say that you’re a fan of soccer?

Magnus: Ha! Yes, it would be fair to say that I’m a fan of soccer. In fact as soon as we’re done with this I’m going to go put on my soccer gear and go play a soccer game for old men; you have to be over 40 to play in that league.

Andrew: You must be aware of robocup’s robotic soccer league. Currently, they’re pitting teams of robots against each other. Their goal is to field the team of robots to beat the world cup championship human team by 2050. I think that shows a high level of optimism and the future of multi-agents teams and in swarm robotics perhaps as well. What do you see the future as, how will we get there?

Magnus: Robots are already better than people at many things. Robots are really good at not getting bored. They’re really good at responding quickly to things. They’re really good at crunching big numbers. I think if you do a robotic 100 meter dash.

We’re not far away from having bipedal robots that are faster than Usain Bolt, the fastest runner in the world. As a soccer player if you look at the artistry of Messi we’re way far away from that. I think part of that is actually not an algorithmic question. Where we’re lagging right now a lot of times is that actuation and power supplies. The humanoids of today that are not tethered, they have to schlep very large battery packs with them and, if you’re lucky, 3 degrees of freedom foot. Messi’s foot has many more degrees of freedom and he’s small and agile.

Andrew: Perhaps the World Cup is megatronics challenge. What to you would be the swarm robotics coming of age moment where you say that’s the goal?

Magnus: That’s actually a really good question. A big part of what’s driving me is purely intellectual curiosity. I think there are deep scientific questions about how do you get truly emergent behaviors out from simple local rules.

Andrew: What do you mean by an emergent behavior?

Magnus: Simple emergent behavior is how do you send out robots to cover an area where all they’re doing is seeing very locally– what’s going on? A higher level question is that you have these little neurons and all they’re doing is sending spike trains around and, all of a sudden, you have conscious human beings. That’s a very high level and emergent behavior question. Once we understand that to me that’s a really interesting question and it has repercussions way beyond robotics. The big question is not, how do I get 200 robots to mow my lawn more effectively? Even though I do think we’re going to have teams of robotic lawn mowers, small little guys, which are more agile than these big robotic lawn mowers that are out there now.

But, scientifically understand, how do we make these distributed decisions? Along the way we’re going to have robots on Mars. We’re going to have swarms of robots on Mars. We’re going to have robot helpers in hospitals and on the manufacturing floor. These things are coming. Ultimately computers are useful tools, but they’re only useful in as much as we do useful things with them. I think people tend to think of robots as, “Oh my God, we’re going to have an army of terminators!” I hope not. I hope we’re going to use robots to make our lives better. And use them to find energy, food, and travel to remote locations around the globe. I hope that we, as people, will be able to use them as tools for good as opposed to tools for evil … like an army of terminators.

Andrew: A fascinating discussion, thank you for joining us!

Magnus: You bet, thank you for having me!

This audio interview was transcribed and edited for clarity. 

The big players

Yaskawa HC10 — Motoman/Yaskawa unveiled a brand new robot, which seems to have the same principle as the Green Fanuc: it’s an OEM industrial robot covered with a rubber skin (blue this time) to eliminate pinch points and sharp edges. As I understand it, the difference between their concept and Fanuc’s is that the Yaskawa robot monitors force in each joint, whereas Fanuc measures it at the base of the robot. Yaskawa also showed its new YRC 1000 controller, which is used to control the new robot. the HC10 is due for release next year.

Other Green Fanucs — Talking about Fanuc, they released a complete line of collaborative robots: CR-4iA, CR-7iA and CR-7iA/L, in addition to the existing CR-35iA. What I understand is that they are using the exact same principle, but scaled down a bit for the different models. So a soft cover, force-torque sensor at the base of the robot and the regular programming method. I guess they had a good response with their first model and wanted to re-use it. In any case, it is a good concept and does not require a totally new robot… so why not use it again. All robots are offered with vision and force torque sensing.

ABB Roberta dressed in white — It has been a while since we saw Roberta. Do you remember the purchase of Gomtec by ABB? This was the first time we saw Roberta with its new look. In fact, the robot is now part of the YuMi family. The team has been working on porting the robot into the ABB architecture, so it would make sense that software and mechanical devices had to be adapted to run like an ABB unit, while keeping all the interesting stuff that Gomtec had developed in the past. To be honest it looks quite good (especially in white). The hardware of the robot is flawless and the sensor at its wrist makes control nice and easy. I don’t have any info on the official release date. ABB was also showing the original 2-armed YuMi. PS: No offence, but I am pretty stoked to see Roberta in a legit marketing campaign and not just as a flowery backdrop.

Kawasaki duAro — Kawasaki presented its new addition: the dual arm scara robot, duAro. The robot is a great fit for electronic assembly where most motions are planar. The robot is designed to be collaborative. In electronic assembly lines where a lot of dexterity (i.e. human workers) is needed, it is just normal to power- and force-limit these devices. The robot was announced last summer.

New kids on the block

Life Robotics CORO — This is a new Japanese company that expects to begin selling robots next year. What’s innovative is that the robot seems to have one retractable segment to increase its reach. Initially, the robots will only be sold in Japan. Original use of prismatics makes this robot’s footprint very compact. They have been pretty secretive about pricing, which makes me suspect they will offer it at a really affordable price.

TM Robotics (subdivision of Quanta Storage, maker of the Apple mac book) — A company that comes from the IT industry will certainly give you an easy-to-use, totally integrated solution, and this is what is happening here. The robot is smart, simple and safe, and it has integrated vision, integrated wifi and uses a tablet as its teach pendant. The Taiwanese robot is supposed to be released next year. This is going to be an interesting one to follow.

The usual suspects

• Kawada had an impressive demonstration integrating several robots. The Nextage being their industrial robot, it seems to work well and have great capability, but as far as I know, it is only available in Japan. Kawada was also showing its HRP-2 and HRP-3.
• Universal Robots was showing applications with some local partners using Robotiq Grippers. Notice that the UR product line remains untouched with the UR3, UR5 and UR10.
• Rethink Robotics was showing Sawyer with Sumimoto, a local partner in Japan.
• KUKA was focusing completely on LBR iiwa in their booth, showing assembly tasks.

As you can see, the offerings for collaborative robots are increasing rapidly. All the big robot manufacturers are coming up with not just one collaborative robot, but entire product lines of them. This could be a game changer for collaborative robotics, and help move it from a niche, innovative market space into the mainstream for manufacturing companies.

I wonder how the new ISO TS 15066 standard for collaborative robot safety will affect the growth and evolution of these new products? Will we all have to go back to the drawing board and adjust our product design, or have we built enough safety into collaborative robots already?

And lastly, how will these new product lines affect the market for end effectors? Pneumatics are not often seen on collaborative robots — will this stimulate demand for electric end effectors or other technologies?

https://youtu.be/EYu3–oYi10

More information on them in an upcoming post, stay tuned!

#iREX2015 Tweets

Researchers are developing an exoskeleton prototype that makes it easier to carry heavy loads. Image courtesy of Robo-Mate.

‘We’re getting sufficient power for sufficiently low weight,’ said Jan Veneman, a roboticist at Tecnalia Research and Innovation in Donostia, Spain.

Exoskeletons typically consist of a framework that covers a person’s limbs. The framework has joints alongside the person’s own skeletal joints, and these are powered by motors or springs to assist in strength or endurance.

Exoskeletons with motors are more flexible and more powerful, whereas those with springs can be made much lighter, according Konrad Stadler, an engineer at the Zurich University of Applied Sciences in Switzerland. Stadler is the technical project manager of the Robo-Mate project, and is working on an exoskeleton that could help manual workers. As part of the EU-funded project, he and his colleagues are designing different exoskeleton ‘modules’ to suit the different tasks carried out by manual workers.

In order to help stimulate innovation in manufacturing, the EU has teamed up with industry and academia in public, private partnerships such as the Partnership for Robotics in Europe and Factories of the Future.

Compression forces

‘Musculoskeletal injuries and disorders are caused by lower back compression forces, which result from the load carried and the worker’s own body weight, as well as the posture of the worker,’ says Stadler.

‘Lifting loads with outstretched arms will produce much larger compression forces in the lower back than lifting the same object close to the body.’

By slipping on a Robo-Mate exoskeleton, a worker trying to carry a heavy object with outstretched arms ought to feel more like he or she is carrying it close to the body. Stadler and colleagues are currently testing a prototype in the lab, and hope to be doing industrial tests next year.

Robo-Mate has made a prototype of an exoskeleton. Image courtesy of Robo-Mate.

But exoskeletons are not all about strength. Veneman is investigating whether exoskeletons can also be used to keep people stable as part of the EU-funded BALANCE project.

The human body normally keeps itself balanced with two mechanisms: altering the stiffness of joints, or moving hands and feet to more stable positions. ‘It is very subtle where you place your foot,’ he explained. ‘If you place it too far, or too close, you can easily lose balance.’

The aim of the BALANCE exoskeleton is to learn how the body does this naturally, and see whether the exoskeleton can assist the body to do the same thing. It is currently in the design stage, but Veneman says the prototype should offer minimal resistance to movement, so that it can follow accurately the movements of healthy and infirm people.

Give a push

Once the system understands the human body’s balancing mechanisms sufficiently well, it should be able to replicate it. In theory, you should be able to give someone wearing the BALANCE prototype a push, and the system will move the person’s feet to stop them falling over.

That will require a rather complex exoskeleton with multiple joints, and sensors that can detect instability.

A simpler exoskeleton, and one with a good chance of commercialisation in the near future, is the one being developed within the EU-funded AXO-SUIT project, which is part of the EU’s Active and Assistive Living programme to enhance quality of life for the elderly. The idea behind AXO-SUIT is to help elderly or infirm people perform basic everyday tasks, such as standing up, walking, reaching and grasping.

The technology behind AXO-SUIT is not dissimilar to that in an electric bike, says project lead coordinator Shaoping Bai of Aalborg University in Denmark. Rather than completely taking over, the motors assist action that the user is already trying to perform, such as bending down to water the plants and standing back up again.

AXO-SUIT is currently at the prototype stage. In about five years, however, Bai hopes that his group will have a commercial version costing between EUR 8 000 and EUR 10 000.

That may sound expensive, but Bai believes the exoskeleton could replace a lot of costly personal care for the elderly. ‘It would be helpful for them to maintain their quality of life while staying at home, rather than move to a care centre,’ he said.

More info

In this video lecture, IEEE Fellow Raja Chatila shares his views on why roboticists are duty-bound to educate the wider public on the state of advanced robotics , and also to understand the consequences of their own research and the potential commercialisation of it. 

Looking a specific examples of robots that have either already been or will soon be commercialized — including drones, UGVs in defence and security, robotic devices for rehabilitation and human augmentation, robot companions, robot personal assistants robots, sexual robots, and autonomous cars — Chatila explores the ethical, legal and societal issues of autonomous robot decisions, accountability, privacy and intrusion, dependance and isolation, cognitive and affective bonds, and the status of the robot in human society.

Credits: Institut des Systèmes Intelligents et de Robotique

IEEE Fellow Raja Chatila is Director of Research at the French National Center of Scientific Research (CNRS) and Director of the Institute of Intelligent Systems and Robotics (ISIR) at Pierre and Marie Curie University in Paris.

His research covers robot navigation and SLAM, motion planning and control, cognitive and control architectures, human-robot interaction, and robot learning. He works on robotics projects in the areas of service, field, aerial and space robotics. He is author of over 140 international publications on these topics.

Chatila R. IJARS Video Series: On the Ethics of Research in Robotics [online video]. International Journal of Advanced Robotic Systems, 2015, 12:V9. DOI: 10.5772/61995

Credit: ETH Zurich / Alessandro Della Bella

In 21 countries across the globe, hundreds of people are preparing for Cybathlon 2016, where cutting edge robotic assistive technologies will help people with disabilities to compete in a series of races. This summer the Cybathlon practice session took place at the Swiss Arena in Kloten so that the teams could test out the courses. Watch the trailer for the rehearsal games!

Cybathlon 2016 is organised by ETH Zurich and will showcase six disciplines:

• BCI Race Functional Electrical Stimulation (FES) Bike Race
• Powered Arm Prosthesis Race
• Powered Leg Prosthesis Race
• Powered Exoskeleton Race
• Powered Wheelchair Race

The finals will take place on October 8, 2016.

Find out more at www.cybathlon.com

If you liked this article, you may also be interested in:

• Exoskeleton with haptic sensors helps paralysed man to kick off World Cup
• Novel robots for gait and arm rehabilitation: Interview with Robert Riener
• Control strategies for active lower extremity prosthetics and orthotics
• Registration now open for Cybathlon 2016
• Noonee: The chairless Chair

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

As usual, safety was one of the main topics in every talk. Once again, everybody agrees: safety is about the application, and you must do a risk assessment. But how? … That’s the big fuzzy open question.

Millennials, manufacturing and collaborative robots

Henrik Christensen from Georgia Tech made the point again that young people entering the workforce have played more than three full years of video games and we should try to leverage this skill set. We should also check which type of console people use nowadays to play; it’s not only about the game controller.

Jim Lawton from Rethink Robotics also mentioned the disinterest of millennials in traditional manufacturing. Easy-to-use and productive robots have the potential to interest younger people with the maker mentality. They could even be interested enough to want to use robots to build stuff as a job.

Collaborative Robots – Vendor Roundup

RIA invited many robot suppliers to present their take on the technology, its applications and market.

Rethink Robotics – The Software Company

Jim Lawton, Technology Executive – Rethink Robotics

Jim Lawton surprised the audience by stating that he works for a software company that makes robots. He explained the usual impetus behind the cobot wave: demography, cost of labor, enabling technologies. Now that we (almost) have safe, cheap and easy-to-use robots, what is coming up? First, Jim underlined the low price or fast ROI of the collaborative robots. He also agrees with the fact that millennials are not interested in manual tasks, and that bringing robots to the manufacturing chain is stimulating for them. Robots such as Baxter and Sawyer work like people do: they can carry human sized parts and do pretty much what a human can do. Finally, he said that robots can adapt to variability through machine learning and that they can be repurposed to a different application quickly.

UR – The Robots of the People

Scott Mabie, General Manager – Americas – Universal Robots

I was wondering about the number of Universal Robots on the market, and Scott Mabie told us that more than 5,000 Universal Robots have been sold so far. It is quite impressive to see how fast collaborative robots are growing. Some major manufacturers such as Maytag even feature the robot in their commercial.

https://youtu.be/zr-BUV8ORTY

ABB – The Volvo of Collaborative Robots

Phil Crowther, Global Product Manager Small Robots – ABB Robotics

If a 3-point safety belt could exist in robotics, ABB would invent it. Crowther presented their rigorous (and complex) approach to safety, which they obviously take very seriously. I could finally play with YuMi in the exhibition area. The booth personnel seemed a little stressed by everybody touching the robot, as if something might go wrong. That is the joy of robots for non-experts! From the little that I saw, it worked perfectly well.

It was the first time I got a look at their clean, user-friendly interface to program the YuMi. Other large end-users who tried it also liked the idea of leveraging the existing know-how of the more powerful and proven ABB tools if they needed to do more complex tasks. Just like Volvo cars, it is a very nice piece of engineering designed with the human in mind. As a gripper manufacturer, I can really appreciate the tight combination of the technology in their arm, the compact packaging of power, air and even embedded vision in the end effector. Crowther also talked a bit about the future of robot teach pendants, where smart phones and cloud computing would take the place of industrial PCs and large pendants. We have an interesting article on this subject by the way.

FANUC – The Lord of Robots

Paul Santi, General Manager – Powertain Systems Group – FANUC America Corporation

FANUC has been a reigning industrial robotics company in North America for years, winning the battle for market share by leveraging solid technology and top system integrators. You can clearly see that the FANUC presentation was colored by their history with their most important customer base: the automotive industry.

I agree with Santi that some things don’t change. You still need reliability and productivity increases. If I take the point of view of most automakers, I also agree that you still need integration for interfacing with machines and controlling the process. Their high payload, full featured, integration heavy approach for their new CR-35iA makes complete sense for the automotive industry. It makes no sense for SMEs who don’t need all the features and can’t get a competitive ROI if they can’t do most of the integration themselves.

Precise Automation – The Underdog

Brian Carlisle, CEO – Precise Automation

I was quite impressed by Precise’s presentation and I really think that they have a great, underestimated potential in the market. One reason is probably because we don’t know them as well as the other players. Carlisle presented the general idea that they are used to making safe robots, and leveraging the intimate knowledge of the physics of robots and software. They now have a very interesting lineup of Linear, SCARA and 6-axis robots. With targeted applications for a low payload (<3 kg), their robots are well suited for parts handling, laboratory automation and other lightweight applications. We have recently featured them in our eBook on Collaborative Robots.

Collaborative Robot User Panel

RIA invited three different collaborative robot users to share their experience in choosing, implementing and leveraging collaborative robots.

GM – The Expert User

Marty Linn, Robotic Principal Engineer – General Motors

Since General Motors was an early adopter of industrial robots (let’s say they almost invented them), you would think that such company would have bunch of collaborative robots working alongside humans to cover all the difficult or redundant tasks. We’re not there yet. In fact, with applications requiring a big payload, it is hard to incorporate a robot with a 5 kg payload to do a heavy task. Another disadvantage of collaborative robots is their relatively small reach. With the new Fanuc CR-35iA, GM is getting one step closer to the introduction of collaborative robots in the automotive industry. They have been testing the CR-35iA for almost a year now in 2 different applications. The fact that the robot can carry heavy parts (up to 35 kg) and has a long reach makes the robot pretty interesting. Compared to SMEs, GM does not need the intuitive programming platform that most collaborative robots are promoting, as they have the in-house knowledge and the repeatability volume to program the robot with standard programming methods. This is why the Green Fanuc is great for them; they are a traditional user.

Glidewell Laboratories – The High Mix / Low Volume Process

Daniel Phee, Mechanical Design Engineer – Glidewell Laboratories

Glidewell Laboratories also shared their experience of using a collaborative robot. Glidewell is manufacturing custom dental crowns in California. With a production of 3,000 to 3,500 crowns per day, they decided to introduce Universal Robots into their process to do the machine tending. Using nine UR robot arms, the company is saving 150 human hours per week. Watch the video to have a better idea of their installation.

Sepcoe Erie – The Job Shop

Chris Pedersen, Operations Manager and Mechanical Engineer, Sepcoe Erie

Chris and his team installed a Universal Robots in a month and a half, got an ROI within 6 months, doubled capacity and reduced scrap by half. Moreover, they improved the workers morale by eliminating a station that was stressful, dirty and highly redundant. That is a great success story on how collaborative robots can increase the productivity of SMEs.

This panel included a wide spectrum of users that illustrated the differences between large and small companies with respect to collaborative robots. GM stuck to their preferred supplier to keep the same programming and interface to resolve ergonomic issues, and smaller companies deployed new platforms on their floors to enhance productivity. Between all these factors the deployment time should also be considered: a few months for the smaller companies; a few years for GM, which needed to do a lot of up front work. For all of them, one concern was employees’ perceptions of the new robots before installation. All agreed that after installation and training, the employees loved the robots that did redundant jobs. That is what collaborative robots are all about.

#ICRW2015 Tweets

Using a BCI the robot was able to find targets that the person could see but the robot could not (Photo: Iturrate et al., 2015).

Brain-Machine Interfaces (BMIs) — where brain waves captured by electrodes on the skin are used to control external devices such as a robotic prosthetic — are a promising tool for helping people who have lost motor control due to injury or illness. However, learning to operate a BMI can be very time consuming. In a paper published in Nature Scientific Reports, a group from CNBI, EPFL and NCCR Robotics show how their new feedback system can speed up the training process by detecting error messages from the brain and adapting accordingly.

One issue that bars the use of BMIs in everyday life for those with disabilities is the amount of time required to train users, who must learn to modulate their thought processes before their brain signals are clear enough to control an external machine. For example, to move a robotic prosthetic arm, a person must actively think about moving their arm — a thought process that uses significantly more brainpower than the subconscious thought required to move a natural arm. Furthermore, even with extensive training, users are often not able to perform complex movements.

It has been observed, however, that the brain emits very different waves when it experiences success at controlling a BMI than when it experiences failure. With this in mind, the research team developed a new feedback system that records error signals from the brain (called ‘error-related potentials’, or ErrPs) and uses these to evaluate whether or not the correct movement has been achieved. The system then adapts the movement until it finds the correct one, becoming more accurate the longer it is in use.

In order to determine the ErrP, twelve subjects were asked to watch a machine perform 350 separate movements, where the machine was programmed to make the wrong movement in 20% of cases. This step took an average of 25 minutes. After this first training stage, each subject performed three experiments where they attempted to locate a specific target using the robotic arm. As expected, the time taken to locate a target reduced as the experiment continued.

Three experiments showed that a robot improved its ability to find the position of a fixed point using error-related brain activity. (Iturrate et al. 2015)

https://youtu.be/jAtcVlTqxeA

This new approach finds obvious applicability in the field of neuroprosthesis, particularly for those with degenerative neurological conditions who find that their requirements change over time. The system also has the potential to automatically adapt itself without the need for retraining or reprogramming.

Reference

I. Iturrate, R. Chavarriaga, L. Montesano, J.  Minguez and J. del R. Millán, “Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control,” Nature Scientific Reports, vol. 5, Article number: 13893, 2015. doi:10.1038/srep13893

If you liked this article, you may also be interested in:

• DARPA’s HAPTIX project hopes to provide prosthetic hands with sense of touch
• Lifehand 2 prosthetic grips and senses like a real hand
• Control strategies for active lower extremity prosthetics and orthotics
• The gateway to advanced neuroprosthetics: Jessica Feldman talks BrainGate and BCI
• Bionic athletes compete in disciplines drawn from everyday life

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Many robots are capable of picking up a part if the part is presented to the robot at a particular location. However, if the part has shifted from the expected location, the robot may no longer be able to grasp it … and if the part is sufficiently distant from its expected location, the robot could – as it attempts to grasp it – bump into it, push it further away, and jam the material handling system. This can, in turn, trigger a system fault and shut down the whole line — all because the robot does not know where the boundary between success and failure lies.

Robots function well in large scale industrial applications when reliability is designed into the system. This is accomplished by designing specialized hardware and software, extensively testing the system to identify potential failure spots, and developing contingency plans to handle failures when they occur. When a robot fails at a task, it often requires human intervention to clear the fault and restart the process, and this can be expensive. As such, robots are not typically used to execute a task until extremely high levels of reliability can be achieved — this is why customized hardware and software costs are typically only justified when the production volume is sufficiently high and the tasks are repetitive (such as in automotive assembly lines).

If we want to be able to use robots in small production batch operations or for non-repetitive tasks, however, we will need to prevent major system failures from occurring in the first place. Imagine a robot that could estimate the likelihood of completing its task before it even begins, and that calls for a human to help it figure out the problem when it foresees failure. A human operator could then provide the robot with assistance with the portion of the task that it finds challenging (for example, determining the orientation of a part, or finding a grasping strategy), leaving the robot to execute the remainder of the task itself.

In most situations, human intervention — at the right time, before the problem occurs — is far less costly than recovering from a system shutdown.

My students have been building a robot, called RoboSAM (ROBOtic Smart Assistant for Manufacturing), to demonstrate this concept in bin picking applications, which challenge a robot’s ability to perceive the desired object in the environment and to successfully pick it up and deliver it in a known pose. If RoboSAM is not sure whether it can pick the desired part from a bin containing many different parts, then it calls a remotely located human operator for help. The “human-on-call” concept, as we call it, is fundamentally different from the human-in-the-loop approach, which requires the human operator to actively monitor the manufacturing cell and take control away from the robot when the robot is about to make a mistake. “Human-on-call” requires the robot to call the human operator when it decides that it needs help. 

This model allows a single remotely situated human operator to help multiple robots on an “as need” basis. It also enables robots to be deployed on more difficult tasks. For the foreseeable future, many tasks in small and medium manufacturing companies fall into this category, making the human-on-call concept the right economic model for deploying robots in these contexts.

People often ask what humans will do when robots become more widespread. In my opinion, humans will be needed to teach robots how to do different tasks and bail robots out when they are confused. The key will be to develop technologies that allow robots to ask for help when needed. Recent work in our lab is a step in that direction.

If you like this post, you may also be interested in:

• Factory-in-a-Day: EU FP7 invests €7.9M to make robotics affordable for SMEs
• Low-cost robots like Baxter, UR5 and UR10 successfully entering small and medium enterprises (SMEs)
• Takeoff in robotics will power the next productivity surge in manufacturing
• What’s hot in robotics? Four trends to watch
• ABB launches YuMi and acquires gomTec
• Team RBO from Berlin wins Amazon Picking Challenge convincingly
• People and Robots: UC’s new multidisciplinary CITRIS initiative wants humans in the loop
• National Science Foundation and federal partners award $31.5M to advance co-robots in US

Jibo may be a robot, but the last thing the team wants is for Jibo to sound like a robot. In these two video interviews, Jibo’s design team talks about how they selected Jibo’s voice, how that then manifests itself as the voice you’ll hear when you interact with Jibo, as well as the engineering challenges of Text-To-Speech (TTS) technology and how the team solved for them.

Blade Kotelly, VP of Design 

https://youtu.be/3xYlzFdLrAs

Sriddhar Raghavan and Roberto Pieraccini

https://youtu.be/OddZWgc0na8

Transcript included.

In this episode, Ron Vanderkley speaks with Dr. Eleanor Sandry of Curtin University about her new book Robots and Communication. In the interview, we explore human to animal communication and what we can learn from it; human to humanoid robots interaction; and human to non-humanoid robots interactions. Also, we discuss Western and Eastern perceptions of robotics.

Dr. Eleanor Sandry

Eleanor Sandry is a Lecturer in the Department of Internet Studies at Curtin University in Perth, Western Australia. Her research is focused on developing an ethical and pragmatic recognition of, and respect for, otherness and difference in communication. Much of her work explores communication theory and practice by analysing human-robot interactions in science fact, science fiction and creative art.

Links:

• Download mp3 (16.6 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Transcript

Ron Vanderkley: Good morning Eleanor, if I can first ask you to introduce yourself to The Podcast listeners.

Eleanor Sandry: Hi there, well I’m Eleanor Sandry. I’m a lecturer in Internet Studies at Curtin University in Western Australia. My research is about communication in general, not just on theinternet and I’m particularly interested in human-robot interactions so that we can understand communication more broadly.

Ron Vanderkley: What got my attention to your subject was an up and coming book that appeared at a conference listing, if you can briefly tell us a little bit about the work on this book?

Eleanor Sandry: Yeah, so the book is called Robots in Communication and it’s been published by Paul Grave in the Pivot Series, which means it’s available online and print on demand. It’s a short format book, 45,000 words. It’s shorter than your average monograph and that’s because I wanted it to be accessible to a range of different people. It’s pitched towards people who are interested in robots, people who are interested in communication – human communication theories – so it’s trying to cater for quite a large audience. What it does is look at human-robot interactions and it tries to unpick them in some detail from a number of different theoretical perspectives.

I’m particularly interested in broadening the size of communication beyond just the transmission of information and ideas of success that surround accurately transmitting it. I’m interested in non-verbal communication; the idea that it isn’t just about what we say, but it’s about us using facial expressions and also movements more generally. I found that robots were just agreat way of looking at a huge range of different types of communication because they are sometimes built to look as human-like as possible, but on other occasions are completely machine-like, and yet what you find is the people who interact with them often develop some kind of relation to them. Even to a Roomba and other iRobot floor cleaners. Just looking at that gives me a whole range of different forms of communication to consider and things I can say about communication in general.

Ron Vanderkley: You speak about this idea of humanoid robots and the non-robots as a point of contact. What actually steers you towards the humanoid? Is it the facial features, the gestures, themovements?

Eleanor Sandry: I think that most people are studying humanoid robots because of particular facial features and expressions, in some cases. But, in others, it’s to do with physical ability to move around in human-like environments. One ofthereasons the Atlas robot – which is about to compete in the darker challenges – is human-like is because it’s expected to go and work in environments that are tailored for humans, but where humans can no longer enter, for example because of some catastrophic disaster.

In fact, I am much more drawn to non-humanoid robots and I’m particularly interested in the way that people will still read pretty much anything that moves as somehow trying to communicate with them. I’m also conscious of the fact that people who are trying to work with robots do better if they do develop this kind of relationship with them. Part of what I have been writing in the book is actually about looking at how those relationships develop and seeing those, say, between humans in ordinance explosive disposal and their robots. Relationshipsbetween people and robots are not necessarily a bad thing or something that we should be worried about; it actually allows those teams to operate better in real world environments.

Although a lot of people say that a sociable robot needs to be able to make gestures, maybe to also have facial expressions; that depends on the structure of the robot. I think there is a lot to be learnt from human-animal interactions. The things that humans and animals are capable of doing together open up possibilities for lots of other forms, for robots to be seen as effective teammates, basically.

Ron Vanderkley: For instance, canine interactions, this kind of trust and understanding. That is a whole different facet and we are still coming to termswith how wecan do that in robotics. Are you picking that apart and trying to figure out how wecan do that?

Eleanor Sandry: Yeah, I’m particularly interested in the idea of the need to have, almost, a trust and respect relationship in order for the team to operate effectively together. In the human-canine relationship, there would always be an overarching power relation. The human is almost always in control of the overallsituation and is working with the dog to achieve a particular purpose, for instance tracking a scent trail. The same is true with humans and robots and, at the moment, say with ordinance disposal robots, broadly, the human is in control almost 100 per cent of the time: they’re still mainly radio controlled or controlledvia wire connection. There are some things that they can perform autonomously like righting themselves if they fall over and maybe also returning to base.

What I’ve been trying to unpick, I suppose, is the idea that if a robot is going to become more autonomous, it will be out of contact with its human controller. There needs to be a more complex relation between the human andthe machine in order for that team to operate well. Thehuman effectively has to relinquish control to the machine at the point of which it moves out of radio contact, they have to trust in the robot to do its job. All of these ideas – even the terminology of me saying to do its job – isproblematic, historically, in understanding human-animal relations. Because many people think that a dog doesn’tknow what its job is, or isn’t going to take responsibility, and yet people who train dogs will tell you that this is precisely the way that you need to think about the dog in order tounderstand what is happening in the team, because you don’t have the nasal senses that the dog does.

It’s actually important that you trust a dog to do its job, so when it appears to be moving off a trail, in fact its followinga path that you have no idea exists and it will bring you to the final point; will complete the task effectively. I think thereare possibilities of the same thing between human robot teams, where the robot has senses that a human doesn’t. You have to learn to trust in those senses and I think it’s an interesting way to think about these relations. I think it has practical possibilities in the robotics community, but I also think it tells me all sorts of interesting things about how humans and other things interact.

Ron Vanderkley: The idea that we could try to use speech recognition between robot andobject, that being the only means of communication in the animal or non-animal world, is not entirely true. It’s looking at gestures, our voice.Is this also something that we may not have looked at in the robotic world?

Eleanor Sandry: Well I think there is a tendency. It’s not so much that it hasn’t been looked at but that the final goal is seen as being able to create a robot that it’s possible to activate entirely with just your voice and that will understand human expressions and gestures perfectly. I think that what’s being missed is that the robot could understand things differently from us. It’s not actually necessary to embed the robot in the world ina human-like way. It can be embedded in its own way and understand us in particular ways, even if that’s using sensors on hands that help direct the robot. Actually looking at those as real possibilities, not a problem that needs to be overcome, has potential forincreasing the flexibility of the way robots work.

The other reason I’m particularly interested in that is because I hope to start writing more about critical disability studies. That has really opened my eyes to the fact that technologies that are currently being directed towards very human-like interfaces may actually make it harder for people with disabilities to interact with a robot that might otherwise be very helpful to them in living their everyday lives independently. Really looking at a whole range of different interfaces for machines like this is important; not just thinkingof trying to perfect this notion of the human-like interface. I just don’t want them to cut down possibilities I suppose.

When I talk to people sometimes about robots, I get two reactions: normally, I get the people who are really firmly interested in human-like forms and want to make sociable robots as human-like as possible, and then I sometimes get other people that say “we never make robotshuman-like.” I get this divergent reaction and, really, I’m somewhere in the middle, trying to say “well, can’t we think about this differently? Can’t we think about this froma communication perspective, where you start seeing all the different ways that people and things and animals can communicate?” Because you’re quite right, of course, dogs don’t just respond to voice, they respond to tone of voice. They also understand things in ways that we can’t really possibly fully appreciate because wejust don’t have the senses that a dog does.

Ron Vanderkley: The other aspect I was considering is that we are talking about robots actually understanding us. I spoke to a guy in the UK about, I think, a robot called Hector, where it’s entertainment. It’s a robot telling the general public what is going on, with hand gestures, eye movements, et cetera. It’s a teaching tool, almost. I found that very interesting. Is that also something that you’re looking at?

Eleanor Sandry: I suppose it’s not my primary area of interest but that doesn’t mean I don’t think it’s important. I think that sometimes when I talk it’s easy for me to get obsessed with all the non-humanoid robots and it makes it sound a bit like I’m actually saying that we shouldn’t have humanoid robots and I don’t think that’s correct at all. I’m sure there are going to be situations where a humanoid robot works much better and that is probably a very good example. If you have a robot that you want to actually present to people and you want, effectively, to present it in a human-like way, then that is what you’re aiming for. You’re aiming for something that inspires the audience to pay attentionto it because of its abilities – that may sometimes be very important.

I think that thinking that that’s the only way to build a robot – that can be captivating, or attract attention, or draw people into communication – is probably incorrect and there are other examples. Certainly robots in art, for example, where a totally non-humanoid machine completely captivates its audience through its movements and the things it does, the sound it makes. It’s a very different idea; it’s not to try to transmit information directly but it’s still building a kind of relation between the human andthe machine that may lead us to question the ways that things exist in the world, which I think is usually quite a good thing. Also when things are framed by particular tasks, so anything where people and the robot aretrying to complete a task together kind of frames the communication that takes place.

I think it offers many more possibilities for non-humanoid communication in that particularsituation because you know what you’re aiming to do. And yourcapability of reading non-verbal communication is probably much greater in that situation. With the example of the robot up on stage, often a humanoid robot is going to win the game there.

Ron Vanderkley: When you look at the history of, say, robot interaction targeted at children or things like the first robot dog, where children actually relate to that almost as a real dog, and treat it as a real animal, do you think that’s the precursor to a closer relationship with non biological entities?

Eleanor Sandry: Well, I think its one direction in which to go. I’m slightly concerned by the idea of trying to create robots that effectively replace a relationship that someone would have with something that already exists in the world. It’s one of the reasons that I’m not so interested in humanoid developments, and also in the very animal-like robot developments, because it may notbe a good thing to replace the relationships that people already have with living things with a robot equivalent. Children are actually a fascinating example of humans who are freed from the need to justify all the decisions they make about how they’re going to interact with something and so, yes, theyreact to things, like a robotic dog,strongly. Their understanding, if something goes wrong with it, that it needs to rest or sleep, shows just how strong their understanding of the robot is.

I think that’s one of the reasons why I’m interested in robots that appear more overtly machine-like, because they remind you that you’re in quite acomplex new relationship with something that’s different. It’s different froman animal, it’s different from a human; it is its own thing.For example, I looked at interactions between humans and Guy Hoffman’s robot, which is a robotic desk lamp, a lighting assistant, and they did experiments together. People’sinteractions during the interaction were very much as if it was somewhat alive, not necessarily like an animal or anything else, it was an alive desk lamp. But you could still switch it off; it was still a machine as well.

Humans are capable of holding both of those ideas. I think children may be less worried about that, so as a specific example they do show just how far those relations can go. I suppose I’m interested in how you can signify that those relationships are actually different, that it is still a machine. However, I don’t really think that robots should replace human-animal relations. I just read something recently about a different researcher in Australia who has written about how he thinks, in the future, humans will have robotic animals and that immediately made me think of Doandroids dream of electric sheep? The original book has a very strong section in the front that is about the ownership of robotic animals being all that was possible for most people. The richest people had a real animal. It seemed to be that future idea and I hope and pray that we are not going to go that way.

Ron Vanderkley: No. There’s a concern throughout the entireinternetaboutthe future of autonomous robots, AI being this Skynet-kind of world, by those who I think are quite smart people. It kind of sends the wrong signals. I just wanted to see your take on things.

Eleanor Sandry: Well, my take on it is artificial intelligence has a long way to go before it ever getsto that kind of level. I also think that the way that it’s being developed at the moment, it’s not necessarily ever going to be the same as a human-like intelligence. An artificial intelligence is going to be different almost certainly. Yes, I get frustrated at the hype around the technology, and you’re right, it’s very respected voices that are raising these kinds of issues,and in the near future as well. I simply don’t think it’s going to happen in the near future and I don’t actually think it’s going to happen the way that some people are suggesting it might either.

I would definitely want to draw back from that and point out just how far away artificial intelligence is from being at that level. I mean Google Drive. These cars don’t drive everywhere; they drive in one particular place, where it’s been mapped sufficiently for them to be able to operate. I don’t think that people are necessarily aware of that when it’s reported in the mainstream media.

Ron Vanderkley: Not at all, and humanoid robots actually cost an awful lot of money to build …

Eleanor Sandry: Huge amount.

Ron Vanderkley: I don’t even know if Sony is actually able to clear a real amount of money.

Eleanor Sandry: I doubt it very much. I mean, that robot has been in a series of developments, way over a decade now, and still has issues being placed into a normal physical environment and operating successfully. It has to be ina controlled environment. I think that the general public probably isn’t aware of that. The difference between seeinga robot acting in a laboratory, in a controlled setting, and actually being able to interact in the normal physical world that we live in is something completely different. They are streets apart.

I saw something about the darker challenge basically explaining that they would be very impressed if one of the Atlas robots, or any of the other bipedal robots, actually manages to make it through the course without falling down. I think that that’s a realistic understanding of what’s possible at the moment. The idea that robots with four legs might actually operate more easily in those types of environments is fair and should be perceived as well. It’s a very interesting question.

Ron Vanderkley: Absolutely, and just in a previous podcast I was talking to some guys about soft robotics. I mean that’s completely going …

Eleanor Sandry: Totally, a very fascinating area.

Ron Vanderkley: Yeah, completely, a different direction and its early days.

Eleanor Sandry: Yeah, very early days, but there is so much potential for soft robotics but also mixtures of soft and hard. I saw ages ago that, rather than using a hand-like gripper to pick things up, someone had developedsomething that could actually… it was almost like something filled with coffee.

Ron Vanderkley: Like a bean bag.

Eleanor Sandry: Like a bean bag yes, that could grip. Those sorts of innovative ideas that are thinking of totally different ways of solving the task, rather than trying to make something that has a human-like hand, seem to me to offer tremendous potential and so I’m fascinated by the idea of what a robot might end up looking like in the future. I’m kind of encouraged that maybe robots in the future won’t look like us and actually that would be a really good thing.

Ron Vanderkley: That’s another human concern. Someone once put it that if we try and design something that looks so close to a humanand almost mimics the face, there’s total distrust. Whereas if it looks like a robot there is more trust.

Eleanor Sandry: I think there definitely is potential. I mean whenever I write about robots I always end up having to utilize the uncanny valleyexample. I’m kind of quite frustrated with it now: that idea that, as a robot becomes more and more human-like,eventually it’s actually read as zombie-like and there is a strong drop off in the level of trust in the robot. Some people suggest that you could climb the valley on the other side and eventually end up with aperfect human-like machine. Then, as has been written about in many different science fiction books, you have the question of what is the difference between the robot and the human and …

Ron Vanderkley: Silicon versus Calvin.

Eleanor Sandry: I think, really, we’re nowhere near achieving that at the moment. Even the very, very human-like faces you can see pretty easily from their expressions and from other things about them. I don’t know why we want to aim for that. I think that we have …

Ron Vanderkley: Perfection.

Eleanor Sandry: Yes, but it seems to be a perfection that’s driven by a100 percent understanding of humans being the pinnacle of evolution and, therefore, that should be the pinnacle of robotic development. That just seems to me an unfortunate decision to make. There are many more options for robotics than trying to achieve human-like status. I think that there are many other ways that machines could be far more interesting, far more useful but also maybe even develop new relations with people that don’t have to be like something that already exists.

Ron Vanderkley: One of the ideas in modern day sci-fi, the cyborg…

Eleanor Sandry: Yes, the idea of augmenting human ability, of fixing people, which is a very hot topic in disability studies because,in general, there aremany people with disabilities who do not wish to be fixed.Again, it’s driven by this idea of the human being the current pinnacle of evolution. Let’s see howfar we can take it. I hesitate to mention this but, as Dona Harrower said many, many years ago, in 1985 actually, we are already cyborg. I’m effectively a cyborg: I wear contact lenses; that’s the only way I can exist in this world. I would not be alive without those developments – of glasses or contact lenses. But where people begin to decide to amputate limbs, in order to have blades, because that means they can ran faster, that’s another thing science fiction is written about. There are all sorts of – not just ethical – questions, but bigger questions around that.

Ron Vanderkley: You brought up sci-fi as a sounding board. Do you use that as a sounding board for some of your ideas or your thoughts?

Eleanor Sandry: Often I do, yes: in fact, in robots and communication. Science fiction to me is a really great place to look because it’s sometimes overlooked as being a kind of subgenre. For some people, for many years, it was not real literature and not something that was studied much in scholarly circles. But it provides a real sounding board and some of the best science fiction writers write about embedding technology in societiesin ways that allow you to really think about some of these questions ina more developed way,because someone has written a story about it. It seems to provide a reallygreat way of thinking through some of the questions that are very difficult to picture in your head, whereas science fiction provides you with those kinds of thought experiments that can be so important.

In my thesis – what I was doing for my PhD- I did write more about science fiction: the idea of culture and human-machine interactions in culture were to me very interesting because they push way into the far future I guess.

Ron Vanderkley: Coming back on the idea that technology is going to take over. Is that something that you think that’s going to be more and more of a problem, as sci-fi seems to dwell on the bad things and not the positives, or is that just good for entertainment?

Eleanor Sandry: Well, it’s definitely good for entertainment. It’s also a very Western perspective on robots andtechnology. Whereas in Japanese Manga, the whole idea of technology, and robots in particular, is far more positive. Also, it’s a great pity that we lost [Iain] Banks because that was someone writing about a utopian feature, problematic in lots of ways, but at least he offered some kind of positive vision of the way things could work. You’re right, in general, there will still often… they’re not exactly disaster movies are they? But they’re kind of …

Ron Vanderkley: The good guy always seems to win in the end.

Eleanor Sandry: In the end but there is usually a potential for total disaster in the middle, isn’t there?

Ron Vanderkley: Okay, so in closing, what do you think is going to happen in the field of robotics, well, in Australia?

Eleanor Sandry: Australia is interesting and I’m still trying to get to grips with what’s happening in different parts of Australia. I think that the level of funding here may be very different from other countries. There are interesting things going on and I really need to spend more time getting to grips with them because I keep on reading things about what people are doing in Melbourne, starting with robotic arms, the disabled, people with disabilities, and I’m interested that there is stuff going on in Sydney. I’m seeing people from the Center for Social Robotics there, and so I’m particularly interested in what they’re doing and the cost as well.

I think robotics will continue to develop here. There is probably quite astrong drive towards thinking of robots in terms of self-driving machines, mind technology. That’s always going to be a big thing in this country. There are definitely innovative thinkers in Australia who produce interesting solutions to different problems and I would like to carry on finding out what they’re doing and writing about it.

Ron Vanderkley: Okay and to cap that off, if I can firstly thank you for giving the time to chat. On behalf ofThe Podcast, thank you very much.

Eleanor Sandry: Well, thank you for having me. It’s been great to talk.

All audio interviews are transcribed and edited for clarity with great care, however, we cannot assume responsibility for their accuracy.

In this episode, Audrow Nash speaks to several robotics companies at the company showcase at RoboBusiness 2014, which took place in Boston, Massachusetts.

The following is an ordered list of the people interviewed.

1. Norm Domholt (1m 40s), CEO of NPC Robotics,  talks about their modular remote operate vehicle Black Dog.
2. Annemarie Laudanski (6m 20s), Lead Field Applications Engineer at Xsens, demos their inertial motion capture system.
3. Carl Vause (9m 14s), CEO of Soft Robotics, speaks about the capability of the soft robotic manipulators, which can be used to pick fruit, among other applications.
4. Konrad Fagertun (13m 56s), COO of NLinks, tells about their new robot which drills holes in ceilings – a laborious and difficult task for humans.
5. Asim Ikram (15m 51s), CEO of Neo Robotics, discusses the market for robots in the Gulf Cooperation Council region.
6. Harri Valpola (21m 40s), Head of Technology at Zen Robotics, talks about the companies waste sorting robotic system.
7. Dick Zhang (26m 15s), CEO of Identified Technology, speaks about their self-piloting aerial drone systems for automated energy, mining, and construction site monitoring, surveying, and mapping.

Links:

• Download mp3 (18.8 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Transcript

Norm Domholt (1m 40s), CEO of NPC Robotics:

Audrow Nash: Hi, can you introduce yourself and your company?

Norm Domholt: Yes, thank you. My name is Norm Domholt and I’m president of NPC Robotics. Today, at the show, we’re featuring and debuting our latest invention. We’ve been working on it for three years and it’s called the Black Dog.

Audrow Nash: Now, you have the Black Dog here. Can you describe it?

Norm Domholt: Black Dog is a remote control mobile platform. It’s designed modularly, so that it can be repaired in the field without probing with any sort of instruments and with the use of one particular tool – a quarter inch hex wrench. All the parts can be taken apart and replaced in minutes.

Audrow Nash: Can you tell me a bit more about how it’s modular?

Norm Domholt: We realized, through our experience with Battle Box, that the teams that seemed to win were the teams that designed their robots to be repaired quickly and without the need to take apart wiring.

It’s something that can be plugged in and replaced quickly. So, we decided that there should be around eight components in here that can be replaced with the use of this one tool and they should all be modular. There are no internal wires that you have to worry about, so everything unplugs and plugs together very quickly.

Audrow Nash:  So it’s a trunk-treaded robot that has two motors driving everything and it’s three feet long, a foot tall and two feet wide?

Norm Domholt: Yes, 42 inches long, 22 inches wide and 12 inches tall. It weighs 200 pounds and it goes four miles an hour. It’s got a lot of torque. Each motor is two-and-a-half horsepower – that’s a 24volt lithium battery.

Audrow Nash: Can you give me some examples of how it could be fitted with different components in the field?

Norm Domholt: Yes. Let’s say a SWAT team buys our robot power base; there are many different applications and mission-specific needs that they could have throughout a day. For example, they go to a house with something suspicious going on inside. They can put a telescoping camera, called RibbonLift, on the robot, within – say – ten seconds. They can then maneuver it to the building from a safe distance. If they hear shots fired and see there’s no need for the camera, they can pull the robot back within seconds, take their camera off and put a 300 pound bullet-proof shield on instead. Then, a SWAT person could operate this with a thumb throttle, while crouching behind the shield, and approach the house from a safe position. He can direct it to come back and hurl a teargas canister through the window. It could be adapted to any particular use very quickly.

Audrow Nash: Very good, and can you tell me at what stage of production your company is?

Norm Domholt: We’re debuting this today. We were hoping it would be a little further along and we could operate it. Some of the electronics are not completed yet; the mechanics are almost done and the modularity is complete. The technology that’s going in is already in place and, in another two months, we’ll be testing at facilities – breaking down doors, going through hallways, up and down stairs. We’ll be testing to see how far – wirelessly – this thing can go into a building.

Audrow Nash: Who would be interested in this product?

Norm Domholt: The same people who would be interested in a multi-purpose tool. Police can use it, the home-owner… Who would use a power-base? SWAT, the oil industry, hazardous waste…

Audrow Nash: So, lots of applications?

Norm Domholt: A lot of applications, yes.

Audrow Nash: Thank you.

Norm Domholt: You’re welcome, thank you.

Annemarie Laudanski (6m 20s), Lead Field Applications Engineer at Xsens:

Audrow Nash: Hi, can you introduce yourself and tell me about your company?

Annemarie Laudanski: Hi, my name is Annemarie Laudanski and I work for Xsens, we do animation promotion capture.

Audrow Nash: Can you tell me about what you have set up at this booth and describe it?

Annemarie Laudanski: Today we’re trying out our inertial motion capture system. It’s a full body- measure-capture system built of 17 inertial sensors. In each of these, there are 3-D accelerometers, gyroscopes and magnetometers. They’re used to measure the movement of arms and legs and send information wirelessly to the computer, where it’s translated into a bio-chemical model. So, we can measure exactly what someone is doing and portray that on our software.

Audrow Nash: At the booth you have a dancer?

Annemarie Laudanski: We do, yes. We hired Ashley. She’s the cheerleader for the Boston Celtics and she’s with us during the conference to showcase the use of our system.

Audrow Nash: You mentioned you were using exonorometers, gyroscopes and Megnatometers. So, it doesn’t require any external visual system to track them? You’re just taking the raw data from all of these sensors, running that through, and then feeding it to a model, to develop a person?

Annemarie Laudanski: That’s right. The exonorometers measure how fast each limb segment is moving, the gyroscopes measure the rate of turn of the limb and then we use the megnatometers to correct the drift in the other sensors. A fusion of these three sets of sensor data is sent wirelessly to the computer where it’s combined with our calibration procedure to build a human model. The calibration takes an average reading from the sensors, while the subject is standing still for five-seconds. This is a known position for taking sensor data, that then informs each of the trackers, so that the model is built correctly.

Audrow Nash: I see. As the dancer moves, it’s happening in real time.

Annemarie Laudanski: Yes, all of it is done in real time so all of the sensors are sending data at 30 bites per second to the computer and everything is shown with a latency of, at most, ten milliseconds, which is mostly due to the Microsoft operating system we are working with.

Audrow Nash: What are some applications?

Annemarie Laudanski: The applications are really wide for this type of technology. They go from entertainment purposes, where people are using it for movie and game design, to movement science, where it’s used for sport, rehabilitation or ergonomics and also human-robot interaction. So, you can even have someone in the system and driving a robot, or building up a database of human movements that can be used to program one and allow it to move like a human would.

Audrow Nash: Thank you.

Annemarie Laudanski: Thank you.

Carl Vause (9m 14s), CEO of Soft Robotics:

Audrow Nash: Hi, can you introduce yourself and the company?

Carl Vause: I’m Carl Vause and this is Soft Robotics Incorporated. We’re a Boston-based company that got spun out as a research group at Harvard about a year ago. We’re taking soft robotics technology and making it real, producing commercial products to solve new problems across a variety of markets.

Audrow Nash: Can you tell me about what you’ve displayed at the booth?

Carl Vause: At the booth today we have two soft hands: A very large hand for handling pumpkins, shoe bags and objects of very weighty geometry, and a small-handed, or short, robotic arm. What we wanted to show here is that, currently, there are a lot of problems with robotic manipulation. Manipulators don’t do well with soft, curvy, easily damaged objects and they don’t do well in unstructured gripping scenarios.

What we have solved using soft actuators is the ability to pick up anything with this arm, from a prickly cactus, which you would never pick up with your bare hand, to a racket ball or a cupcake. We want to show that, with almost no control system, building a very clever, soft actuator can solve a problem that has been vexing robotics: How to develop a gripper that is as adaptable and conformal as a human hand. People are spending millions of dollars to solve this problem using traditional hard robotics.

Audrow Nash: Can you describe the hand and the smaller robotic arm?

Carl Vause: One is a four-finger hand. It’s based on proprietary actuator geometry. We use vacuum to curl the fingers out of the way and they pressurize to create a grasp. We have a three-finger gripper as well. It actually has fingernails, so it can access a low profile object like a circuit board.

Audrow Nash: Can you tell me some potential applications of these soft hands? I know there are a lot.

Carl Vause: A lot, but how do you interface a robot with something that has an unknown shape? Or when the position is unknown? Or something easily damaged? Agricultural processing is interesting. Picking lettuces heads, for example. There’s a danger of introducing E. coli into the food chain, so washing and cleaning lettuce. Robotics companies and the US Food and Drug Administration want to automate that task. But robots today can’t handle lettuce, because their heads vary in size, shape and weight, and they are delicate objects. So, one of the things that we really wanted to do with our soft robotic technology is unlock that market for automation, and automate data handling.

They can even work in wet conditions; there are no electrical connections here. You can sterilize them and wash them down with bleach at the end of the day. It really does bring automation into the agricultural process. It makes it a reality.

There’s also a light-weight version for working side-by-side with a human collaborator. In a lab, for example where you need to be careful what you touch. A manipulator can pick up a range of lab instruments, or tools, and hand them safely to you. It has the credentials of collaboration.

Audrow Nash: The ones that you have on display are on open feedback loops. As in, they don’t sense what they’re picking up, they just actuate. You’ve talked a little bit about embedding sensors into the fingers in the hands. Can you tell me more about that?

Carl Vause: We can easily embed sensors to give an object grasp detection. We can embed it in our proprietary manufacturing process, in simple off-the-shelf sensors and feed them back into something very basic. If you want it very simple, take an Arduino and provide passive grasp detection. That’s very simple.

Audrow Nash: Thank you.

Carl Vause: Alright.

Konrad Fagertun (13m 56s), COO of NLinks:

Audrow Nash: Hi, can you introduce yourself and tell me about your company?

Konrad Fagertun: Yes, my name is Konrad Fagertun. I’m a co-founder of NLink Robotics. We’ve come all the way from Norway to Boston, to showcase the world’s first mobile drilling robot. We’re excited to be here at Robobusiness, showing what we’ve been working on for the last two years.

Audrow Nash: Can you describe the robot to me?

Konrad Fagertun: Yes. The challenge in the construction industry is hard labor – manual work – and the hardest of all is drilling holes in concrete ceilings. So, we made a mobile drilling robot. It comprises a mobile lift – that can drive around – and, on top of that, is a robotic arm holding a rotary hammer drill. It thrusts that up to the ceiling and can drill holes wherever you want. It uses lasers and cameras to position itself, so it can drill to a millimeter’s precision. This can be done hundreds of thousands of times, say for a large shopping mall or an office building. It’s repetitive work.

The robot is a little bit bigger than a man, and can reach several feet high, so you don’t have to get up on scaffolding or stairs to do the work; the operator can stand on the ground, safely at a distance. But, if he wants to get close, no problem, the robot is certified to work with humans, so it doesn’t matter if it crashes into your arm, it won’t hurt you and it’ll stop. It’s useful for electrical contractors, plumbers, mechanical engineers…

Audrow Nash: Thank you.

Konrad Fagertun: Thanks.

Asim Ikram (15m 51s), CEO of Neo Robotics:

Audrow Nash: Can you introduce yourself and your company?

Asim Ikram: Yes, my name is Asim Ikram and I’m the CEO of Neo Robotics in Dubai.

Audrow Nash: What do you guys do?

Asim Ikram: We’re a new start up in Dubai, not limited to robotics. In that region, they are not exposed to robots at the moment. We are starting at the lower level, to introduce smart apps and ICT, Information Communication Technologies. Then, we want to switch to robots, because the customers are the same.

Audrow Nash: So, basically you’re acting as a liaison between companies in the US and outside for bringing robots into Dubai?

Asim Ikram: Into the GCC region – the Gulf Cooperating Countries. There are six of them in the Middle East:  Saudi Arabia, Bahrain, Oman, Kuwait, Emirates and Qatar. They have a strong focus on technology, they’re teaching robotics and megatronics and have labs at almost every university. The industry will develop but, at the moment, there’s no pull from for robots.

Audrow Nash: What are some of the more popular technologies that are being brought to Dubai?

Asim Ikram: Not so much industrial robotics, but there are new production lines coming up that have industrial robots in the blueprint. Those lines should start operating in six months. The financing and planning is already done. Healthcare and education, that’s much further ahead. On the education side, there’s big competition between the private schools that are teaching things like systems programming. Now they’re also trying to get robots into the classrooms and teach with them. There’s a lot of movement there.

Audrow Nash: What robotics opportunities do you think there are in Dubai for engineers and entrepreneurs from the rest of the world?

Asim Ikram: There’s a lot of work in real-world drone applications. Already, highway police are using drones. They’ve also started making some national identity card deliveries with drones. Real applications of technology are moving fast. On the security side, I’ve seen a lot of civil defense robots for bomb disposal.

Audrow Nash: How did you decide to take this opportunity and form a company?

Asim Ikram: I’ve been working in this market for some time and I’ve been seeing a lot of pull in different areas. Healthcare and education are moving very fast and the industrial side will come up in the next one or two years. There are a few robotics service companies, but not many locally so I found a gap. It’s a good time to start something.

Harri Valpola(21m 40s), Head of Technology at Zen Robotics:

Audrow Nash: Can you introduce yourself and your company?

Harri Valpola: Yes, hello. I’m Harri Holopainen. I’m one of the founders of Zen Robotics Company in Helsinki, Finland. Currently, our main application is robots that have learnt to pick up waste. A little bit like WALL-E…

Audrow Nash: It looks a little different than WALL-E. Can you describe the robotic system you have?

Harri Valpola: Whereas WALL-E is cute, a small Hollywood robot, our robot is more like an large industrial scale model. It doesn’t look like a human; it’s not cute, but it works very efficiently. It’s powerful. It can lift and throw objects that weigh up to 20 kilos – 40 pounds – like they were nothing.

Audrow Nash: It looks like a conveyor belt and has robotic arms that move around and pick objects off of the conveyor belt and throw them to sort them. Am I correct?

Harri Valpola: Yes. It works in a similar way to human hands. All around the waist is a conveyor belt. The robot recognizes the objects and sorts them, exactly as a human would do, but faster. It’s also stronger and it doesn’t break.

Audrow Nash: What kind of sensors are you using to identify the material? What waste are you sorting now?

Harri Valpola: We are currently focused on construction and demolition waste. The sensors we are using are tailored for that particular waste. The most important material we’re sorting is  wood, followed by mineral-based parts and metal. We’re using sensors that can discriminate between these. They include metal detectors and spectrometers.

Audrow Nash: Where have you implemented the system and what sort of feedback are you getting from customers?

Harri Valpola: Currently, we have three customers in Europe; two in Finland and one in Holland. We’ve been developing this system for two years and now it’s working and customers are happy. We’re ready to scale up, now we have the product, so that other customers can start making use of this solution.

Audrow Nash: What’s the benefit of sorting the waste?

Harri Valpola: In Europe, at least, there are many companies that are floating in waste. Population densities are so high, there’s no place to put it. The only real option is to sort the waste. There are many valuable raw materials there. Sorting stones is not important but there are valuable metals, like copper and aluminum, and also wood. That’s particularly useful in Holland because they don’t have forests, and it can be recycled and used as a construction material.

Audrow Nash: Thank you.

Harri Valpola: Thank you.

Dick Zhang (26m 15s), CEO of Identified Technology:

Audrow Nash: Interviewer:  Can you introduce yourself and tell me about your company?

Dick Zhang: My name is Dick Zhang from Identified Technologies. We build and scale drones for commercial data capture. We work with construction and energy industry clients and help deliver data and insights. We get this information either by deploying robots on the ground, or by utilizing commercial planes and helicopters. It’s very expensive, it’s very dangerous and it’s very slow.

We deliver insights by collecting massive scale amounts of data on what has happened in the past, what’s going on now and what’s going to happen in the future, through predictive capabilities.

We collect the data via a three-part system: an autonomous drone, a dock station, that swaps batteries and uploads data, and a client dashboard that can be used to interact with and control the system from the cloud, from anywhere in real time.

We’ve been working with really exciting partners in energy and construction for the last six months. I run the business, look for exposure and develop partner and client relationships.

Audrow Nash: Can you tell me a bit about the demonstration – the drone, the quadcopter – you have on your booth?

Dick Zhang: The drone itself, everything inside, in terms of hardware, is really off-the-shelf components. Our dock station, that swaps batteries and outputs data, that’s all proprietary mechanical design and hardware. The software system is also proprietary. So a lot goes into flight control – safety checks, process checks, maintenance, things like that.

Audrow Nash: Swapping the batteries, is that automated?

Dick Zhang: Yes, that is automated. On the docking station, there’s a carrousel with six batteries that are always charged. It inserts another battery when the drone needs to fly.

To tell you a story, what inspired all of this was a talk with one of our clients. They run 5,000 job sites around the world. They bought an Internet tool but had no idea how they were going to train 1,500 people to use it. So I automated a system and made it self-operating and self-sufficient. We were able to employ 1,500 really rapidly across the country and across the world.

Audrow Nash: Is it totally self-sustaining?

Dick Zhang: Yes, right now all you need is a standard power plug. Future generations will evolve generators, solar panels and wind fans. But, apart from the source of power, everything happens on its own. It flies itself, captures data and swaps batteries.

Audrow Nash: What kind of navigation do you use on the drones?

Dick Zhang: Navigation in drone technology has become really sophisticated over the last couple of years. We have a combination of GPS and inertial sensors on board, so that, three or four hundred times every second, the drone is saying to itself: “I need to be in position XYZ. Right now I’m not there, but I’m close. Let me get there.” So, four hundred times every second, it’s making these corrections and adjustments so that it can fly along the path that the computer programmed.

Audrow Nash: Other than the IMU and the GPS, what other sensors do you use? Do you have a vision system for landing on the platform that swaps the batteries, and some other sensors?

Dick Zhang: Yes. Now we are working with cameras. We’ve done some early tests with our partners on infrared and gas effect. We do a lot of work in energy. As part of this start up growth, we’re going through a crafty process of crawl, walk and run. It’s the vision that enables the most capabilities, to help our clients right now, with the resources that we have. Now, to touch on how we land, that’s a huge problem. So, we built a proprietary vision system based on ultrasound. There’s a beacon and a dock and a transceiver, where you listen and triangulate.

Audrow Nash: Okay, so you use GPS to get it close enough…

Dick Zhang: Exactly.

Audrow Nash: Then the ultrasound takes over by localizing the sound somehow?

Dick Zhang: Yes, exactly.

Audrow Nash: Okay, thank you.

Dick Zhang: That was awesome. Thank you.

All audio interviews are transcribed and edited for clarity with great care, however, we cannot assume responsibility for their accuracy.

By: Vanessa Bleich
Next year’s Cybathlon will host people with physical disabilities equipped with advanced assistive devices including robotic technologies. The goal of this unique competition is to remove barriers between people with physical disabilities, researchers and the general public and to promote the development of assistive technologies that are useful for daily life. We talked to Robert Riener, the Cybathlon’s main organizer and Professor of Sensory-Motor Systems at ETH.

Robert Riener is the initiator and organizer of the Cybathlon. (Photo: ETH Zurich / Giulia Marthaler)

You are the initiator and main organizer of the Cybathlon. How did the idea for this event come about?
As a researcher, I’m always on the lookout for opportunities to present my research to the general public. A few years ago, I read an article in the newspaper about a man who climbed the stairs at the Willis Tower in Chicago using a motorised prosthetic leg. It inspired me to think about a similar event that could be held here in Switzerland – an event that would extend beyond a single race to include many other disciplines. Through talks with colleagues and employees, I gradually developed the idea further. I also thought increasingly about what exactly we wanted to communicate and achieve with this event.

What do you hope to achieve through the Cybathlon?
Our original idea, which was to present our research findings to a wider public, of course remains a goal that we continue to pursue. But we would also like to raise people’s awareness of the needs of people with physical disabilities and the obstacles they face in everyday life. Unfortunately, many people without disabilities know very little about the challenges faced today by people with physical disabilities. Alongside technical assistance, for example, accessibility is also extremely important. We believe this knowledge, in turn, can break down more barriers.

Besides overcoming reservations about the field, we want to motivate developers to create technologies that provide real support to people in daily life. That is why the races feature everyday obstacles, such as stairs, ramps and doors. We hope Cybathlon’s competitive nature will boost the development of powered assistive technology in the long term. It’s already having a positive impact: new student groups have been formed and research projects have emerged. Both students and researchers are working at full speed on their prototypes.

What are the main difficulties when developing electrically powered devices?
The weight-performance relationship is one of the greatest challenges. For example, if you want to build a wheelchair that climbs stairs, it must be wider for stability reasons. When combined with the additional mechanics and propulsion system, it’s automatically heavier. The same goes for the development of the exoskeleton. The integrated batteries must be very efficient to allow the wearer to move freely and independently throughout the day, but these batteries weigh a lot.

Cybathlon

The Cybathlon will take place on 8 October 2016 at the Swiss Arena in Kloten. People with varying degrees of disability, aided by cutting-edge assistive technology, will participate in the unique competition. A team is comprised by a technology provider and at least one pilot who directs the technological device. As a result, two medals await each winning team at the Cybathlon: one for the parathletes and another for the provider. The technological device that is deployed can either be a product that is already available on the market or a prototype from a research lab.

The competition features six different disciplines:

• Powered leg prosthesis race
• Powered arm prosthesis skills course
• Powered exoskeleton obstacle course
• Powered wheelchair race
• Functional electrical stimulation bike race
• Brain-computer interface virtual race

Robert Riener, head of ETH Zurich’s Sensory-Motor Systems Lab, is the initiator and main organizer of the Cybathlon.

Are there other challenges besides the power supply?
While the performance data for the powered leg prosthesis is already very good, the control is more problematic. The wearer’s intended movements must be correctly conveyed to the leg prosthesis. This means that the prosthesis should automatically detect what the person wants to do, be it standing up, sitting down or climbing stairs. This is less of a problem when it comes to conventional, passive prostheses because the wearer intuitively propels them by moving or swaying.

What is the current state of research in the area of powered assistive technology?
The technology is already quite advanced today in some areas. For example, it is now possible to run quicker with certain leg prostheses than a human with natural legs could. But such prostheses are only helpful for specific movements. You may be able to walk quickly, but not be able to sit down, walk up stairs, or drive a car. Most of the available aids have significant deficits with regards to everyday functions and potential users have therefore failed to embrace them. This is where the idea behind the Cybathlon comes in: systems need to be developed that provide help to people with disabilities where they need it the most – in daily life.

How does the Cybathlon differ from other competitions such as the Paralympics?
People with disabilities can participate and compete in both events. The Paralympics, however, is primarily a sporting competition. Technology remains in the background and the use of motorised wheelchairs or powered prostheses is not permitted. As a result, many people are unable to compete. At the Cybathlon, on the other hand, «pilots» with greater restrictions who primarily direct the technical aids – are also eligible to participate. Although people are at the centre of our event, technology also plays a relatively large role.

The Paralympics are focused on sporting disciplines. In addition to races honouring the athlete who reaches the finish line first, the Cybathlon competition also includes skills and obstacle courses featuring everyday tasks and barriers. It’s not only the time that counts; pilots also receives points for their skills. Just like in classic sporting events, competition drives both the pilots and the technology developers at the Cybathlon. They inspire each other to deliver their best performance.

What has the response been to the planned event up until now?
There’s been a great deal of interest in the event. We have received very positive feedback from the scientific community, the media and well-known Swiss organisations for the disabled. The disability associations share our view that the Cybathlon could contribute to the development of current devices. Some critics, however, argue that these sorts of efforts should be focused exclusively on making buildings and the environment barrier-free. But the world does have barriers and you can’t just make them disappear – there are hills, forest paths and so forth. Researchers are developing technologies so they no longer pose obstacles in the future.

• Novel robots for gait and arm rehabilitation: Interview with Robert Riener
• Introducing the Cybathlon
• Registration now open for Cybathlon 2016

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Image: MIT d’Arbeloff Laboratory

In this episode, Audrow Nash interviews Federico Parietti, a PhD candidate at the Massachusetts Institute of Technology, about his research on supernumerary robotic limbs that can be used in manufacturing and for rehabilitative purposes, among other uses.

The videos below demonstrate how supernumerary limbs can be used to assist in tasks. This research was done in the same lab that Federico works in.

Federico Parietti

Federico Parietti is currently a PhD candidate at the Massachusetts Institute of Technology, where his research focuses on the design and control of wearable robots and man-machine interfaces. Previously, Parietti was a Research Associate and Visiting Scholar at Carnegie Mellon University and an International Student at ETH Zurich, in Switzerland.

Links:

• Download mp3 (10.5 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Transcript

Audrow Nash: Hi, welcome to Robots Podcast. Can you introduce yourself?

Federico Parietti: I work on wearable robots at the Mechanical Engineering Department at MIT. I’m happy to be here with you today.

Audrow Nash: We are too. Can you tell me the goals and motivation behind your research?

Federico Parietti: I work on wearable robots – essentially robots that you can put on. These robots are built to give humans something new with respect to what they have in their natural bodies: they can make you stronger, can make you faster, or can provide you additional information, like for example, the Google Glass. What I’m working on right now is physical augmentation; by wearing my robots, a human can essentially become stronger, or avoid fatigue, or they can acquire skills that were not possible before.

Audrow Nash: Can you tell me what it looks like?

Federico Parietti: Essentially, it’s two additional arms. When you’re wearing the robot, you’re essentially wearing a type of backpack harness and you’re carrying around two additional arms that can help you do tasks, coordinate with your hands. It can also reach for the ground or reach for the environment and support you.

Audrow Nash: And also they’re located at the wearer’s hips?

Federico Parietti: Yes. And since as you point out these two additional arms are essentially springing out of your hips, they can act as legs, but they can also act as arms. They can reach where you can’t reach, even behind you almost like a tail, so there’s really no limit. That’s why we call them additional limbs, because there’s not really a particular definition.

Audrow Nash: So how is this different from exoskeletons, with the extra limbs?

Federico Parietti: That’s a very important point, because these limbs are independent. They can move on their own without having to copy the motion of the human. In an Ironman style exoskeleton, the robot has to follow the human, so that as you move the robot follows you because the robot is built as a suit.

In our case, the robot is wearable, but it’s also independent. So you can move, you can do whatever you want, and the robot is free to coordinate in an independent way. Some people say this is a “Doctor Octopus” kind of robot, because their arms are independent.

Audrow Nash: What kind of sensors and actuators are on the wearable system?

Federico Parietti: Each one of the two robotic limbs has three actuators. We have two rotation motors in the shoulder of the robot and they can pick the direction where the robot moves. And then the legs are telescopic so they stand and retract. This means that the robot can essentially reach any point in space, within a workspace that is essentially a sphere.

On the robot we have four sensors and position sensors so the robot knows where it is and what it’s touching and how strong it’s pushing. And then we also use sensors on the human, which is very important because we have to detect human motion, and sometimes also human muscle activity and brain activity. And we also have some experiments on gaze, so we can detect where the human is looking so the robot coordinate.

Audrow Nash: The arms are attached using a ball-and-socket method. Can you tell me a bit about that and the advantages of it?

Federico Parietti: The arms are attached to the base of the robot through a ball and socket – a joint that is a sphere. And it’s what we have essentially in our shoulder that allows us to rotate our shoulder around in all directions, because it’s like a sphere. The advantage is that whenever the robot is pushing on the ground, the forces go from the ground through the center of that sphere, and this means that you can sit on the robot arms without wasting energy. The robot stays there and you don’t really need the motors to lift you.

Audrow Nash: Can you give me some applications and situations where these supernumerary limbs might be used?

Federico Parietti: Our first scenario of application was aircraft assembly. We visited aircraft assembly factories and realized that for some tasks there were two or three workers collaborating for the same task, but only one of the two or three was doing the important task, for example drilling. The other two were just helping with boring and very tiring tasks like lifting parts or following the first worker with a vacuum cleaner. So we asked, why can’t a robot do that? Why can’t a wearable system do that and help the two workers?

Then we also realized that a lot of workers get tired because they have to work in uncomfortable positions. A robot like this can compensate for the weight of the user and make working much easier and less tiring. And then finally we also realized that there are a lot of people that, either because they’re rehabilitating or because they’re old, they need additional legs or additional arms in order to keep a comfortable lifestyle to avoid the risk of falling down. And so this robot can also help them.

Audrow Nash: So when they’re taking their step one leg is in front of the other and the weak directions are perpendicular or diagonal to the way they’re heading?

Federico Parietti: Yes, this is is a very interesting point. When we walk we are unstable and so we risk falling down, and falling is very dangerous for us because we are bipedal – it’s catastrophic essentially. The curious thing is that even in martial arts, like in judo for example, when you have both feet on the ground, if you push in the direction that is perpendicular to the line that joins your feet, you fall down. So a lot of martial arts techniques block your legs and then push you in a perpendicular direction so that you fall down easily.

That’s the weak point, but how can we use a robot to improve it, to actually remove that weakness? If we put the robot in the weak direction, then you can’t fall anymore because even if you slip in that direction, there is a robotic leg already in the ground there. So the idea is that when we walk, the robotic legs swing where our legs are not and so we’re always very stable.

Audrow Nash: So they swing similar to our arms being opposite to the foot that’s stepping forward?

Federico Parietti: Yes, it’s something similar to that essentially.

Audrow Nash: And they plant themselves on the ground in theory?

Federico Parietti: Actually for the purpose of balance assistance, they don’t even need contact with the ground; they just need to be in the right place close to the ground.

Audrow Nash: How does this research compare to similar work?

Federico Parietti: The fact that our limbs can be both arms and legs makes us both similar and different to a lot of robots, both robotic manipulators that are typically arms, and to exoskeletons that are typically legs.

Let’s talk first about the legs. There are a lot of exoskeletons right now that are used by paralyzed people to walk again so that they don’t have to use a wheelchair. These exoskeletons follow the legs, and they move the legs along a trajectory. They essentially do only one step after the other, but they can’t really keep balance so users still have to use their arms and crutches to balance. Our robot would be complementary with these exoskeletons, so that if you’re paralyzed, you can use a simple exoskeleton to move your legs and these additional “robotic crutches” to avoid falling down.

There are also robotic walkers that have been developed for elderly people, but these are essentially robots with wheels and so you can’t go up and down stairs, and they still require the elderly person to use their hands. In our case you don’t really need to use your hands, the robot just follows you.

Audrow Nash: What are some challenges of this research and what are some lessons learned from these challenges?

Federico Parietti: There are so many challenges. One of them is to make the robot comfortable. People have to wear this robot, so you have to make it as easy to wear and as light as possible. Weight might seem an easy thing to do, but if you need a robot that can lift you, you’re going to need some mass, and some pretty strong actuators. Right now, the goal of having the robot light, but still functional and powerful is really driving us towards exotic materials like carbon fiber or fiberglass. We can’t use metal anymore; it’s too heavy. That was one lesson learned: try to make it as light as possible, because that’s essential for users.

Another challenge is the interaction with the human, because we have independent arms – so just copying what the human is doing is not enough. We need to have some kind of independent source of information from the human. We have to find a way so that the human can control these legs without sacrificing anything. So for example, if we use our arms to control robotic arms, additional robotic arms, that would essentially miss the point, because we’d still be using our two arms, right?

How can we use human actions to extract independent signals so you can drive additional arms without limiting your own? For example, we’ve strapped an accelerometer to a person’s legs so that when they are just walking normally, we are detecting the frequency of the steps. The robot knows when they are stepping without them having to tell the robot, and the robot can just synchronize with their steps autonomously. The idea is that you can still do whatever you want with your arms and you can walk at any speed with your legs, but the robot synchronizes without you telling it how. Of course this is a simple periodic example, but we are interested in how we can we keep this a very intuitive interaction.

Audrow Nash: Once you have an intuitive interaction, do you think you’ll have the arms and legs swapping back and forth? Like being used as both, within the same system?

Federico Parietti: Yes, there’s really no limit. Once you have a good interface, you can do whatever you want. You can use the robot to pass you objects that you need, but if you’re using your hands you can’t really reach for them. You can use it for locomotion, for safety … We actually don’t know how many uses there are for additional arms, because that’s not in our nature; no one has ever had additional arms. I bet that if we provide a reliable human-machine interface people will just come up with new uses.

Audrow Nash: Going forward, what are your plans and goals?

Federico Parietti: It would be good to bring this robot out of the lab and into society, either for manufacturing applications and helping workers, or for rehabilitation, or assisting elderly people.

We already have a lot of patents either filed or about to be filed on these robots. It’s a pretty solid and unique technology because of the independence of the arms, and we think it’s very promising. This is a very good moment for start-ups and ventures, and this could be a good application.

Audrow Nash: What advice do you have for beginning researchers?

Federico Parietti: Do what you like. Refuse to do something that you don’t like, because it’s hard. This is the fourth product I’ve built in the last three years, so it’s a lot of work, it’s a lot of learning, there’s a lot of obstacles and sometimes nothing works. The only way to survive is if you love what you’re doing. So if you love it, there is really no problem, you can do whatever you want. I would say to researchers to pick the field that you really love, stick with that even if people tell you to go elsewhere, because doing what you like is essential to actually doing it well.

Audrow Nash: It’s interesting that you brought up how many iterations you’ve done with this. Can you talk a bit about that? The rapid turn-around time you’ve had?

Federico Parietti: I think it is a little bit extreme in our lab; no single product for me has ever lasted more than six months – and this is not because they break, it’s because we move on to the next thing. We review a product to test a new control idea, or to test a new design idea, and then we see if it works or if it doesn’t, and then we start immediately to build a new product that takes in account those lessons. That’s how we realized the robot that we have now, which is much, much lighter with respect to the first one. We realized, for example, that joint torque is not that important if the robot is heavy. It’s much better to have a slightly weaker robot but also lighter robot IF you use those torques in a smart way, like with the ball and socket joint for example. So right now we are talking about the latest product, but in six months we’ll have a new one.

Audrow Nash: Do you think all these iterations are beneficial?

Federico Parietti: Yes, it’s very beneficial. Sometimes we come up with applications that are new, and if I was stuck using my first product, I wouldn’t be able to use the robot for it because it was designed to do something else. All these iterations allow us to be very active and follow the hottest topic; we can incorporate the feedback that we receive at conferences very quickly.

Audrow Nash: Wrapping up, what do you believe is the future of robotics?

Federico Parietti: I see the future of robotics as divided in two huge areas: there are autonomous robots and there are wearable robots. The goal of autonomous robots is to be as intelligent as possible in order to act as a support or helper to the human, but an independent helper – a kind of robot butler. And then there are wearable robots, and their goal is not really to help as an external agent, but to make the wearer better. It’s essentially human augmentation, and that’s the field I’m working in.

The challenge for us is not really to make the robot smart or to give it artificial intelligence. Our challenge is to figure out how to make the robot as useful as possible to a human. It’s not an easy challenge, because we don’t currently have sensors that tell us what the human is thinking, or what the user wants to do. Our robot must be very smart in understanding what the user wants to do and doing it in the best possible way. Essentially we’re in the process of building extensions to the human body, so we need to understand both the mechanical and the neural part of the process, and that’s the challenge that I think is unique to wearable robots.

Audrow Nash: Thank you.

Federico Parietti: You’re welcome.

All audio interviews are transcribed and edited for clarity with great care, however, we cannot assume responsibility for their accuracy.