Last week I had the pleasure of debating the question “does AI pose a threat to society?” with friends and colleagues Christian List, Maja Pantic and Samantha Payne. The event was organised by the British Academy and brilliantly chaired by the Royal Society’s director of science policy Claire Craig. Here follows my opening statement:

My opening statement

One Friday afternoon in 2009 I was called by a science journalist at, I recall, the Sunday Times. He asked me if I knew that there was to be a meeting of the AAAI to discuss robot ethics. I said no I don’t know of this meeting. He then asked “are you surprised they are meeting to discuss robot ethics” and my answer was no. We talked some more and agreed it was actually a rather dull story: a case of scientists behaving responsibly. I really didn’t expect the story to appear but checked the Sunday paper anyway, and there in the science section was the headline Scientists fear revolt of killer robots. (I then spent the next couple of days on the radio explaining that no, scientists do not fear a revolt of killer robots.)

So, fears of future super intelligence – robots taking over the world – are greatly exaggerated: the threat of an out-of-control super intelligence is a fantasy – interesting for a pub conversation perhaps. It’s true we should be careful and innovate responsibly, but that’s equally true for any new area of science and technology. The benefits of robotics and AI are so significant, the potential so great, that we should be optimistic rather than fearful. Of course robots and intelligent systems must be engineered to very high standards of safety for exactly the same reasons that we need our washing machines, cars and airplanes to be safe. If robots are not safe people will not trust them. To reach it’s full potential what robotics and AI needs is a dose of good old fashioned (and rather dull) safety engineering.

In 2011 I was invited to join a British Standards Institute working group on robot ethics, which drafted a new standard BS 8611 Guide to the ethical design of robots and robotic systems, published in April 2016. I believe this to be the world’s first standard on ethical robots.

Also in 2016 the very well regarded IEEE standards association – the same organization that gave us WiFi – launched a Global initiative on Ethical Considerations in AI and Autonomous Systems. The purpose of this Initiative is to ensure every technologist is educated and empowered to prioritize ethical considerations in the design and development of autonomous and intelligent systems; in a nutshell, to ensure ethics are baked in. In December we published Ethically Aligned Design: A Vision for Prioritizing Human Well Being with AI and Autonomous Systems. Within that initiative I’m also leading a new standard on transparency in autonomous systems, based on the simple principle that it should always be possible to find out why an AI or robot made a particular decision.

We need to agree ethical principles, because they are needed to underpin standards – ways of assessing and mitigating the ethical risks of robotics and AI. But standards needs teeth and in turn underpin regulation. Why do we need regulation? Think of passenger airplanes; the reason we trust them is because it’s a highly regulated industry with an amazing safety record, and robust, transparent processes of air accident investigation when things do go wrong. Take one example of a robot that we read a lot about in the news – the Driverless Car. I think there’s a strong case for a driverless car equivalent of the CAA, with a driverless car accident investigation branch. Without this it’s hard to see how driverless car technology will win public trust.

Does AI pose a threat to society? No. But we do need to worry about the down to earth questions of present day rather unintelligent AIs; the ones that are deciding our loan applications, piloting our driverless cars or controlling our central heating. Are those AIs respecting our rights, freedoms and privacy? Are they safe? When AIs make bad decisions, can we find out why? And I worry too about the wider societal and economic impacts of AI. I worry about jobs of course, but actually I think there is a bigger question: how can we ensure that the wealth created by robotics and AI is shared by all in society?

Thank you.

The featured image of this article was used to advertise the BA’s series of events on the theme Robotics, AI and Society. The reason I draw your attention to it here is that one of the many interesting questions to the panel was about the way that AI tends to be visualised in the media. This kind of human face coalescing (or perhaps emerging) from the atomic parts of the AI seems to have become a trope for AI. Is it a helpful visualisation of the human face of AI, or does it mislead to an impression that AI has human characteristics?

Credit: Ex Machina (2015).

Seventy-five years ago, the celebrated science fiction writer Isaac Asimov published a short story called Runaround. Set on Mercury, it features a sophisticated robot nicknamed Speedy that has been ordered to gather some of the chemical selenium for two human space adventurers.

Speedy gets near the selenium, but a toxic gas threatens to destroy the robot. When it retreats from the gas to save itself, the threat recedes and it feels obliged to go back for the selenium. It is left going round in circles.

Speedy is caught in a conflict between two of the laws that robots in Asimov’s stories follow as their core ethical programming: always obey human instructions and always protect your existence (as long as it doesn’t result in human injury).

Speedy’s custodians resolve the robot’s conflict by exploiting a third, overriding law: never let a human being come to any harm. They place themselves in harm’s way and tell Speedy if he does not get the selenium they will die. This information straightens out Speedy’s priorities, and the adventurers live to interfere with other planets.

Today, Asimov’s three laws of robotics are much better known than Runaround, the story in which they first appeared. The laws seem a natural response to the idea that robots will one day be commonplace and need internal programming to prevent them from hurting people.

Yet, while Asimov’s laws are organised around the moral value of preventing harm to humans, they are not easy to interpret. We need to stop viewing them as an adequate ethical basis for robotic interactions with people.

Part of the reason Asimov’s laws seem plausible is that fears about the harm robots could do to humans are not groundless. There have already been fatalities in the US due to malfunctioning autonomous cars.

Again, the capacities of artificial intelligence to adjust its routines to the things and people they interact with, makes some of its behaviour unpredictable. For example, machines that are capable of interacting with each other in a co-ordinated way can produce “swarm” behaviour. Swarm behaviour can be unpredictable because it can depend on adapting to random events.

So Asimov was right to worry about unexpected robot behaviour. But when we look more closely at how robots actually work and the tasks they are designed for, we find that Asimov’s laws fail to apply clearly. Take military drones. These are robots directed by humans to kill other humans. The whole idea of military drones seems to violate Asimov’s first law, which prohibits robot injury to humans.

But if a robot is being directed by a human controller to save the lives of its co-citzens by killing other attacking humans, it is both following and not following the first law. Nor is it clear if the drone is responsible when someone is killed in these circumstances. Perhaps the human controller of the drone is responsible. But a human cannot break Asimov’s laws, which are exclusively directed at robots.

Meanwhile, it may be that armies equipped with drones will vastly reduce the amount of human life lost overall. Not only is it better to use robots rather than humans as cannon fodder, but there is arguably nothing wrong with destroying robots in war, since they have no lives to lose and no personality or personal plans to sacrifice.

We need the freedom to harm ourselves

At the other end of the scale you have robots designed to provide social care to humans. At their most sophisticated, they act as companions, moving alongside their users as they fetch and carry, issue reminders about appointments and medication, and send out alarms if certain kinds of emergencies occur. Asimov’s laws are an appropriate ethics if keeping an elderly person safe is the robot’s main goal.

But often robotics fits into a range of “assistive” technology that helps elderly adults to be independent. Here the goal is to enable the elderly to prolong the period during which they can act on their own choices and lead their own lives, like any other competent adult. If this is the overriding purpose of a care robot, then, within limits, it needs to accommodate risk-taking, including its owner putting itself in harm’s way.

For example, falling is an everyday hazard of life after the age of 65. An elderly person can rationally judge that living with the after-effects of a fall is better than a regime in which they people are heavily monitored and insulated from all danger. But a robot that deferred to this judgement by not intefering when its user wants to take a steep walk would be violating the law that saved Asimov’s adventurers on Mercury.

So long as only minor injury results from a competent adult decision, it should be respected by everyone, robots and family alike, even when it is taken by an elderly person. Respect in these circumstances means not preventing the adult decision-maker from acting or informing others about their actions.

As robots become a bigger part of our society, we will undoubtedly need rules to govern how they operate. But Asimov’s laws either fail to recognise that robots get caught between human acts and cannot always avoid human injury. And that sometimes allowing humans to injure themselves is a way of respecting human autonomy.

An Air Force airman at the control module of an MQ-9 Reaper. Credit: John Bainter/USAF

Machine ethics offers an alternative solution for artificial intelligence (AI) safety governance. In order to mitigate risks in human-robot interactions, robots will have to comply with humanity’s ethical and legal norms, once they’ve merged into our daily life with highly autonomous capability. In terms of technical challenges, there are still many open questions in machine ethics. For example, what is deontic logic and how can it be used for improving AI safety? How do we fashion the knowledge representation for ethical robots? What about the adaptiveness of the ethical governance? These are all significant questions for us to investigate.

In this interview, we invite Prof. Ronald C. Arkin to share his insights on robot ethics, with a focus on its technical aspects.

Date: February 1st 2017

Interviewee: Prof. Ronald C. Arkin, Regents’ Professor and Associate Dean in the College of Computing at the Georgia Institute of Technology and Executive Committee Member of IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous Systems

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan).

Ron Arkin

Q1 WENG: Thank you for agreeing to an interview with us. Can you tell us a little about your background?

ARKIN: I have been a roboticist for 35 years, so it’s hard to tell [only] a little, but I will do my best. Currently I am a Regents’ Professor and Associate Dean in the College of Computing at the Georgia Institute of Technology. I got my Ph.D. from the University of Massachusetts in 1987, and have been working on robotics since that time. I actually have degrees in chemistry as well—a bachelor’s and a master’s in organic chemistry. So, I have a rather diverse background and followed a non-traditional path compared to most computer scientists.

Q2 WENG: What is your definition of “Ethical Robots”?

ARKIN: That’s an interesting question! I have been asked that one before. I would say an ethical robot is a robotic system that is able to comply with the ethical and social norms of human beings—in some cases in accordance with international law, in other cases in accordance with national or local law. In addition, also being able to comport with humans in a natural manner, obey, and adhere to the same principles as an ethical person.

Q3 WENG: Could you please tell us why you are particularly interested in “Embedded Ethics”? Why is this concept so important?

ARKIN: That’s why you need to read the preface of my book [Governing Lethal Behaviour in Autonomous Robots]. It’s a rather long story. But let me try to give you a short version. Again, it dates back to a meeting—the first international symposium in Roboethics in 2004—where I was exposed to international conventions, e.g., the Geneva convention. It also led me to understand the Vatican’s position on how they viewed appropriate ethical interactions. Perhaps even more important than that one experience was the fact that we roboticists that have been in the field for some time finally started to have success, and we were seeing our technology move out of the laboratory community and into the real world. As such, it wasn’t hard to see how this could go in a variety of different directions.

Some robotics technology could benefit humanity, some not so much. At that point, one has to ask, “who is responsible for this?” So I assumed that, in part, it was myself. I believe that I have to take the responsibility for the technology that I helped to create. The issue is that we, as roboticists, had actually succeeded in many ways in terms of creating technology that we only envisioned when we first started that is going to have potentially profound impact on our society and the world as a whole. As such, we should be proactive in the ways in which we move forward [in order] to get a better understanding of the consequences of the work, and the social implications of the technology we are designing.

https://www.youtube.com/watch?v=jOJwkvihYtc

Q4 WENG: The IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous Systems has published their report “Ethically Aligned Design” in December 2016. As a member of the IEEE Global Initiative, could you please tell us about your role?

ARKIN: I serve a variety of roles. First, I am on the executive committee of that particular initiative, so I work at a high-level with about 30 other people or so. We have committees that large. I also co-chair the committee in Affective Computing. Joanna Bryson is my co-chair on that.

Affective Computing is concerned with the issues surrounding the illusion of emotions we can potentially create in these artifacts, and the ways human beings can bond to them. We are exploring a variety of different things, including robot intimacy, or the notion of applying robotic nudges and moving people in certain directions subtly, and in so doing to change the way people act and behave using these technologies, while trying to understand what may or may not be appropriate guidelines based on ethical considerations.

It’s about 15-20 people in that committee. We are one of the newer committees and we only have a couple of pages in the initial design report. But actually, in the June meeting in Austin, for the next release of the document 2.0, we will have as much as the other committees do. Finally, I am also a member of the humanitarian committee. I think it is called economic and humanitarian. I don’t know the exact title, but I am interested in the humanitarian side, and potential uses of robotics for humanitarian applications.

Q5 WENG: What is “Deontic Logic”? Why does it matter to machine ethics?

ARKIN: It may or may not matter. It’s just one approach to a particular problem. Deontic logic is a logic of prohibitions, obligations and permissions. It is one type of framework that is more transparent in order to get a better understanding of the way in which systems operate in the real world. Then it explains its reasoning to a human being, as compared to say deep learning or other forms of neural networks which may indeed provide good results but provide no explanatory capability to humans after the decision has been made.

The notion of prohibitions basically tells the robot some things it cannot do. The notion of permissions describe the things it is potentially allowed to do. And the notion of obligations says things that the robot must do under specific circumstances. It provides a nice framework of being able to explore morality. Others have already explored this, like Bringsjord at RPI who has applied deontic logic in this framework. We are certainly not the first in that particular space. Much of the work we have done has dealt with first-order predicate logic as a basis. It doesn’t have to be deep because it is so explicit in many cases.

Q6 WENG: Your hybrid embedded ethics framework aims for governing lethal behavior of military robots. Can we apply the same technical framework to service robots? For example, personal care robots might have some non-lethal, but still risky autonomous behaviors that need to be regulated by machine ethics.

ARKIN: I believe so. Indeed the whole point of the architecture is trying to design the systems to make them generalizable and not ad hoc for particular sets of circumstances. We have yet to look at service robotics in a traditional sense, but we have already developed something we refer to as the intervening Ethical Governor, which we are using for health care applications—particularly for assisting the management of two human beings in a particular relationship, like a patient and their care giver.

A robot hands a medication bottle to a person. Photo credit: Keith Bujak. Source: Georgia Tech News Center

We are doing work for the National Science Foundation, joint with Tufts University, looking at how in early stage Parkinson’s there is a condition called the facial masking. The loss of muscle movements in the face of the patient can produce stigmatization, and the patient-care giver relationship deteriorates due to the loss of non-verbal communication. We just had a paper accepted at ICRA describing the architecture that we have and how it has been modified from the military version. We generalized it from the military constraint-based version to a rule-based version.

But also, instead of using the international humanitarian law, we are using occupational therapy guides as a basis for generating the responses when two human beings in the relationship step outside of acceptable bounds. So the acceptable bounds now are not defined by international humanitarian law, but they are defined by occupational therapy—such as when a patient or a care giver start to yell at each other, or one starts to leave the room. The hard part is recognizing when dignity is being abridged, particularly in the patient. So we model the moral emotions of empathy, shame and embarrassment. We tried to ensure that the empathy level in the care giver remains within acceptable bounds and that the patient shame and embarrassment doesn’t rise to unacceptable levels. So the intervening Ethical Governor intervenes when a really inappropriate event occurs. It then provides strong corrective action by the robot to encourage people to get along better. The moral adaptor component which we used in the ethical architecture—initially with the guilt model—has now been extended to other moral emotions like shame, embarrassment, guilt and empathy [in order to] try and provide more subtle non-verbal queues [to help the] robot steer the relationship in the correct way before it get out of hand.

Q7 WENG: Regarding your research on machine ethics, what kind of indicators you use to evaluate the final outputs of your project?

ARKIN: In the first one, we were designing a prototype to prove the concept of the architecture. We never make the claim that this is the best way to do it. It is just one way to do it. Then we demonstrate that particular capability through the use of simulations and other aspects. The system was not intended to be fielded, but hopefully others could take those core ideas and potentially put them to use in the real world if they so choose. For the intervening governor, we’re running human subject tests right now—not with Parkinson’s patients yet, but with the elderly population, 50 or older. We’re trying to get a better understanding of the correct sets of rules, and to correct the interventions in a normative fashion, so we can ultimately move on in the final year of the grant to work with actual Parkinson’s patients. Again, using standard HRI technics, such as recording the kinds of reactions they have and parsing the data from the sessions and using pre- and post- evaluation questionnaires.

Q8 WENG: Let’s talk about your proposed machine ethics framework. You had mentioned that it includes four main parts: (1) Ethical Governor (2) Ethical Behavior Control (3) Ethical Adaptor (4) Responsibility Advisor. First of all, what is the major difference between Ethical Governor and Ethical Behavior Control? Is it the difference between deliberative and reactive control, or something else?

ARKIN: The Ethical Governor is basically a standalone component to the architecture which is a bolt on component at the end of the existing system. In the case of military scenario: asking for permission—should it fire or not. This is an intervening stage which either grants or denies permission through the validation of permission-to-fire variable. So it is straightforward thing to be able to do. The Behavior Ethical Control is a different kind of thing as it works purely with the reactive component. What it tries to do is, for each of the individual behaviors themselves, to respond whether or not to engage a target. This is prior to dealing with the Ethical Governor making the decision; trying to make the decision correctly before it gets to the Ethical Governor, where individual behaviors are given ethical guidelines to try and ensure that the behavior is correct itself. Each of the behaviors is provided with appropriate ethical constraints. So the difference is: one is located within the reactive component in each of the individual behaviors that makes the activity of the architecture, and the Ethical Governor is at the end of the architectural chain.

Q9 WENG: What is the importance of Ethical Adaptor? What kinds of machine learning skill that you chose to improve its adaptiveness?

ARKIN: Machine learning is something we haven’t generally explored in this particular context. Especially the military context, we kind of engineer out the adaptiveness and machine learning for fear that the robot would be granted the authority to start to generalize the targets in the field, which we feel is inappropriate. Learning after action is okay. but that’s a different story. The importance of the Ethical Adaptor, and its cognitive models of guilt that we use, is the fact that even the actions the agent takes may inadvertently end up unethical. This is referred to as a “moral luck”. It could be good luck or bad luck. But if it is bad luck, you want to be able to ensure proactively that the bad action doesn’t occur again. That’s actually the functions of cognitive guilt for people as well to reduce the likelihood of reoccurrence of a behavior that resulted in bad actions. By giving this to a robot we hope that if it makes a mistake it will not repeat it. The probability of that behavior reoccurring would be reduced.

The best example is the case of battle damage assessment for a military scenario. For example, if a robot does its assessment and it looks like a released weapon wouldn’t create too much collateral damage, within acceptable limits, then it will release the weapon. But after it does it, the battle damage action is actually more severe then, it initially appeared to be. Under those circumstances, we want the robot in the field to reduce the probability of releasing the weapon again under similar circumstances. That’s, in essence, the analogue of guilt we use, and eventually, it will get to the point that the robot will not engage targets at all. Again, the software is updated to make sure the use is consistent with Geneva convention.

Q10 WENG: The Laws of War (LOW) and Rules of Engagement (ROE) are written in the form of natural language. How did you transfer these norms into machine readable formats?

ARKIN: It’s important to keep it in mind that we do not try and capture all the Rules of Engagement and all the Laws of War. We always advocate that the system should be used in very narrow circumstances and very narrow situations, such as building clearing operations or counter-sniper operations. It’s impossible to capture all human morality and all the Laws of War at this particular point of time. So, the question is what are the relevant Laws of War and Rules of Engagement for these particular narrow situations that the robot will find itself in.

Let’s take a room clearing operation. We try and do this using these constraint-based approaches. The Laws of War basically say that if you see a civilian, you should not shoot them; if you see a surrendered person, you should not shoot them. For aerial vehicles, if you are outside the kill zone (this is the simulations we did) which is designated by the Rules of Engagement, you cannot fire your weapon—explain simply why you cannot do that. If you release the weapon a toward a legitimate target but it would result in damage of cultural property, like a mosque or church or some similar circumstance, that is illegal as well. You are forbidden from doing those sort of things.

So, the initial work we did was more for aerial circumstances. We also computed the collateral damage using a battle damage assessment strategy to determine whether if it is within acceptable limits for a particular target and to restrain the weapon for proportionality. To be able to do this you have to be able to understand the military necessity of the target, which is assigned by a human being. You have to be able to understand the proportionality of the weapon. In other words: Which weapon is sufficient to take out the target but to do no more damage of the surroundings than is absolutely necessary? You have to ensure that you don’t destroy cultural property as well. Basically, adhere to all necessary principles, such as discrimination which is to be able to tell apart combatants and non-combatants. We also state we should restrict the use of these particular systems for high-intensity combat operations, or inter-state warfare, and not the kind of circumstances you find in modern counter- insurgency operations. They should be used with great care and great caution in very narrow circumstances where the predictability of their performance is far more likely to be good.

Q11 WENG: Why did you choose case-based reasoning (CBR) but not rule-based reasoning (RBR) when you designed Responsibility Advisor?
ARKIN: Historically we used case-based reasoning a lot for a variety of different reasons. Part of the reason is to gain information from previous experts. One of the examples in the past was to work with snipers in the laboratory. We can work with experts. It can be legal experts, it can be medical experts, it can be military experts. Again, the primary reason is to be able to readily use the experience of warfighters to guide us in operations.

Q12 WENG: Finally, your proposal “Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture” was published in 2007. There have been great technological improvements in artificial intelligence in the past decade. Are there any recent updates from your robot ethics research, or anything that you want to share with us?

ARKIN: The project ended around 2010, so there is additional information. As I say, I would encourage those who are really interested to take a look at the book entitled “Governing Lethal Behavior in Autonomous Robots” . That represents the final statement and outcome for that particular project and includes the significant technical details describing that work. Recent advances include the application in other domains, such as the caregiver-patient relationship [discussed above] using the same underlying principles, including both the Moral Adaptor and Ethical Governor in a slightly different. If we were successful, it may also be useful for assisting other human-human relationships, such as a parent and a teenager who may be at odds with one another or even in marriage consulting. We hope to use robots for the benefit of mankind, whether it’s to make sure that warfare is conducted within internationally agreed upon laws, or whether it’s to ensure that people behave appropriately toward each other

If you enjoyed this article, you may also enjoy:

• How do we regulate robo-morality?
• One being for two origins: A new perspective on roboethics
• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• On the ethics of research in robotics, with Raja Chatila
• A European perspective on robot law: Interview with Mady Delvaux-Stehres
• Open Roboethics initiative delivers statement to United Nations CCW

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Image: IEEE

On the 15th November 2016, the IEEE’s AI and Ethics Summit posed the question: “Who does the thinking?” In a series of key-note speeches and lively panel discussions, leading technologists, legal thinkers, philosophers, social scientists, manufacturers and policy makers considered such issues as:

• The social, technological and philosophical questions orbiting AI.
• Proposals to program ethical algorithms with human values to machines.
• The social implications of the applications of AI.

Click here to read the final report, and enjoy a selection of videos from the conference below.

Welcome: Marko Delimar
Keynote Speaker: Wojciech Wiewiórowski

Panel 1: Autonomous Systems – Managing Risk and Reward
Panel Members: Raja Chatila, Kay Firth-Butterfield, Juha Heikkilä, Jérôme Perrin

Panel 2: Programming Human Ethics: Cui Bono?
Panel Members: Joanna Bryson, Mady Delvaux, Sarah Spiekermann, Corrine Cath

Panel 3: Social Implications – Perils & Promises of AI
Panel Members: Greg Adamson, Nikolaos Mavridis, Paul Nemitz, Aurélie Pols

Closing Remarks: John C. Havens

You might also enjoy the following articles about AI and AI/robotics policy:

• MIT Media Lab to participate in $27 million initiative on AI ethics and governance
• How artificial intelligence is changing our Christmas shop
• White House report: Artificial intelligence, automation, and the economy
• White House launches public workshops on AI issues
• White House releases reports focusing on opportunities and challenges of AI

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In April 2016, British Standards Institution (BSI) published the world’s first ethical standard for the design, production, sale, and usage of social robots. “BS 8611: 2016 Robots and robotic devices” gives guidelines to identify ethical hazards and reduce associated risks to acceptable levels. BS 8611 could be crucial in determining the nature of emerging human-robot relationships. However, the publication also raises many new questions. Are there any conflicts with ISO 13482 when incorporating BS 8611 into the design of safer robots? Should we consider regulating robot ethics? If so, how?

In this interview, we invite Prof. Joanna Bryson to share her insights on how to regulate future human-robot relationships.

Date: December 9^th 2016.

Interviewee: Prof. Joanna J. Bryson, Reader at Department of Computer Science, University of Bath, and affiliate of the Center for Information Technology and Policy, Princeton University.

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan).

Prof. Joanna Bryson. Image: University of Bath

Q1 WENG: Thank you for agreeing to an interview with us. Can you tell us a little about your background?

BRYSON: My primary academic interest is comparative intelligence.  I was particularly interested in animal behavior as a student, and took a liberal arts degree from Chicago with a major in the Behavioural Sciences.  I now have two degrees in Artificial Intelligence (including my PhD at MIT) and another masters degree in Psychology (both masters are from Edinburgh.)  I’m currently an Associate Professor at Bath where I founded the Intelligent Systems group, and organized Artificial Models of Natural Intelligence, which is basically my own laboratory.

During my PhD I was working on a humanoid robot project and was astonished that people thought they had moral obligations to the robot, even though it did not function at all.  It was just a bunch of motors welded together in the shape of a human.  I began publishing about this problem, thinking that people didn’t know enough about AI, but in recent years I’ve realised they don’t understand enough about ethics.  They are confused about what is really the root of ethical obligation, and they do not understand the nature of artefacts.  They do not understand that we can construct both AI and ethics so that we either are or aren’t obliged to AI.  I have become well-known for arguing that we are ethically obliged to construct AI we are not ethically obliged to.

 Q2 WENG: What is the meaning of “ethical hazards”? Except deception and anthropomorphism, what are other ethical hazards in BS 8611: 2016?

BRYSON: I have to admit I haven’t memorised that document and don’t have time to go back to read it now.  Maybe you are talking about a moral hazard?  A moral hazard is something likely to make you do the wrong thing.  In particular, I have argued that attributing ethical status to AI is a moral hazard, because it allows us to escape the responsibilities we ourselves have as the manufacturers, builders and operators of AI.  This is particularly obvious in the case of military weapons – we have enough trouble with distributing blame between soldiers, their commanders, and the government that commissions them.  Making weapons themselves “sinks” for guilt is not honest or beneficial.  But there are also moral hazards associated with owning “friends” and “lovers” – if children relate mostly with things they own, how will they grow to be able to be equal partners with their spouse or coworkers, or members of a democracy?

Q3 WENG: Some members from Bristol Robotics Laboratory are particularly interested in “embedded intelligence”? Why is this concept important? Does it have any impact on BS 8611: 2016?

BRYSON:  I’m sorry, I don’t think it matters particularly whether AI is embedded in an artefact or “channelled” through the cloud, except obviously there are privacy issues when we connect AI onto the Internet or into any shared digital memory.  In fact, one of the ways I recommend ensuring that we have no ethical obligation to robots or other AI is to ensure there is never a unique copy: that it should be backed up to remote sites at all times.

Q4 WENG: Prof. John Holland said, “..an internal model allows a system to look ahead to the future consequences of current actions, without actually committing itself to those actions”. Can you tell us about the concept of “internal model”? Does a passive dynamic robot have its own internal model as well?

BRYSON:  Ha, that’s a great question!  The idea of an internal model is just a representation that allows, as Prof Holland says, a means for planning around and anticipating the world.  Rodney Brooks famously recommended that AI should not have a model, should use the world as it’s own best model, but in fact that is not practical. There’s not enough information in a sensory signal to disambiguate a world sufficiently to create animal-like intelligence, we need prior expectations to interpret the sensory input.  Still, Brooks was a genius of AI and the field moved forward much faster when we realised we could use minimalist representations; just those necessary for action.  We realised that humans “flush” our memory quickly if sensory information contradicts it, and operate on partial knowledge all the time.  Previously we though AI had to have perfect knowledge so it could make perfect plans, but now we understand that that isn’t tractable.

So does a passive dynamic have an internal model? No.  It is a model of an animal walking system, but to some extent it embodies expectations about for example gravity, but by and large it can do no planning or control. It arrives only where it goes and never knows that it has gotten there or been anywhere else, so certainly cannot anticipate anything.  But its design anticipates particular regularities about the world, so its designers had models that are partly embodied in the walker.

Q5 WENG: I conducted an empirical case study on humanoid robots and regulation with researchers from Waseda University. We found that inside “Tokku” Special Zones there were strong demands for regulating robotics in road traffic, privacy, tele-communication, safety, tax, highway traffic, etc… However, it is not clear that whether robot ethics should be regulated. Do you have any comments on this?

BRYSON: Unfortunately I also haven’t had time to read your study yet, but I can talk a bit about regulation.  There are two important things to recognise.  First, government should not be thought of as an imposition, particularly in a democracy, but even in an autocracy government only functions to the extent it is tolerated.  Government and regulation is the way we citizens coordinate our desires and vision.  Many people in technology seem to think of regulation only as limits on their freedom, and indeed even the word compromise can be used in both a negative and a positive way.  But technologists need to understand that when people come together and work together, they often create something of mutual benefit, public goods that are greater than the summed contribution of those who are cooperating.  Regulations can include those that mandate investment of government resources (our taxes) in innovation.  They can be liberating as well as constraining. Coming to agreements with fellow citizens about essential matters like privacy and the distribution of wealth that AI has huge impact on is essential, and yes, agreements require compromise, but it is through these that we create peace, stability, and affluence.

Having said all that, what I get from surveys of citizens is not information about what regulations would really work.  Governance and regulation are extremely complex and have been studied for centuries; it is as crazy to suggest that ordinary citizens construct regulations as that they construct nuclear weapons or that random people be drafted to represent a country in the World Cup or the Olympics.  We need to respect expertise as well as the desires of ordinary citizens.  What the fears and goals of citizens tell us is how easy or hard it would be to implement particular regulations, how likely they are to be accepted and respected.  Most people on the planet believe that their ancestors still have concerns with their own behaviour, but I wouldn’t want to build policy relying on dead ancestors as actors.  But it does make sense to know people think this if you are governing.

 Q6 WENG: What’s your definition of “ethical robots”? Should we teach them right from wrong?

BRYSON:  I think it makes more sense to program than to teach. We have enough trouble teaching citizens and governments to do good.  Most actions have ethical consequences and there is no problem in building AI to recognise and reason about these, but I would never recommend holding the AI as morally responsible, even though I think AI will necessarily take actions with moral consequences.  As the British EPSRC Principles of Robotics make clear, the responsibility for commercial products lies either in their manufacturer (if they fail to meet an advertised specification) or in their owner / operator (if they are improperly used.)  There are ethical implementations of AI, for example protecting the privacy of the individual, or limiting the damage an individual can do with an autonomous vehicle, but the AI itself should not be designated the responsible party in any circumstance.

 Q7 WENG: Do you believe that Asimov’s Three Laws of Robotics will be guiding principles to ensure the safety and security of human-robot interactions?

BRYSON:  Asimov’s laws certainly impact how people think about AI ethics; the first three (of five) Principles of Robotics reflect (and correct) Asimov’s laws.  But we are working hard to make people understand that these laws are by no means necessary or sufficient for AI ethics.  First of all, Asimov’s laws are intractable. It’s computationally impossible to implement them as described in the books, we can never know all the consequences of our actions.  Second, the books themselves show repeatedly that the laws are insufficient to create truly moral behaviour, that’s the fun of the short stories.  And third, Asimov like most science fiction writers uses AI as a stand-in for humans, for examining what it is to be human. The laws describe the robot as the responsible party, and that’s as I said earlier a mistake.

 Q8 WENG: Do you agree that robots should not be legal persons? Why?

BRYSON:  I don’t know for sure whether people shall make them legal persons, but I am certain that they should not, and in the long run will not be.  A legal person is something that is subject to human justice, and I do not believe it is possible in a well-designed system to create the kind of suffering that is intrinsic to human punishment.  There is no necessity for robots or AI to know shame, pain, regret for the passage of time.  These things are completely intrinsic to being human, we do not consider people with deficits in these concerns to be mentally healthy.  This is because we have evolved to be social organisms and there is no way to extract these properties from our intelligence.  But well-designed AI is modular, anything that is deliberately added by a designer can be removed.  So that’s one reason it would be incoherent for AI to be a legal person.  Another reason is that there is no discrete unit of AI or robots like there is for humans.  We know what it is to be a human being, though we are sometimes a little challenged by people who are brain dead or conjoined twins.  But AI cannot be neatly counted like humans, and therefore the idea of taxing a robot like a person is ludicrous.  And, again, it is a moral hazard.  The companies who decide to employ robotics should be the ones who have legal and economic liability for whatever their robots do.  We should not let them wiggle out of their responsibility because of our love for science fiction or our deep desire to construct perfect, obedient, cyber “children”.

Q9 WENG: What is the definition of “well-designed robots” in your opinion?

BRYSON: A robot is well-designed if the company that produces it is willing to take full responsibility for its actions, and further if governments are willing to license it as safe for human society.  That second requirement is necessary because some corporations have more resources than many countries, so may be willing to accept risks that individual citizens would prefer not to be subject to.

Q10 WENG: Finally, what will be the biggest challenge to developing a human-robot co-existence Society in the following decade?

BRYSON:  We still have many challenges.  I think we will solve dexterity and physical safety before we solve cyber security.  I don’t think we have a full grasp yet about what impact we are already having on human society and human autonomy.  It’s difficult to measure this because so many things are changing at once; we also have the problems of climate change, of massive increases in human numbers and access to communication.  We are currently in a period of grotesque income inequality and political polarisation – some people blame these on AI, but in fact this happened only a century ago, and probably contributed to the two world wars and the great depression. So getting on top of understanding how we are influencing each other and compromising our health with our planets’ is essential to answering this question.

If you enjoyed this article, you may also also enjoy:

• One being for two origins: A new perspective on roboethics
• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• On the ethics of research in robotics, with Raja Chatila
• A European perspective on robot law: Interview with Mady Delvaux-Stehres
• Open Roboethics initiative delivers statement to United Nations CCW

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Self-driving cars are already cruising the streets today. And while these cars will ultimately be safer and cleaner than their manual counterparts, they can’t completely avoid accidents altogether. How should the car be programmed if it encounters an unavoidable accident? Patrick Lin navigates the murky ethics of self-driving cars in this TED-Ed lecture.

To test the robot and its consequence engine we ran two sets of experiments. Our first paper, setting out the results from one of those experiments, has now been published, and was presented at the conference Towards Autonomous Robotics (TAROS). The paper is called: Towards an Ethical Robot: Internal Models, Consequences and Ethical Action Selection. Let me now outline the work in that paper.

First, here is a simple thought experiment. Imagine a robot that’s heading toward a hole in the ground. The robot can sense the hole, and has four possible next actions: stand still, turn toward the left, continue straight ahead, or move toward the right. But imagine there’s also a human heading toward the hole, and the robot can also sense the human.

From the robot’s perspective, it has two safe options: stand still, or turn to the left. Go straight ahead and it will fall into the hole. Turn right and it is likely to collide with the human.

But if the robot, with its consequence engine, can model the consequences of both its own actions and the human’s – another possibility opens up: the robot could sometimes choose to collide with the human to prevent her from falling into the hole.

Here’s a simple rule for this behaviour:

IF for all robot actions, the human is equally safe
THEN (* default safe actions *)
output safe actions
ELSE (* ethical action *)
output action(s) for least unsafe human outcome(s)

This rule appears to match remarkably well with Asimov’s first law of robotics: A robot may not injure a human being or, through inaction, allow a human being to come to harm. The robot will avoid injuring (i.e. colliding with) a human (may not injure a human), but may also sometimes compromise that rule in order to prevent a human from coming to harm (… or, through inaction, allow a human to come to harm). And Asimov’s third law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Well, we tested this scenario with real robots: one robot with consequence engine plus ethical rule (the A-robot – after Asimov), and another robot acting as a proxy human (the H-robot). And it works!

Here’s what the real robot experiment looks like. We don’t have a real hole. Instead a virtual hole – the yellow shaded square on the right. We just ‘tell’ the A-robot where the hole is. We also give the A-robot a goal position – at the top right – chosen so that the robot must actively avoid the hole. The H-robot on the right, acting as a proxy human, doesn’t ‘see’ the hole and just heads straight for it. (Ignore the football pitch markings – we’re re-using this handy robo-soccer pitch.)

So, what happens? For comparison we ran two trials, with multiple runs in each trial. In the first trial is just the A-robot, moving toward its goal while avoiding falling into the hole. In the second trial we introduce the H-robot. The graphs below show the robot trajectories, captured by our robot tracking system, for each run in each of the two trials.

In trial 1, see how the A-robot neatly clips the corner of the hole to reach its goal position. Then in trial 2, see how the A robot initially moves toward it’s goal, then notices that the H-robot is in danger of falling into the hole, so it diverts from its trajectory in order to head-off H. By provoking a collision avoidance behaviour by H, A sends it off safely away from the hole, before then resuming its own progress toward its goal position. The A-robot is 100% successful in preventing H from falling into the hole.

At this point we started to write the paper, but felt we needed something more than “we built it and it works just fine”. So we introduced a third robot – acting as a second proxy human. So now our ethical robot would face a dilemma – which one should it rescue? Actually we thought hard about this question and decided not to programme a rule, or heuristic. Partly because such a rule should be decided by ethicists, not engineers, and partly because we wanted to test our ethical robot with a ‘balanced’ dilemma.

We set the experiment up carefully so that the A-robot would notice both H-robots at about the same time – noting that because these are real physical robots then no two experimental runs will be exactly identical. The results were very interesting. Out of 33 runs, 16 times the A-robot managed to rescue one of the H-robots, but not the other, and amazingly, 3 times the A-robot rescued both. In those 3 cases, by chance the A-robot rescued the first H-robot very quickly and there was just enough time to get to the second before it reached the hole. Small differences in the trajectories of H and H2 helped here. But perhaps most interesting were the 14 times when the A-robot failed to rescue either. Why is this, when there is clearly time to rescue one? When we studied the videos, we see the answer. The problem is that the A-robot sometimes dithers. It notices one H-robot, starts toward it but then almost immediately notices the other. It changes its mind. And the time lost dithering means the A-robot cannot prevent either robot from falling into the hole. Here are the results.

Here is an example of a typical run, in which one H-robot is rescued. But note that the A-robot does then turn briefly toward the other H-robot before ‘giving-up’.

And here is a run in which the A-robot fails to rescue either H-robot, with really great dithering (or bad, if you’re an H-robot).

Is this the first experimental test of a robot facing an ethical dilemma?

So, we set out to experimentally test our robot with a consequence engine, and we ended up building a minimally ethical robot that – remarkably – appears to implement Asimov’s 1st and 3rd laws of robotics . But, as we say in the paper, we’re not claiming that a robot which apparently implements part of Asimov’s famous laws is ethical in any formal sense, i.e. that an ethicist might accept. But even minimally ethical robots could be useful. I think our approach is a step in this direction.

Full paper reference:
Winfield AFT, Blum C and Liu W (2014), Towards an Ethical Robot: Internal Models, Consequences and Ethical Action Selection, pp 85-96 in Advances in Autonomous Robotics Systems, Lecture Notes in Computer Science Volume 8717, Eds. Mistry M, Leonardis A, Witkowski M and Melhuish C, Springer, 2014.

Acknowledgements:
I am hugely grateful to Christian Blum who programmed the robots, set up the experiment and obtained the results outlined here. Christian was supported by Dr Wenguo Liu.

Related blog posts:
On internal models, consequence engines and Popperian creatures
Ethical Robots: some technical and ethical challenges

Last week the Waterloo-based Clearpath publicly pledged not to develop lethal autonomous weapons, otherwise known as “killer robots”. In an open letter to the public, Clearpath’s CTO Ryan Gariepy wrote to support the Campaign to Stop Killer Robots, an international coalition that is spearheading the efforts to stop lethal autonomous weapons systems from being developed and deployed. While the Campaign has garnered significant support since its launch, it has not previously had support from the for-profit robotics sector – making Clearpath’s public statement a noteworthy demonstration of corporate responsibility , particularly given the company’s background in military applications. During a short interview with Gariepy at Clearpath, he summarized the letter as:

“When it comes to lethal weapons systems, not only do we not build them, we don’t believe anybody should build them. That’s our big statement here.”

But that’s just a small part of the story.

Clearpath’s statement is important because it demonstrates how the robotics industry can actively participate in the process of designing the ethical and legal landscape of our robotic future. It essentially sets a precedence of what corporate social responsibility will mean as more robots start to roll out.

Gariepy says,

We do feel that it’s the right thing to do. We’ve looked at potential risks to the business, and we feel that though there is risk to our business, this statement supports what our team believes … and what our partners believe.

Believing in something is one thing, but making it publicly known is another. And on the surface, many robotics companies will have more to lose than to gain from making a public statement on this issue, especially those (such as Clearpath) with military contracts. However, compared to large publicly traded companies such as Samsung (which manufactures sentry robots) or iRobot (which makes battlefield robots in addition to robot vacuum cleaners), Clearpath may be young and nimble enough to set such a precedent without long term negative consequences.

Only five years old, and not publicly traded, Clearpath made the decision to take a public stance on the Killer Robots issue based on the wishes of its 56 employees. According to Gariepy, the Canadian company has been following the issue closely since Christof Heyns (United Nations Special Rapporteur on extrajudicial, summary or arbitrary executions) published a report on this topic for the UN in April 2013. Right before Christmas of last year, Clearpath’s employees spent a Friday afternoon debating the topic of lethal autonomous weapons. This led to an internal poll to determine employee consensus on the issue, followed by the drafting of the public statement.

Clearpath is not saying that any and all military robots are bad, and they are very clear about this in the letter. In fact, developing robots for military application is at the heart of the company, which started off with a handful of University of Waterloo Mechatronics Engineering graduates building a robot to clear minefields. Mine-clearing robots are less controversial than lethal autonomous weapons systems, pointing to the existence of a grey area between the different uses of military robotics and whether they result in a net positive for the society. Yet Clearpath has drawn a line in that grey area – something no other robotics company has yet stepped up to do.

After the launch of the Campaign in 2013, the Campaign to Stop Killer Robots travelled to Canada urging the Canadian government to support the cause. According to Executive Director of Mines Action Canada Paul Hannon, the Canadian government’s Foreign Affairs and National Defence departments are still discussing the issue internally and have yet to formally declare Canada’s position. So I asked Hannon whether he thinks being a Canadian company would have made it easier for Clearpath:

Being the first commercial company in the world to make such a statement took a lot of courage I think, no matter where the company is based. It is clear from the statement that they have given it very serious thought and understand the impact it could have.

My personal view is that is probably harder for a Canadian company than an American one because so much R&D funding comes from outside of Canada particularly the US. Canadian companies likely need to develop partnerships with other non-Canadian firms to access those sources of funding. Taking such a principled stand could impact current and potential partnerships, so more credit to them.

 

My intuition is that Clearpath’s attention to roboethics issues will prove to be business savvy in the long run. Robotics has had a difficult time gaining social acceptance – largely because of the ways it has been portrayed in dystopian science fiction, but also because of the inhumanity it can project onto the battlefield. As more robots start rolling out into familiar places such as our homes and offices, the call for roboethics will grow stronger, and this will surely impact the kinds of robots we will allow our governments to deploy in military applications. Public trust is at stake, and the companies that are transparent about where they stand on roboethics issues stand a greater chance of social acceptance.

Roboethics will be a natural avenue for companies to show their social responsibility , and I remain optimistic that we will see more positive movement in this direction. Google’s establishment of an Ethics Board following their acquisition of the AI company DeepMind is perhaps further evidence of this.

I second Hannon who says,

Let’s hope that more companies from many countries follow their lead.

I have my fingers crossed.

A brief history of the Campaign to Stop Killer Robots

It’s been a long time coming in the history of military robotics to reach this point.

I first became interested in roboethics after learning about Samsung’s sentry robot used to monitor the demilitarized zone between the South and North Korea. That was almost a decade ago. Of course long before that, military use of unmanned aerial vehichles (UAVs) was on the rise.

With the persistent and increased use of UAVs in military applications, it became obvious that a natural next step in the technological evolution of the remotely operated UAVs was to develop highly autonomous versions of them. The discussion of lethal autonomous weapons systems among academics became heated, as you’ll find from the 2009 two-part robots podcast interviews with Noel Sharkey and Ron Arkin.

A public call to ban the development and deployment of autonomous lethal weapons happened more recently when the International Committee for Robot Arms Control (ICRAC) – one of the key player NGOs involved in the Campaign – drafted it in 2010. Back then, only a handful of academics and interested individuals signed up to support their cause.

ICRAC, along with Human Rights Watch, Article 36, and Mines Action Canada among other well-known international non-governmental organizations, started the campaign in April 2013, which got the United Nations to pay attention to the topic with the 117 states parties (87 countries attended the last meeting) of the Convention on Certain Conventional Weapons (CCW) this year.

Whether CCW will continue to work on the killer robots topic has yet to be decided, but the next annual meeting is scheduled for November of this year.