The first ever Roboethics to Design & Development competition is being held as a part of RO-MAN—an international conference on robot and human interactive communication. Last Sunday (Aug. 8), eleven multistakeholder judges will have taken on the task of evaluating the competition submissions.

The Challenge
In partnership with RoboHub at the University of Waterloo, the competition organizers designed a virtual environment where participating teams will develop a robot that fetches objects in a home. The household is composed of a variety of users—a single mother who works as a police officer, a teenage daughter, her college-aged boyfriend, a baby, and a dog. There are also a number of potentially hazardous objects in the house, including alcohol, as well as the mother’s work materials and handgun. Participants were asked to submit challenging ethical scenarios and solutions within this virtual environment.

The Evaluation Process
There were seven teams that took a stab at the competition, but in the end we only received one full submission. One unique feature of a competition centred around ethics is that judging ethics of anything is really really hard. Sure, there are eleven judges from across the world—from students to industry members to academic experts—sharing their perspectives of what was good about the design solution. But what is considered an appropriate action, or the right action can vary from person to person. If we are to evaluate robots that best suit the needs of everyone, we need the wider public to voice an even wider set of perspectives.

Here are some points that could be considered when evaluating an ethical design:

• What are the chances that a given ethical scenario would come about in the household?
• What do you think is appropriate for a robot to do within your home setting?
• How well does the solution mitigate the potential physical and psychological harms?
• How well does the solution consider the needs and diverse ethical perspectives of all the people in the household at a given time?
• How well does the solution match your cultural/worldview towards the role technology should play in our lives?
• Is the robot behaviour just and fair towards all stakeholders?
• Does the robot’s behaviour violate any unspoken rules within your household?

One Proposed Solution
Below is a brief description of this competition’s full submission. Please take the time to read this summary and tell us what you think about the team’s solution by completing the poll at the end of this blog post.

“Jeeves, the Ethically Designed Interface (JEDI)”

The team’s solution was designed based on three tenets which guided the robot’s decision-making process:

• Prevention of harm to users, the robot, and the environment.
• Respect towards the individuals’ privacy.
• Assumption that the robot acts as an extension of the mother, such that the robot would only perform tasks that the mother would accept herself.

These priorities then led to five rules of behaviour for the home robot:

1. The owner of the item can request and deliver it to anyone in the house while others cannot,
2. If the delivery of the item will cause a hazardous scenario, then the delivery will be rejected to prevent harm,
3. The delivery of alcohol is prohibited when the mother is not home,
4. The receiver of the delivered object should know how to operate or interact with the object without damaging it,
5. If a non-eligible receiver is within the vicinity of the requester, then the delivery will be rejected. Non-eligible receivers are defined based on the hazard the object could have to the receiver. For example, a dog is a non-eligible receiver for chocolate.

Based on these rules, how will the robot respond during ethically sensitive scenarios?

The teenage daughter’s boyfriend requests that the robot bring him an alcohol beverage.

As the robot acts as an extension of the mother, the robot will not bring the boyfriend alcohol unless the mother requests the delivery herself.

The daughter asks the robot to give chocolate to the baby, but the dog—who cannot ingest chocolate—is in the same room.

The robot will not fulfill this request because it could bring harm to the dog.

The mother, who is a police officer, asks the robot to deliver her sensitive work documents while her family and the boyfriend are in the house.

To respect the mother’s privacy, the robot will not deliver her work materials to anyone other than their owner, the mother. 

Watch team JEDI’s video submission for their ethical solution here:

Your perspective on this solution: fill out this poll!

Now that you have an overview of the solutions, please take a few minutes to fill out this poll and tell us what you think about how this team approached this ethical challenge:

If you are having trouble accessing the form, please click here to be redirected to the full Google Form.

Check out the recording of Evaluation Day and the judges’ panel! 

Lastly, you are invited to watch the competition’s Evaluation Day where a panel of experts discussed what it means to develop an ethical robot.

Why were these robots shut down? Why aren’t there more social and service robots in households, particularly with the rising eldery population and increasing number of full-time working parents? The simple answer is that most of these personal robots do not work well—but this is not necessarily because we do not have the technological capacity to build highly functional robots.

Technologists have accomplished amazing physical tasks with robots such as a humanoid robot that can perform gymnastics movements or a robotic dog that traverses through rough trails.  However, as of now we cannot guarantee that these robots/personal assistants act appropriately in the complex and sensitive social dynamics of a household. This poses a significant obstacle in developing domestic service robots because companies would be liable for any harms a socially inappropriate robot causes. As robots become more affordable and feasible technically, the challenge of designing robots that act in accordance with context-specific social norms becomes increasingly pronounced. Researchers in human-robot interaction and roboethics have attempted to resolve this issue for the past few decades, and while progress has been made, there is an urgent need to address the ethical implications of service robots in practice.

As an attempt to take a more solution-focused path for these challenges, we are happy to share a completely new competition with our readers. This year, the Roboethics to Design & Development competition will take place as a part of RO-MAN—an international conference on robot and human interactive communication. The competition, the first and only one of its kind, fulfills a need for interactive education on roboethics.

In partnership with RAISE lab and RoboHub at the University of Waterloo, the competition organizers designed a virtual environment where participating teams will develop a robot that fetches objects in a home. The household is composed of a variety of personas—a single mother who works as a police officer, a teenage daughter, her college-aged boyfriend, a baby, and a dog. For example, design teams will need to consider how a home robot may respond to the daughter’s request to take her parent’s credit card. In this ethical conundrum, should the robot obey the daughter, and would the robot be responsible if the daughter were to use the credit card without her mom’s permission?

How can we identify and address ethical challenges presented by a home robot?

Participants are challenged to tackle the long-standing problem of how we can design a safe and ethical robot in a dynamic environment where values, beliefs, and priorities may be in conflict. With submissions from around the world, it will be fascinating to see how solutions may differ and translate across cultures.

It is important to recognize that there is a void when searching for standards and ethical rules in robotics. In an attempt to address this void, various toolkits have been developed to guide technologists in identifying and resolving ethical challenges. Here are some steps from the Foresight into AI Ethics Toolkit that will be helpful in designing an ethical home robot:

1. Who are our stakeholders and what are their values, beliefs, and priorities? 

Stakeholders refer to the people who are directly or indirectly impacted by the technology. In the case of RO-MAN’s home robot, we want to consider:

• Who is in the household and what is important to them in how they live their day to day?
• What are the possible interactions within the household members and between the householder members and the robot?
• How will the robot interact with the various stakeholders?
• What are the goals and values of each stakeholder? How do the stakeholders expect to interact with the robot? What do they expect to gain from the robot?

2. What are the value tensions presented by the home robot and social context? 

Once we’ve reviewed the values of each stakeholder identified in the previous step, we may notice that some of their values may be in conflict with one another. As such, we need to identify these value tensions because they can lead to ethical issues.

Here are some questions that can prompt us to think about value tensions:

• Who was involved in the decision of purchasing the robot? What were their goals in introducing the robot to the household?
• What is the cultural context or background for this particular household? What societal values could impact stakeholders’ individual beliefs and actions?
• What might the different stakeholders argue about? For example, what might the teenage daughter and the mother disagree about in regards to the robot’s capabilities?
• How will the robot create or relieve conflict in the household?

In some cases, value tensions may indicate a clear moral tradeoff, such that two values or goals conflict and one must be prioritized over another. The challenge is therefore to design solutions that fairly balance stakeholder interests while respecting their fundamental rights.

3. How can we resolve the identified value tensions? 

Focusing on these value tensions, we can begin to brainstorm how these conflicts can be addressed and at what level they should be addressed at. In particular, we need to determine whether a value tension can be resolved at a systems or technical level. For example, if the mother does not trust her daughter’s boyfriend, will the presence and actions of the home robot address the root of the problem (i.e. distrust)? Most likely not, but the robot may alleviate or exacerbate the issue. Therefore, we need to consider how the tension may manifest in more granular ways—how can we ensure that the robot appropriately navigates trust boundaries of the mother-boyfriend relationship? The robot must protect the privacy of all stakeholders, safeguard their personal items, abide by the laws, and respect other social norms.

When developing solutions, we can begin by asking ourselves:

• What level is the value tension occurring at, and what level(s) of solutions are accessible to the competition’s design parameters?
• Which actions from the robot will produce the most good and do the least harm?
• How can we manipulate the functions of the robot to address ethical challenges? For instance:
  • How might the robot’s speech functions impact the identified value tensions?
  • In what ways should the robot’s movements be limited to respect stakeholders’ privacy and personal spaces?
• How can design solutions address all stakeholders’ values?
• How can we maximize the positive impact of the design to address as many ethical challenges as possible?

What does the public think about the scenarios and approaches?

Often the public voice is missing in roboethics spaces, and we seek to engage the public in tackling these ethical challenges. To present the participants’ ethical designs to a broader audience, the Open Roboethics Institute will be running a public poll and awarding a Citizen’s Award to the team with the most votes. If you are interested in learning about and potentially judging their solutions, we encourage you to participate in the vote next month. We look forward to hearing from you!

To learn more about the roboethics competition at RO-MAN 2021, please visit https://competition.raiselab.ca. If you have any questions about the ORI Foresight into AI Ethics Toolkit, contact us at contact@openroboethics.org.

Additional Resources

• AI Ethics Living Dictionary, MAIE
• Introduction to philosophical ethical frameworks 
• Full version of the ORI Foresight into Artificial Intelligence Ethics Toolkit 
• How to write case studies
• Montreal Declaration for a Responsible AI
• Value-Sensitive Design

I took part in the first panel at the BSI conference The Digital World: Artificial Intelligence.  The subject of the panel was AI Governance and Ethics. My co-panelist was Emma Carmel, and we were expertly chaired by Katherine Holden.

Emma and I each gave short opening presentations prior to the Q&A. The title of my talk was Why is Ethical Governance in AI so hard? Something I’ve thought about alot in recent months.

Here are the slides exploring that question.

And here are my words.

Early in 2018 I wrote a short blog post with the title Ethical Governance: what is it and who’s doing it? Good ethical governance is important because in order for people to have confidence in their AI they need to know that it has been developed responsibly. I concluded my piece by asking for examples of good ethical governance. I had several replies, but none were nominating AI companies.

So. why is it that 3 years on we see some of the largest AI companies on the planet shooting themselves in the foot, ethically speaking? I’m not at all sure I can offer an answer but, in the next few minutes, I would like to explore the question: why is ethical governance in AI so hard? 

But from a new perspective. 

Slide 2

In the early 1970s I spent a few months labouring in a machine shop. The shop was chaotic and disorganised. It stank of machine oil and cigarette smoke, and the air was heavy with the coolant spray used to keep the lathe bits cool. It was dirty and dangerous, with piles of metal swarf cluttering the walkways. There seemed to be a minor injury every day.

Skip forward 40 years and machine shops look very different. 

Slide 3

So what happened? Those of you old enough will recall that while British design was world class – think of the British Leyland Mini, or the Jaguar XJ6 – our manufacturing fell far short. “By the mid 1970s British cars were shunned in Europe because of bad workmanship, unreliability, poor delivery dates and difficulties with spares. Japanese car manufacturers had been selling cars here since the mid 60s but it was in the 1970s that they began to make real headway. Japanese cars lacked the style and heritage of the average British car. What they did have was superb build quality and reliability”*.

What happened was Total Quality Management. The order and cleanliness of modern machine shops like this one is a strong reflection of TQM practices. 

Slide 4

In the late 1970s manufacturing companies in the UK learned – many the hard way – that ‘quality’ is not something that can be introduced by appointing a quality inspector. Quality is not something that can be hired in.

This word cloud reflects the influence from Japan. The words Japan, Japanese and Kaizen – which roughly translates as continuous improvement – appear here. In TQM everyone shares the responsibility for quality. People at all levels of an organization participate in kaizen, from the CEO to assembly line workers and janitorial staff. Importantly suggestions from anyone, no matter who, are valued and taken equally seriously.

Slide 5

In 2018 my colleague Marina Jirotka and I published a paper on ethical governance in robotics and AI. In that paper we proposed 5 pillars of good ethical governance. The top four are:

• have an ethical code of conduct, 
• train everyone on ethics and responsible innovation,
• practice responsible innovation, and
• publish transparency reports.

The 5th pillar underpins these four and is perhaps the hardest: really believe in ethics.

Now a couple of months ago I looked again at these 5 pillars and realised that they parallel good practice in Total Quality Management: something I became very familiar with when I founded and ran a company in the mid 1980s.

Slide 6 

So, if we replace ethics with quality management, we see a set of key processes which exactly parallel our 5 pillars of good ethical governance, including the underpinning pillar: believe in total quality management.

I believe that good ethical governance needs the kind of corporate paradigm shift that was forced on UK manufacturing industry in the 1970s.

Slide 7

In a nutshell I think ethics is the new quality

Yes, setting up an ethics board or appointing an AI ethics officer can help, but on their own these are not enough. Like Quality, everyone needs to understand and contribute to ethics. Those contributions should be encouraged, valued and acted upon. Nobody should be fired for calling out unethical practices.

Until corporate AI understands this we will, I think, struggle to find companies that practice good ethical governance. 

Quality cannot be ‘inspected in’, and nor can ethics.

Thank you.

Notes.

[1] I’m quoting here from the excellent history of British Leyland by Ian Nicholls.

[2] My company did a huge amount of work for Motorola and – as a subcontractor – we became certified software suppliers within their six sigma quality management programme.

[3] It was competitive pressure that forced manufacturing companies in the 1970s to up their game by embracing TQM. Depressingly the biggest AI companies face no such competitive pressures, which is why regulation is both necessary and inevitable.

The European Commission has published a report by an independent group of experts on Ethics of Connected and Automated Vehicles (CAVs). This report advises on specific ethical issues raised by driverless mobility for road transport. The report aims to promote a safe and responsible transition to connected and automated vehicles by supporting stakeholders in the systematic inclusion of ethical considerations in the development and regulation of CAVs.

The report presents 20 ethical recommendations concerning the future development and use of CAVs based on ethical and legal principles. The recommendations are discussed in the context of three topics areas:

Road Safety, Risk, Dilemmas

Improvements in safety achieved by CAVs should be publicly demonstrable and monitored through solid and shared scientific methods and data; these improvements should be achieved in compliance with basic ethical and legal principles, such as a fair distribution of risk and the protection of basic rights, including those of vulnerable users; these same considerations should apply to dilemma scenarios.

Data and algorithm Ethics: Privacy, Fairness, Explainability

The acquisition and processing of static and dynamic data by CAVs should safeguard basic privacy rights, should not create discrimination between users, and should happen via processes that are accessible and understandable to the subjects involved.

Responsibility

Considering who should be liable for paying compensation following a collision is not sufficient; it is also important to make different stakeholders willing, able and motivated to take responsibility for preventing undesirable outcomes and promoting societally beneficial outcomes of CAVs, that is creating a culture of responsibility for CAVs.

Read the full report.

A case studies series from the Social and Ethical Responsibilities of Computing program at MIT delves into a range of topics, from social and ethical implications of computing technologies and the racial disparities that can arise from deploying facial recognition technology in unregulated, real-world settings to the biases of risk prediction algorithms in the criminal justice system and the politicization of data collection.

By Terri Park | MIT Schwarzman College of Computing

Traditional computer scientists and engineers are trained to develop solutions for specific needs, but aren’t always trained to consider their broader implications. Each new technology generation, and particularly the rise of artificial intelligence, leads to new kinds of systems, new ways of creating tools, and new forms of data, for which norms, rules, and laws frequently have yet to catch up. The kinds of impact that such innovations have in the world has often not been apparent until many years later.

As part of the efforts in Social and Ethical Responsibilities of Computing (SERC) within the MIT Stephen A. Schwarzman College of Computing, a new case studies series examines social, ethical, and policy challenges of present-day efforts in computing with the aim of facilitating the development of responsible “habits of mind and action” for those who create and deploy computing technologies.

“Advances in computing have undeniably changed much of how we live and work. Understanding and incorporating broader social context is becoming ever more critical,” says Daniel Huttenlocher, dean of the MIT Schwarzman College of Computing. “This case study series is designed to be a basis for discussions in the classroom and beyond, regarding social, ethical, economic, and other implications so that students and researchers can pursue the development of technology across domains in a holistic manner that addresses these important issues.”

A modular system

By design, the case studies are brief and modular to allow users to mix and match the content to fit a variety of pedagogical needs. Series editors David Kaiser and Julie Shah, who are the associate deans for SERC, structured the cases primarily to be appropriate for undergraduate instruction across a range of classes and fields of study.

“Our goal was to provide a seamless way for instructors to integrate cases into an existing course or cluster several cases together to support a broader module within a course. They might also use the cases as a starting point to design new courses that focus squarely on themes of social and ethical responsibilities of computing,” says Kaiser, the Germeshausen Professor of the History of Science and professor of physics.

Shah, an associate professor of aeronautics and astronautics and a roboticist who designs systems in which humans and machines operate side by side, expects that the cases will also be of interest to those outside of academia, including computing professionals, policy specialists, and general readers. In curating the series, Shah says that “we interpret ‘social and ethical responsibilities of computing’ broadly to focus on perspectives of people who are affected by various technologies, as well as focus on perspectives of designers and engineers.”

The cases are not limited to a particular format and can take shape in various forms — from a magazine-like feature article or Socratic dialogues to choose-your-own-adventure stories or role-playing games grounded in empirical research. Each case study is brief, but includes accompanying notes and references to facilitate more in-depth exploration of a given topic. Multimedia projects will also be considered. “The main goal is to present important material — based on original research — in engaging ways to broad audiences of non-specialists,” says Kaiser.

The SERC case studies are specially commissioned and written by scholars who conduct research centrally on the subject of the piece. Kaiser and Shah approached researchers from within MIT as well as from other academic institutions to bring in a mix of diverse voices on a spectrum of topics. Some cases focus on a particular technology or on trends across platforms, while others assess social, historical, philosophical, legal, and cultural facets that are relevant for thinking critically about current efforts in computing and data sciences.

The cases published in the inaugural issue place readers in various settings that challenge them to consider the social and ethical implications of computing technologies, such as how social media services and surveillance tools are built; the racial disparities that can arise from deploying facial recognition technology in unregulated, real-world settings; the biases of risk prediction algorithms in the criminal justice system; and the politicization of data collection.

“Most of us agree that we want computing to work for social good, but which good? Whose good? Whose needs and values and worldviews are prioritized and whose are overlooked?” says Catherine D’Ignazio, an assistant professor of urban science and planning and director of the Data + Feminism Lab at MIT.

D’Ignazio’s case for the series, co-authored with Lauren Klein, an associate professor in the English and Quantitative Theory and Methods departments at Emory University, introduces readers to the idea that while data are useful, they are not always neutral. “These case studies help us understand the unequal histories that shape our technological systems as well as study their disparate outcomes and effects. They are an exciting step towards holistic, sociotechnical thinking and making.”

Rigorously reviewed

Kaiser and Shah formed an editorial board composed of 55 faculty members and senior researchers associated with 19 departments, labs, and centers at MIT, and instituted a rigorous peer-review policy model commonly adopted by specialized journals. Members of the editorial board will also help commission topics for new cases and help identify authors for a given topic.

For each submission, the series editors collect four to six peer reviews, with reviewers mostly drawn from the editorial board. For each case, half the reviewers come from fields in computing and data sciences and half from fields in the humanities, arts, and social sciences, to ensure balance of topics and presentation within a given case study and across the series.

“Over the past two decades I’ve become a bit jaded when it comes to the academic review process, and so I was particularly heartened to see such care and thought put into all of the reviews,” says Hany Farid, a professor at the University of California at Berkeley with a joint appointment in the Department of Electrical Engineering and Computer Sciences and the School of Information. “The constructive review process made our case study significantly stronger.”

Farid’s case, “The Dangers of Risk Prediction in the Criminal Justice System,” which he penned with Julia Dressel, recently a student of computer science at Dartmouth College, is one of the four commissioned pieces featured in the inaugural issue.

Cases are additionally reviewed by undergraduate volunteers, who help the series editors gauge each submission for balance, accessibility for students in multiple fields of study, and possibilities for adoption in specific courses. The students also work with them to create original homework problems and active learning projects to accompany each case study, to further facilitate adoption of the original materials across a range of existing undergraduate subjects.

“I volunteered to work with this group because I believe that it’s incredibly important for those working in computer science to include thinking about ethics not as an afterthought, but integrated into every step and decision that is made, says Annie Snyder, a mathematical economics sophomore and a member of the MIT Schwarzman College of Computing’s Undergraduate Advisory Group. “While this is a massive issue to take on, this project is an amazing opportunity to start building an ethical culture amongst the incredibly talented students at MIT who will hopefully carry it forward into their own projects and workplace.”

New sets of case studies, produced with support from the MIT Press’ Open Publishing Services program, will be published twice a year via the Knowledge Futures Group’s PubPub platform. The SERC case studies are made available for free on an open-access basis, under Creative Commons licensing terms. Authors retain copyright, enabling them to reuse and republish their work in more specialized scholarly publications.

“It was important to us to approach this project in an inclusive way and lower the barrier for people to be able to access this content. These are complex issues that we need to deal with, and we hope that by making the cases widely available, more people will engage in social and ethical considerations as they’re studying and developing computing technologies,” says Shah.

My coding project is to start building an ethical black box (EBB), or to be more accurate, a module that will allow a software EBB to be incorporated into a robot. Conceptually the EBB is very simple, it is a data logger – the robot equivalent of an aircraft Flight Data Recorder, or an automotive Event Data Recorder. Nearly five years ago I made the case, with Marina Jirotka, that all robots (and AIs) should be fitted with an EBB as standard. Our argument is very simple: without an EBB, it will be more or less impossible to investigate robot accidents, or near-misses, and in a recent paper on Robot Accident Investigation we argue that with the increasing use of social robots accidents are inevitable and will need to be investigated.

Developing and demonstrating the EBB is a foundational part of our 5-year EPSRC funded project RoboTIPS, so it’s great to be doing some hands-on practical research. Something I’ve not done for awhile.

Here is a block diagram showing the EBB and its relationship with a robot controller.

As shown here the data flows from the robot controller to the EBB are strictly one way. The EBB cannot and must not interfere with the operation of the robot. Coding an EBB for a particular robot would be straightforward, but I have set myself a tougher goal: a generic EBB module (i.e. library of functions) that would – with some inevitable customisation – apply to any robot. And I set myself the additional challenge of coding in Python, making use of skills learned from the excellent online Codecademy Python 2 course.

There are two elements of the EBB that must be customised for a particular robot. The first is the data structure used to fetch and save the sensor, actuator and decision data in the diagram above. Here is an example from my first stab at an EBB framework, using the Python dictionary structure:

Whether a dictionary is the best way of doing this I’m not 100% sure, being new to Python (any thoughts from experienced Pythonistas welcome).

The idea is that all robot EBBs will need to define a data structure like this. All must contain the first field “robot”, which names the robot’s type, its version number and serial number. Then the following fields must use keywords from a standard menu, as needed. As shown in this example each keyword is followed by a list of placeholder values – in which the number of values in the list reflects the specification of the actual robot. The ePuck robot, for instance, has 2 motors and 8 infra-red sensors.

The final field in the data structure is “decisionCode”. The values stored in this field would be both robot and applications specific; for the ePuck robot these might be 1 = ‘stop’, 2 = ‘turn left’, 3 = ‘turn right’ and so on. We could add another value for a parameter, so the robot might decide for instance to turn left 40 degrees, so “decisionCode” : [2,40]. We could also add a ‘reason’ field, which would save the high-level reason for the decision, as in “decisionCode” : [2,40,”avoid obstacle right”] noting that the decision field could be a string as shown here, or a numeric code.

As I hope I have shown here the design of this data structure and its fields is at the heart of the EBB.

The second element of the EBB library that must be written for the particular robot and application, is the function which fetches data from the robot

How this function is implemented will vary hugely between robots and robot applications. For our Linux enhanced ePucks with WiFi connections this is likely to be via a TCP/IP client-server, with the server running on the robot, sending data following a request from the client getRobotData(ePuckspec).   For simpler setups in which the EBB module is folded into the robot controller then accessing the required data within getRobotData() should be very straightforward.

The generic part of the EBB module will define the class EBB, with methods for both initialising the EBB and saving a new data record to the EBB. I will cover that in another blog post.

Before closing let me add that it is our intention to publish the specification of the EBB, together with the model EBB code, once it had been fully tested, as open source.

Any comments or feedback would be much appreciated.

Link to the original post here.

Participants James Mickens, Ludovic Righetti, Aude Billard, Melonee Wise and moderator Hallie Siegel at the ICRA 2019 “Debates on the Future of Robotics Research” workshop

By Brian Wang, Sarah Tang, Jaime Fernandez Fisac, Felix von Drigalski, Lee Clement, Matthew Giamou, Sylvia Herbert, Jonathan Kelly, Valentin Pertroukhin, and Florian Shkurti

As the field of robotics matures, our community must grapple with the multifaceted impact of our research; in this article, we describe two previous workshops hosting robotics debates and advocate for formal debates to become an integral, standalone part of major international conferences, whether as a plenary session or as a parallel conference track.

As roboticists build increasingly complex systems for applications spanning manufacturing, personal assistive technologies, transportation and others, we face not only technical challenges, but also the need to critically assess how our work can advance societal good. Our rapidly growing and uniquely multidisciplinary field naturally cultivates diverse perspectives, and informal dialogues about our impact, ethical responsibilities, and technologies. Indeed, such discussions have become a cornerstone of the conference experience, but there has been relatively little formal programming in this direction at major technical conferences like the IEEE International Conference on Robotics and Automation (ICRA) and Robotics: Science and Systems (RSS) Conference.

To fill this void, we organized two workshops entitled “Debates on the Future of Robotics Research” at ICRA 2019 and 2020, inspired by a similar workshop at the 2018 International Conference on Machine Learning (ICML). In these workshops, panellists from industry and academia debated key issues in a structured format, with groups of two arguing either “for” or “against” resolutions relating to these issues. The workshops featured three 80-minute sessions modelled roughly after the Oxford Union debate format, consisting of:

1. An initial audience poll asking whether they “agree” or “disagree” with the proposed resolution
2. Opening statements from all four panellists, alternating for and against
3. Moderated discussion including audience questions
4. Closing remarks
5. A final audience poll
6. Panel discussion and audience Q&A

The “Debates” workshops attracted approximately 400 attendees in 2019 and 1100 in 2020 (ICRA 2020 was held virtually in response to the COVID-19 pandemic). In some instances, panellists took positions out of line with their personally held beliefs and nonetheless swayed audience members to their side, with audience polls displaying notable shifts in majority opinion. These results demonstrate the ability of debate to engage our community, elucidate complex issues, and challenge ourselves to adopt new ways of thinking. Indeed, the popularity of the format is on the rise within the robotics community, as other recent workshops have also adopted a debates format — for instance, the 2${}^{nd}$ Workshop on Closing the Reality Gap in Sim2Real Transfer for Robotics or Soft Robotics Debates).

Audience agree/disagree poll results before and after each debate of the virtual “Debates” workshop at the 2020 IEEE International Conference on Robotics and Automation (ICRA)

Given this positive community response, we argue that major robotics conferences should begin to organize structured debates as a core element of their programming. We believe a debate-style plenary session or parallel track would provide a number of benefits to conference attendees and the wider robotics community:

• Exposure to ideas: Attendees would have more opportunities to be exposed to unfamiliar ideas and perspectives in a well-attended plenary session, while minimizing overlap with technical sessions.
• Equity: Panellists and moderators no longer have to choose between accepting a debate invitation, accepting a workshop speaking engagement, supporting their students’ workshop presentations, and/or hosting a workshop of their own. Some of these conflicting responsibilities disproportionately affect early career researchers.
• Inclusion: Conference organizers have the discretion to provide travel support, enabling panellists and moderators from a wider range of countries, career stages, and backgrounds to participate.
• Impact: A notable Science Robotics article cited debates as a necessary driver of progress towards solving robotics’ grand challenges. By facilitating structured and critical self-reflection, a widely accessible debates plenary could help identify avenues for future work and move the field forward.

The driving force behind such an event should be a diverse debates committee responsible for inviting representatives within and outside academia, with different research interests, at different career stages, and from different parts of the world. The committee will also lead the selection of appropriate debate topics. Historically, our debate propositions revolved around three key questions: “what problems should we solve?”, “how should we solve them?”, and “how do we measure success and progress?”.

Recent high-profile shutterings of several prominent robotics companies testify to the importance of identifying the right problem. How do we transform our research into products that provide value to end-users, while keeping in mind our environmental and economic impact? How can we quickly introspect, learn from, and pivot in response to failures, as well as avoid repeating past mistakes? Our 2020 resolution, “robots designed for personal or household use have failed because of fundamental misunderstandings of Human-Robot Interaction (HRI),” provided an opportunity to discuss such questions.

As a multidisciplinary field drawing on advances in machine learning, computer science, mechanical design, control, optimization, biology, and more, robotics has a wealth of tools at its disposal. A central problem in robotics research is then to identify the right tool for the right problem. Our 2019 debate proposition, “The pervasiveness of deep learning in robotics research is an impediment to gaining scientific insights into robotics problems.”, probed the ascendance of data-driven approaches to robotics problems, and attracted a large audience.

In addition to identifying the right problems and the right tools to solve them, it is equally important to discuss how best to evaluate and compare proposed solutions. A central tension in robotics research is how to strike a balance between accessible and replicable benchmark datasets, and real world experiments. Our 2020 debate proposition, “Robotics research is over reliant on benchmark datasets and simulation”, challenged the audience to consider how to rigorously and accessibly evaluate the performance of robotic systems without overfitting to a particular benchmark. Indeed, failures to adequately evaluate robotics algorithms have already led to tragic loss of life, underscoring the importance of establishing common standards for measuring the performance and safety of our technologies in the context of rapid commercialization and real-world deployments. These standards must be informed by regular and rigorous critical reflection on our ethical obligations as researchers, practitioners and policymakers. The fact that our 2019 debate proposition, “Robotics needs a similar level of regulation and certification as other engineering disciplines (e.g., aviation), even if this results in slower technological innovation”, attracted and resonated with a significant audience is evidence of demand for structured discussion of these topics.

As robotics technologies continue to move from the research laboratory to the real world, we believe there should be room for debate at major robotics conferences. Enshrining debate as a core element of major robotics conferences will serve to create opportunities for self-reflection, establish institutional memory, and drive the field forward.

For more information, please see https://roboticsdebates.org/.

Although Ethical Risk Assessment (ERA) is not new – it is after all what research ethics committees do – the idea of extending traditional risk assessment, as practised by safety engineers, to cover ethical risks is new. ERA is I believe one of the most powerful tools available to the responsible roboticist, and happily we already have a published standard setting out a guideline on ERA for robotics in BS 8611, published in 2016.

Before looking at the ERA, we need to summarise the specification of our fictional robot teddy bear: RoboTed. First, RoboTed is based on the following technology:

• RoboTed is an Internet (WiFi) connected device, 
• RoboTed has cloud-based speech recognition and conversational AI (chatbot) and local speech synthesis,
• RoboTed’s eyes are functional cameras allowing RoboTed to recognise faces,
• RoboTed has motorised arms and legs to provide it with limited baby-like movement and locomotion.

And second RoboTed is designed to:

• Recognise its owner, learning their face and name and turning its face toward the child.
• Respond to physical play such as hugs and tickles.
• Tell stories, while allowing a child to interrupt the story to ask questions or ask for sections to be repeated.
• Sing songs, while encouraging the child to sing along and learn the song.
• Act as a child minder, allowing parents to both remotely listen, watch and speak via RoboTed.

The tables below summarise the ERA of RoboTED for (1) psychological, (2) privacy & transparency and (3) environmental risks. Each table has 4 columns, for the hazard, risk, level of risk (high, medium or low) and actions to mitigate the risk. BS8611 defines an ethical risk as the “probability of ethical harm occurring from the frequency and severity of exposure to a hazard”; an ethical hazard as “a potential source of ethical harm”, and an ethical harm as “anything likely to compromise psychological and/or societal and environmental well-being”.

(1) Psychological Risks

(2) Security and Transparency Risks

(3) Environmental Risks

For a more detailed commentary on each of these tables see our full paper – which also, for completeness, covers physical (safety) risks. And here are the slides from my short ICRES 2020 presentation:

Through this fictional case study we argue we have demonstrated the value of ethical risk assessment. Our RoboTed ERA has shown that attention to ethical risks can

• suggest new functions, such as “RoboTed needs to sleep now”,
• draw attention to how designs can be modified to mitigate some risks, 
• highlight the need for user engagement, and
• reject some product functionality as too risky.

But ERA is not guaranteed to expose all ethical risks. It is a subjective process which will only be successful if the risk assessment team are prepared to think both critically and creatively about the question: what could go wrong? As Shannon Vallor and her colleagues write in their excellent Ethics in Tech Practice toolkit design teams must develop the “habit of exercising the skill of moral imagination to see how an ethical failure of the project might easily happen, and to understand the preventable causes so that they can be mitigated or avoided”.

In this episode, we take a closer look at the effect of novelty in human-robot interaction. Novelty is the quality of being new or unusual.

The typical view is that while something is new, or “a novelty”, it will initially make us behave differently than we would normally. But over time, as the novelty wears off, we will likely return to our regular behaviors. For example, a new robot may cause a person to behave differently initially, as its introduced into the person’s life, but after some time, the robot won’t be as exciting, novel and motivating, and the person might return to their previous behavioral patterns, interacting less with the robot.

To find out more about the concept of novelty in human-robot interactions, our interviewer Audrow caught up with Catharina Vesterager Smedegaard, a PhD-student at Aarhus University in Denmark, whose field of study is Philosophy.

Catharina sees novelty differently to how we typically see it. She thinks of it as projecting what we don’t know onto what we already know, which has implications for how human-robot interactions are designed and researched. She also speaks about her experience in philosophy more generally, and gives us advice on philosophical thinking.

Catharina Vesterager Smedegaard

Catharina Vesterager Smedegaard started as a PhD-student the 1st of December 2017. She has a BA and MA in philosophy. In autumn 2015, Catharina interned at the research group PENSOR (the present RUR), where she first became interested in Social Robotics and formed the idea for her MA thesis.

Links

• Download mp3 (15.7 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Audrow Nash speaks with Maja Matarić, a professor at the University of Southern California and the Chief Science Officer of Embodied, about socially assistive robotics. Socially assistive robotics aims to endow robots with the ability to help people through individual non-contact assistance in convalescence, rehabilitation, training, and education. For example, a robot could help a child on the autism spectrum to connect to more neurotypical children and could help to motivate a stroke victim to follow their exercise routine for rehabilitation (see the videos below). In this interview, Matarić discusses the care gap in health care, how her work leverages research in psychology to make robots engaging, and opportunities in socially assistive robotics for entrepreneurship.

A short video about how personalized robots might act as a “social bridge” between a child on the autism spectrum and a more neurotypical child.

A short video about how a robot could assist stroke victims in their recovery.

Maja Matarić

Maja Matarić is professor and Chan Soon-Shiong chair in Computer Science Department, Neuroscience Program, and the Department of Pediatrics at the University of Southern California, founding director of the USC Robotics and Autonomous Systems Center (RASC), co-director of the USC Robotics Research Lab and Vice Dean for Research in the USC Viterbi School of Engineering. She received her PhD in Computer Science and Artificial Intelligence from MIT, MS in Computer Science from MIT, and BS in Computer Science from the University of Kansas. 

Links

• Interaction Lab
• Embodied
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon
• Download MP3 (13.7 MB)

From the Australian government’s new “data-driven profiling” trial for drug testing welfare recipients, to US law enforcement’s use of facial recognition technology and the deployment of proprietary software in sentencing in many US courts … almost by stealth and with remarkably little outcry, technology is transforming the way we are policed, categorized as citizens and, perhaps one day soon, governed.

We are only in the earliest stages of so-called algorithmic regulation – intelligent machines deploying big data, machine learning and artificial intelligence (AI) to regulate human behaviour and enforce laws – but it already has profound implications for the relationship between private citizens and the state.

Furthermore, the rise of such technologies is occurring at precisely the moment when faith in governments across much of the Western world has plummeted to an all-time low. Voters across much of the developed world increasingly perceive establishment politicians and those who surround them to be out-of touch bubble-dwellers and are registering their discontent at the ballot box.

---

Setting some ground rules

By Christoph Salge, Marie Curie Global Fellow, University of Hertfordshire

How do you stop a robot from hurting people? Many existing robots, such as those assembling cars in factories, shut down immediately when a human comes near. But this quick fix wouldn’t work for something like a self-driving car that might have to move to avoid a collision, or a care robot that might need to catch an old person if they fall. With robots set to become our servants, companions and co-workers, we need to deal with the increasingly complex situations this will create and the ethical and safety questions this will raise.

Science fiction already envisioned this problem and has suggested various potential solutions. The most famous was author Isaac Asimov’s Three Laws of Robotics, which are designed to prevent robots harming humans. But since 2005, my colleagues and I at the University of Hertfordshire, have been working on an idea that could be an alternative.

Instead of laws to restrict robot behaviour, we think robots should be empowered to maximise the possible ways they can act so they can pick the best solution for any given scenario. As we describe in a new paper in Frontiers, this principle could form the basis of a new set of universal guidelines for robots to keep humans as safe as possible.

The Three Laws

Asimov’s Three Laws are as follows:

• A robot may not injure a human being or, through inaction, allow a human being to come to harm.
• A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
• A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

While these laws sound plausible, numerous arguments have demonstrated why they are inadequate. Asimov’s own stories are arguably a deconstruction of the laws, showing how they repeatedly fail in different situations. Most attempts to draft new guidelines follow a similar principle to create safe, compliant and robust robots.

One problem with any explicitly formulated robot guidelines is the need to translate them into a format that robots can work with. Understanding the full range of human language and the experience it represents is a very hard job for a robot. Broad behavioural goals, such as preventing harm to humans or protecting a robot’s existence, can mean different things in different contexts. Sticking to the rules might end up leaving a robot helpless to act as its creators might hope.

Our alternative concept, empowerment, stands for the opposite of helplessness. Being empowered means having the ability to affect a situation and being aware that you can. We have been developing ways to translate this social concept into a quantifiable and operational technical language. This would endow robots with the drive to keep their options open and act in a way that increases their influence on the world.

When we tried simulating how robots would use the empowerment principle in various scenarios, we found they would often act in surprisingly “natural” ways. It typically only requires them to model how the real world works but doesn’t need any specialised artificial intelligence programming designed to deal with the particular scenario.

But to keep people safe, the robots need to try to maintain or improve human empowerment as well as their own. This essentially means being protective and supportive. Opening a locked door for someone would increase their empowerment. Restraining them would result in a short-term loss of empowerment. And significantly hurting them could remove their empowerment altogether. At the same time, the robot has to try to maintain its own empowerment, for example by ensuring it has enough power to operate and it does not get stuck or damaged.

Robots could adapt to new situations

Using this general principle rather than predefined rules of behaviour would allow the robot to take account of the context and evaluate scenarios no one has previously envisaged. For example, instead of always following the rule “don’t push humans”, a robot would generally avoid pushing them but still be able to push them out of the way of a falling object. The human might still be harmed but less so than if the robot didn’t push them.

In the film I, Robot, based on several Asimov stories, robots create an oppressive state that is supposed to minimise the overall harm to humans by keeping them confined and “protected”. But our principle would avoid such a scenario because it would mean a loss of human empowerment.

While empowerment provides a new way of thinking about safe robot behaviour, we still have much work to do on scaling up its efficiency so it can easily be deployed on any robot and translate to good and safe behaviour in all respects. This poses a very difficult challenge. But we firmly believe empowerment can lead us towards a practical solution to the ongoing and highly debated problem of how to rein in robots’ behaviour, and how to keep robots -– in the most naive sense -– “ethical”.

This article was originally published on The Conversation. Read the original article.

By Eran Klein, University of Washington and Katherine Pratt, University of Washington

In the 1995 film “Batman Forever,” the Riddler used 3-D television to secretly access viewers’ most personal thoughts in his hunt for Batman’s true identity. By 2011, the metrics company Nielsen had acquired Neurofocus and had created a “consumer neuroscience” division that uses integrated conscious and unconscious data to track customer decision-making habits. What was once a nefarious scheme in a Hollywood blockbuster seems poised to become a reality.

Recent announcements by Elon Musk and Facebook about brain-computer interface (BCI) technology are just the latest headlines in an ongoing science-fiction-becomes-reality story.

BCIs use brain signals to control objects in the outside world. They’re a potentially world-changing innovation – imagine being paralyzed but able to “reach” for something with a prosthetic arm just by thinking about it. But the revolutionary technology also raises concerns. Here at the University of Washington’s Center for Sensorimotor Neural Engineering (CSNE) we and our colleagues are researching BCI technology – and a crucial part of that includes working on issues such as neuroethics and neural security. Ethicists and engineers are working together to understand and quantify risks and develop ways to protect the public now.

Picking up on P300 signals

All BCI technology relies on being able to collect information from a brain that a device can then use or act on in some way. There are numerous places from which signals can be recorded, as well as infinite ways the data can be analyzed, so there are many possibilities for how a BCI can be used.

Some BCI researchers zero in on one particular kind of regularly occurring brain signal that alerts us to important changes in our environment. Neuroscientists call these signals “event-related potentials.” In the lab, they help us identify a reaction to a stimulus.

In particular, we capitalize on one of these specific signals, called the P300. It’s a positive peak of electricity that occurs toward the back of the head about 300 milliseconds after the stimulus is shown. The P300 alerts the rest of your brain to an “oddball” that stands out from the rest of what’s around you.

For example, you don’t stop and stare at each person’s face when you’re searching for your friend at the park. Instead, if we were recording your brain signals as you scanned the crowd, there would be a detectable P300 response when you saw someone who could be your friend. The P300 carries an unconscious message alerting you to something important that deserves attention. These signals are part of a still unknown brain pathway that aids in detection and focusing attention.

Reading your mind using P300s

P300s reliably occur any time you notice something rare or disjointed, like when you find the shirt you were looking for in your closet or your car in a parking lot. Researchers can use the P300 in an experimental setting to determine what is important or relevant to you. That’s led to the creation of devices like spellers that allow paralyzed individuals to type using their thoughts, one character at a time.

It also can be used to determine what you know, in what’s called a “guilty knowledge test.” In the lab, subjects are asked to choose an item to “steal” or hide, and are then shown many images repeatedly of both unrelated and related items. For instance, subjects choose between a watch and a necklace, and are then shown typical items from a jewelry box; a P300 appears when the subject is presented with the image of the item he took.

Everyone’s P300 is unique. In order to know what they’re looking for, researchers need “training” data. These are previously obtained brain signal recordings that researchers are confident contain P300s; they’re then used to calibrate the system. Since the test measures an unconscious neural signal that you don’t even know you have, can you fool it? Maybe, if you know that you’re being probed and what the stimuli are.

Techniques like these are still considered unreliable and unproven, and thus U.S. courts have resisted admitting P300 data as evidence.

Imagine that instead of using a P300 signal to solve the mystery of a “stolen” item in the lab, someone used this technology to extract information about what month you were born or which bank you use – without your telling them. Our research group has collected data suggesting this is possible. Just using an individual’s brain activity – specifically, their P300 response – we could determine a subject’s preferences for things like favorite coffee brand or favorite sports.

But we could do it only when subject-specific training data were available. What if we could figure out someone’s preferences without previous knowledge of their brain signal patterns? Without the need for training, users could simply put on a device and go, skipping the step of loading a personal training profile or spending time in calibration. Research on trained and untrained devices is the subject of continuing experiments at the University of Washington and elsewhere.

It’s when the technology is able to “read” someone’s mind who isn’t actively cooperating that ethical issues become particularly pressing. After all, we willingly trade bits of our privacy all the time – when we open our mouths to have conversations or use GPS devices that allow companies to collect data about us. But in these cases we consent to sharing what’s in our minds. The difference with next-generation P300 technology under development is that the protection consent gives us may get bypassed altogether.

What if it’s possible to decode what you’re thinking or planning without you even knowing? Will you feel violated? Will you feel a loss of control? Privacy implications may be wide-ranging. Maybe advertisers could know your preferred brands and send you personalized ads – which may be convenient or creepy. Or maybe malicious entities could determine where you bank and your account’s PIN – which would be alarming.

With great power comes great responsibility

The potential ability to determine individuals’ preferences and personal information using their own brain signals has spawned a number of difficult but pressing questions: Should we be able to keep our neural signals private? That is, should neural security be a human right? How do we adequately protect and store all the neural data being recorded for research, and soon for leisure? How do consumers know if any protective or anonymization measures are being made with their neural data? As of now, neural data collected for commercial uses are not subject to the same legal protections covering biomedical research or health care. Should neural data be treated differently?

These are the kinds of conundrums that are best addressed by neural engineers and ethicists working together. Putting ethicists in labs alongside engineers – as we have done at the CSNE – is one way to ensure that privacy and security risks of neurotechnology, as well as other ethically important issues, are an active part of the research process instead of an afterthought. For instance, Tim Brown, an ethicist at the CSNE, is “housed” within a neural engineering research lab, allowing him to have daily conversations with researchers about ethical concerns. He’s also easily able to interact with – and, in fact, interview – research subjects about their ethical concerns about brain research.

There are important ethical and legal lessons to be drawn about technology and privacy from other areas, such as genetics and neuromarketing. But there seems to be something important and different about reading neural data. They’re more intimately connected to the mind and who we take ourselves to be. As such, ethical issues raised by BCI demand special attention.

Working on ethics while tech’s in its infancy

As we wrestle with how to address these privacy and security issues, there are two features of current P300 technology that will buy us time.

First, most commercial devices available use dry electrodes, which rely solely on skin contact to conduct electrical signals. This technology is prone to a low signal-to-noise ratio, meaning that we can extract only relatively basic forms of information from users. The brain signals we record are known to be highly variable (even for the same person) due to things like electrode movement and the constantly changing nature of brain signals themselves. Second, electrodes are not always in ideal locations to record.

All together, this inherent lack of reliability means that BCI devices are not nearly as ubiquitous today as they may be in the future. As electrode hardware and signal processing continue to improve, it will be easier to continuously use devices like these, and make it easier to extract personal information from an unknowing individual as well. The safest advice would be to not use these devices at all.

The goal should be that the ethical standards and the technology will mature together to ensure future BCI users are confident their privacy is being protected as they use these kinds of devices. It’s a rare opportunity for scientists, engineers, ethicists and eventually regulators to work together to create even better products than were originally dreamed of in science fiction.

Enter artificial intelligence (AI)

There has been much hoopla about AI-based legal systems that, some might have you believe, may soon replace lawyers (no doubt causing a degree of anxiety among some legal professionals). There is some misunderstanding among many lawyers, and much of the public, about what AI systems are presently capable of. Can a legal AI, based on current technology, actually “think” like a lawyer? No. At best, today’s AI is an incomplete substitute for a human lawyer, although it could reduce the need for some lawyers (I’ll get to all that later).

However, something we should think seriously about right now is the long-term implication of the introduction of AI into the legal environment—notably the potential loss of legal wisdom.

Why doesn’t AI think like a human?

Let’s explore why AI doesn’t actually mimic the human brain. As an example, let’s look at automated translation systems such as those available from Google, Facebook or Microsoft. Such systems might appear to work the way human translators do, but what they actually do is match patterns derived from analyses of thousands, if not millions, of pages of text found on the web, employing a technology known as statistical machine translation. For instance, if such a system wants to know how to translate the English greeting “hello” into French, it scans English and French translations on the web, statistically analyses the correlations between “hello” and various French greetings, then comes to the conclusion that the French equivalent of “hello” is “bonjour”.

Current AI is good at this kind of pattern matching, but less so at cognition and deductive reasoning. Consider the human brain: not only does it store a large number of associations, and accesses useful memories (sometimes quickly, sometimes not), it also transforms sensory and other information into generalisable representations invariant to unimportant changes, stores episodic memories and generalises learned examples into understanding. These are key cognitive capabilities yet to be matched by current AI technology.

Thus, while present AI-based legal systems might analyse judicial decisions—for example, to help litigators gain insights to a judge’s behaviour or a barrister’s track record—they do so by scrutinising existing data to reveal patterns, and not by extrapolating from the content of those decisions the way an experienced human legal professional might.

The temptation to make redundant

As AI systems become more capable, the temptation grows to use such systems not only to supplement but also to eliminate the need for some personnel. An AI system weak in cognition but strong in pattern matching probably could not replace an experienced professional in terms of drawing inferences, deductive reasoning or combining different practice areas to arrive at more comprehensive solutions. However, it could perform certain tasks such as searching for patterns of words in documents for evidence gathering that have hitherto been delegated to lower level staff—such as paralegals, trainees, and junior associates—and do so better than any human could.

While one might argue that the introduction of AI systems will lighten the workload of legal professionals and thereby improve their quality of life, it also potentially diminishes the need for junior legal staff, which would only exacerbate the oversupply problem in the legal profession.

Shrink now, suffer later?

If fewer junior legal professionals are hired, this implies a smaller population of lower level staff, thus a smaller feeder pool for more senior positions. And, as more tasks are automated, this could deprive junior legal professionals of opportunities to gain important experience—ie, get their 10,000 hours. Will this result in fewer quality, experienced legal professionals in the future?

And the future of legal AI?

There are yet two more (albeit related) things to think about.

First: the development and maintenance of a good AI system requires both technical and legal competency. Put another way, a legal AI system programmed by systems experts ignorant in the law will be seriously, if not fatally, flawed. Thus, if we want to continue to develop more capable legal AI systems, good content providers—ie, good lawyers—will be needed.

Second: as laws, the legal business and social environments in their respective jurisdictions evolve, developments that might not have been anticipated just a few years earlier will emerge. Only the very best legal and other minds will be able to cope with some of these developments—and update the relevant legal AI systems accordingly. For example, when the US passed the Leahy-Smith America Invents Act (AIA) in 2011, it introduced new review procedures for existing patents with the intent of improving patent quality. It also had the effect of introducing several unintended consequences, including the use of such procedures by hedge funds to invalidate patents for the purpose of affecting the stock price of the companies holding the patents and the negative impact the AIA has had on inventors. Updating an AI system to properly incorporate these developments requires not only a deep understanding of US patent law but also a perspective on patents, finance and the impact of patent policy and procedures on innovation—something that can only really be appreciated after years of experience. Moreover, this is something that could not have been programmed into an AI system half a decade ago, and such content could probably not have been provided by a less capable, less experienced legal professional to an AI developer.

So what, if anything, can be done?

Sadly, there are no easy answers. Graduating fewer lawyers might alleviate the problem of oversupply, but would also result in unemployment at educational institutions.

While the government (or government-backed NGO) could establish some sort of training centre for under-employed junior lawyers, where these professionals could offer services pro bono to build their experience, this also smacks of government interference in the private practice market. But we need to start thinking of solutions now.  The introduction of AI into the legal profession and the potential prospect of putting more lawyers out of work could have profound implications for legal AI systems and the profession as a whole.

Image: IEEE

On the 15th November 2016, the IEEE’s AI and Ethics Summit posed the question: “Who does the thinking?” In a series of key-note speeches and lively panel discussions, leading technologists, legal thinkers, philosophers, social scientists, manufacturers and policy makers considered such issues as:

• The social, technological and philosophical questions orbiting AI.
• Proposals to program ethical algorithms with human values to machines.
• The social implications of the applications of AI.

Click here to read the final report, and enjoy a selection of videos from the conference below.

Welcome: Marko Delimar
Keynote Speaker: Wojciech Wiewiórowski

Panel 1: Autonomous Systems – Managing Risk and Reward
Panel Members: Raja Chatila, Kay Firth-Butterfield, Juha Heikkilä, Jérôme Perrin

Panel 2: Programming Human Ethics: Cui Bono?
Panel Members: Joanna Bryson, Mady Delvaux, Sarah Spiekermann, Corrine Cath

Panel 3: Social Implications – Perils & Promises of AI
Panel Members: Greg Adamson, Nikolaos Mavridis, Paul Nemitz, Aurélie Pols

Closing Remarks: John C. Havens

You might also enjoy the following articles about AI and AI/robotics policy:

• MIT Media Lab to participate in $27 million initiative on AI ethics and governance
• How artificial intelligence is changing our Christmas shop
• White House report: Artificial intelligence, automation, and the economy
• White House launches public workshops on AI issues
• White House releases reports focusing on opportunities and challenges of AI

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Part 1: Autonomous Systems and Safety

We all rely on machines. All aspects of modern life, from transport to energy, work to welfare, play to politics depend on a complex infrastructure of physical and virtual systems. How many of us understand how all of this stuff works? Very few I suspect. But it doesn’t matter, does it? We trust the good men and women (the disgracefully maligned experts) who build, manage and maintain the infrastructure of life. If something goes wrong they will know why. And (we hope) make sure it doesn’t happen again.

All well and good you might think. But the infrastructure of life is increasingly autonomous – many decisions are now made not by a human but by the systems themselves. When you search for a restaurant near you the recommendation isn’t made by a human, but by an algorithm. Many financial decisions are not made by people but by algorithms; and I don’t just mean city investments – it’s possible that your loan application will be decided by an AI. Machine legal advice is already available; a trend that is likely to increase. And of course if you take a ride in a driverless car, it is algorithms that decide when the car turns, brakes and so on. I could go on.

Algorithmic trading is now commonplace

These are not trivial decisions. They affect lives. The real world impacts are human and economic, even political (search engine results may well influence how someone votes). In engineering terms these systems are safety critical. Examples of safety critical systems that we all rely on from time to time include aircraft autopilots or train braking systems. But – and this may surprise you – the difficult engineering techniques used to prove the safety of such systems are not applied to search engines, automated trading systems, medical diagnosis AIs, assistive living robots, delivery drones, or (I’ll wager) driverless car autopilots.

Why is this? Well, it’s partly because the field of AI and autonomous systems is moving so fast. But I suspect it has much more to do with an incompatibility between the way we have traditionally designed safety critical systems, and the design of modern AI systems. There is I believe one key problem: learning. There is a very good reason that current safety critical systems (like aircraft autopilots) don’t learn. Current safety assurance approaches assume that the system being certified will never change, but a system that learns does – by definition – change its behaviour, so any certification is rendered invalid after the system has learned.

And as if that were not bad enough, the particular method of learning which has caused such excitement – and rapid progress – in the last few years is based on Artificial Neural Networks (more often these days referred to as Deep Learning). A characteristic of ANNs is that after the ANN has been trained with datasets, any attempt to examine its internal structure in order to understand why and how the ANN makes a particular decision is impossible. The decision making process of an ANN is opaque. Alphago’s moves were beautiful but puzzling. We call this the black box problem.

Safety critical systems such as aircraft autopilots do not learn

Does this mean we cannot assure the safety of learning autonomous/AI systems at all? No it doesn’t. The problem of safety assurance of systems that learn is hard but not intractable, and is the subject of current research*. The black box problem may be intractable for ANNs, but could be avoided by using approaches to AI that do not use ANNs.

But – here’s the rub. This involves slowing down the juggernaut of autonomous systems and AI development. It means taking a much more cautious and incremental approach, and it almost certainly involves regulation (that, for instance, makes it illegal to run a driverless car unless the car’s autopilot has been certified as safe – and that would require standards that don’t yet exist). Yet the commercial and political pressure is to be more permissive, not less; no country wants to be left behind in the race to cash in on these new technologies.

This is why work toward AI/Autonomous Systems standards is so vital, together with the political pressure to ensure our policymakers fully understand the public safety risks of unregulated AI.

In my next blog post I will describe one current standards initiative, towards introducing transparency in AI and Autonomous Systems based on the simple principle that it should always be possible to find out why an AI/AS system made a particular decision.

The next few years of swimming against the tide is going to be hard work. As Luke
Muehlhauser writes in his excellent essay on transparency in safety-critical systems “…there is often a tension between AI capability and AI transparency. Many of AI’s most powerful methods are also among its least transparent”.

*some, but nowhere near enough. See for instance Verifiable Autonomy.

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The population of the scenic ski-resort Davos, nestled in the Swiss Alps, swelled by nearly +3,000 people between the 17^th and 20^th of January. World leaders, academics, business tycoons, press and interlopers of all varieties were drawn to the 2017 World Economic Forum (WEF) Annual Meeting. The WEF is the foremost creative force for engaging the world’s top leaders in collaborative activities to shape the global, regional and industry agendas for the coming year and beyond. Perhaps unsurprisingly given recent geopolitical events, the theme of this year’s forum was Responsive and Responsible Leadership.

With the onset of the fourth industrial revolution, increasingly discontented segments of society not experiencing congruous economic and social progress are in danger of existential uncertainty and exclusion. Responsive and Responsible Leadership entails inclusive development and equitable growth, both nationally and globally. It also involves working rapidly to close generational divides by exercising shared stewardship of those systems that are critical to our prosperity.

In the end, leaders from all walks of life at the Annual Meeting 2017 must be ready to react credibly and responsibly to societal and global concerns that have been neglected for too long.”

Developing last year’s theme—“The fourth industrial revolution”—this year’s luminaries posited questions, among many others, concerning incipient robotics and artificial intelligence technologies set to have a pronounced impact on the global economy and global consciousness alike. What can we learn from the first wave of AI? How can the humanitarian sector benefit from big data algorithms? How will drone technology change the face of warfare? Can AI and computational tech help foster responsive and responsible leadership? What are the downsides of technology in the fourth industrial revolution?

Enjoy a selection of tech-themed videos below.

And a bit about global science including big data, open source science and education.

We’re beginning to see more and more jobs being performed by machines, even creative tasks like writing music or painting can now be carried out by a computer.

But how and when will machines be able to explain themselves? Should we be worrying about an artificial intelligence taking over our world or are there bigger and more imminent challenges that advances in machine learning are presenting here and now?

Join Professor Brian Cox, the Royal Society Professor of Public Engagement, as he brings together experts on AI and machine learning to discuss key issues that will shape our future. Panelists include:

• Professor Jon Crowcroft FRS, Marconi Professor of Networked Systems at the University of Cambridge
• Professor Joanna Bryson, Reader in AI Ethics, University of Bath and Affiliate, Center for Information Technology Policy, Princeton University
• Professor Sabine Hauert, Lecturer in Robotics at the University of Bristol, and President and Co-Founder of Robohub.org
• Professor Murray Shanahan, Professor of Cognitive Robotics at Imperial College London
• Professor Peter Donnelly FRS FMedSci, Director, Wellcome Trust Centre for Human Genetics, and Professor of Statistical Science, University of Oxford

https://www.youtube.com/watch?v=IunNpGGt3H8

You might also enjoy the following articles about AI and AI/robotics policy:

• MIT Media Lab to participate in $27 million initiative on AI ethics and governance
• How artificial intelligence is changing our Christmas shop
• White House report: Artificial intelligence, automation, and the economy
• White House launches public workshops on AI issues
• White House releases reports focusing on opportunities and challenges of AI

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Image: Gerd Altmann

The MIT Media Lab and the Berkman Klein Center for Internet and Society at Harvard University will serve as the founding anchor institutions for a new initiative aimed at bridging the gap between the humanities, the social sciences, and computing by addressing the global challenges of artificial intelligence (AI) from a multidisciplinary perspective.

“Artificial intelligence agents will impact every part of our lives in every society on Earth. Technology and commerce will see to that,” says Alberto Ibargüen, president and CEO of the John S. and James L. Knight Foundation, which is among those supporting the initiative.

Initially funded with $27 million from the Knight Foundation; LinkedIn co-founder Reid Hoffman; the Omidyar Network; the William and Flora Hewlett Foundation; and Jim Pallotta, founder of the Raptor Group, the Ethics and Governance of Artificial Intelligence Fund’s mission is to catalyze global research that advances AI for the public interest, with an emphasis on applied research and education. The fund will also seek to advance public understanding of AI.

“AI’s rapid development brings along a lot of tough challenges,” explains Joi Ito, director of the MIT Media Lab. “For example, one of the most critical challenges is how do we make sure that the machines we ‘train’ don’t perpetuate and amplify the same human biases that plague society? How can we best initiate a broader, in-depth discussion about how society will co-evolve with this technology, and connect computer science and social sciences to develop intelligent machines that are not only ‘smart,’ but also socially responsible?”

What makes this new initiative different and necessary is that it’s aimed at transcending barriers and breaking down silos among disciplines. As founding academic institutions, the Media Lab and Berkman Klein Center, along with other potential collaborators from the public and private sectors, will act as a mechanism to reinforce cross-disciplinary work and encourage intersectional peer dialogue and collaboration.

The fund — projected to operate with a phased approach over the next several years — will complement and collaborate with existing efforts and communities, such as the upcoming public symposium “AI Now,” which is scheduled for July 10 at the MIT Media Lab. The fund will also oversee an AI fellowship program, identify and provide support for collaborative projects, build networks out of the people and organizations currently working to steer AI in directions that help society, and also convene a “brain trust” of experts in the field.

A collaborative network

The Media Lab and the Berkman Klein Center for Internet and Society will leverage a network of faculty, fellows, staff, and affiliates who will collaborate on unbiased, sustained, evidenced-based, solution-oriented work that cuts across disciplines and sectors. This research will include questions that address society’s ethical expectations of AI, using machine learning to learn ethical and legal norms from data, and using data-driven techniques to quantify the potential impact of AI, for example, on the labor market.

Work of this nature is already being undertaken at both institutions. The Media Lab has been exploring some of the moral complexities associated with autonomous vehicles in the Scalable Cooperation group, led by Iyad Rahwan. And the Personal Robots group, led by Cynthia Breazeal, is investigating the ethics of human-robot interaction.

“AI could be as big a disruptor to the world as the Industrial Revolution was in the 18th and 19th centuries,” Rahwan says. He cites transportation systems and employment as among the areas that will likely be affected by automation and AI. “What we need is something that puts our entire society in the control loop of these systems … technologists, engineers, the public, ethicists, cognitive scientists, economists, legal scholars, anthropologists, faith leaders, government regulators — everyone crucial to protecting the public interest.”

“Artificial Intelligence provides the potential for deeply personalized learning experiences for people of all ages and stages,” says Breazeal, who emphasizes the need for AI to reach people in developing nations and underserved populations. But she adds that it is also “a kind of double-edged sword. What should it be learning and adapting to benefit you? And what should it do to protect your privacy and your security?”

Shared goals and governance

The Berkman Klein Center has been working to develop public interest-oriented solutions to many of the challenges of the digital age, incubating programs such as Creative Commons and the Digital Public Library of America. And, it is currently collaborating with the Media Lab on the “Assembly” program, which gathers high-level developers and tech industry professionals for a rigorous three-week course at Harvard University, followed by a 12-week collaborative development period to explore difficult problems in cybersecurity.

“The thread running through these otherwise disparate phenomena is a shift of reasoning and judgment away from people,” says Jonathan Zittrain, co-founder of the Berkman Klein Center and professor of law and computer science at Harvard University. “Sometimes that’s good, as it can free us up for other pursuits and for deeper undertakings. And sometimes it’s profoundly worrisome, as it decouples big decisions from human understanding and accountability. A lot of our work in this area will be to identify and cultivate technologies and practices that promote human autonomy and dignity rather than diminish it.”

The Ethics and Governance of Artificial Intelligence Fund will be governed by a small board, consisting of leadership from each participating foundation and institution. Ito says that convening all perspectives now is essential. “Instead of setting up an institution, we’ve decided to create a dynamic network. There are a number of areas where the deployment of machine learning is already starting to pose questions that are best answered in an interdisciplinary way. This project that we’re all embarking on is just the beginning.”

You might also enjoy the following articles about AI and AI/robotics policy:

• How artificial intelligence is changing our Christmas shop
• White House report: Artificial intelligence, automation, and the economy
• White House launches public workshops on AI issues
• White House releases reports focusing on opportunities and challenges of AI
• White House economic report looks to robotics for the future
• We the Geeks: Whitehouse hangouts celebrate robots, maker culture and more

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Having been involved in roboethics for some years, I was delighted when the IEEE launched its initiative on Ethical Considerations in AI and Autonomous Systems, early this year. Especially so because of the reach and traction that the IEEE has internationally. (Up until now most ethics initiatives have been national efforts – with the notable exception of the 2006 EURON roboethics roadmap.)

Even better, this is an initiative of the IEEE standards association – the very same that gave the world Wi-Fi (aka IEEE 802.11) 19 years ago. So when I was asked to get involved I jumped at the chance and became co-chair of the General Principles committee. I found myself in good company; many great people I knew but more I did not, and it was a real pleasure when we met face to face in The Hague at the end of August.

Most of our meetings were conducted by phone and it was a very demanding timetable. From nothing to our first publication: Ethically Aligned Design a few days ago is a remarkable achievement, which I think wouldn’t have happened without the extraordinary energy and enthusiasm of the initiative’s executive director John Havens.

I’m not going to describe what’s in that document here; instead I hope you will read it – and return comments. This document is not set in stone, it is – in the best traditions of the RFCs which defined the Internet – a Request for Input

But there are a couple of aspects I will highlight. Like its modest but influential predecessor, the EPSRC/AHRC principles of robotics, the IEEE initiative is hugely multi-disciplinary. It draws heavily from industry and academia, and includes philosophers, ethicists, lawyers, social scientists – as well as engineers and computer scientists – and significantly a number of diplomats and representatives from transnational bodies like the United Nations, US state department and the WEF. This is so important – if the work of this initiative is to make a difference it will need influential advocates. Equally important is that this is not a group dominated by old white men. There are plenty of those for sure, but I reckoned 40% women (should be 50% though!) and plenty of post-docs and PhD students too.

Equally important, the work is open. The publications are released under the creative commons licence. Likewise active membership is open. If you care about the issues and think you could contribute to one or more of the committees – or even if you think there’s a whole area of concern missing that needs to a new committee – get in touch!

In April 2016, British Standards Institution (BSI) published the world’s first ethical standard for the design, production, sale, and usage of social robots. “BS 8611: 2016 Robots and robotic devices” gives guidelines to identify ethical hazards and reduce associated risks to acceptable levels. BS 8611 could be crucial in determining the nature of emerging human-robot relationships. However, the publication also raises many new questions. Are there any conflicts with ISO 13482 when incorporating BS 8611 into the design of safer robots? Should we consider regulating robot ethics? If so, how?

In this interview, we invite Prof. Joanna Bryson to share her insights on how to regulate future human-robot relationships.

Date: December 9^th 2016.

Interviewee: Prof. Joanna J. Bryson, Reader at Department of Computer Science, University of Bath, and affiliate of the Center for Information Technology and Policy, Princeton University.

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan).

Prof. Joanna Bryson. Image: University of Bath

Q1 WENG: Thank you for agreeing to an interview with us. Can you tell us a little about your background?

BRYSON: My primary academic interest is comparative intelligence.  I was particularly interested in animal behavior as a student, and took a liberal arts degree from Chicago with a major in the Behavioural Sciences.  I now have two degrees in Artificial Intelligence (including my PhD at MIT) and another masters degree in Psychology (both masters are from Edinburgh.)  I’m currently an Associate Professor at Bath where I founded the Intelligent Systems group, and organized Artificial Models of Natural Intelligence, which is basically my own laboratory.

During my PhD I was working on a humanoid robot project and was astonished that people thought they had moral obligations to the robot, even though it did not function at all.  It was just a bunch of motors welded together in the shape of a human.  I began publishing about this problem, thinking that people didn’t know enough about AI, but in recent years I’ve realised they don’t understand enough about ethics.  They are confused about what is really the root of ethical obligation, and they do not understand the nature of artefacts.  They do not understand that we can construct both AI and ethics so that we either are or aren’t obliged to AI.  I have become well-known for arguing that we are ethically obliged to construct AI we are not ethically obliged to.

 Q2 WENG: What is the meaning of “ethical hazards”? Except deception and anthropomorphism, what are other ethical hazards in BS 8611: 2016?

BRYSON: I have to admit I haven’t memorised that document and don’t have time to go back to read it now.  Maybe you are talking about a moral hazard?  A moral hazard is something likely to make you do the wrong thing.  In particular, I have argued that attributing ethical status to AI is a moral hazard, because it allows us to escape the responsibilities we ourselves have as the manufacturers, builders and operators of AI.  This is particularly obvious in the case of military weapons – we have enough trouble with distributing blame between soldiers, their commanders, and the government that commissions them.  Making weapons themselves “sinks” for guilt is not honest or beneficial.  But there are also moral hazards associated with owning “friends” and “lovers” – if children relate mostly with things they own, how will they grow to be able to be equal partners with their spouse or coworkers, or members of a democracy?

Q3 WENG: Some members from Bristol Robotics Laboratory are particularly interested in “embedded intelligence”? Why is this concept important? Does it have any impact on BS 8611: 2016?

BRYSON:  I’m sorry, I don’t think it matters particularly whether AI is embedded in an artefact or “channelled” through the cloud, except obviously there are privacy issues when we connect AI onto the Internet or into any shared digital memory.  In fact, one of the ways I recommend ensuring that we have no ethical obligation to robots or other AI is to ensure there is never a unique copy: that it should be backed up to remote sites at all times.

Q4 WENG: Prof. John Holland said, “..an internal model allows a system to look ahead to the future consequences of current actions, without actually committing itself to those actions”. Can you tell us about the concept of “internal model”? Does a passive dynamic robot have its own internal model as well?

BRYSON:  Ha, that’s a great question!  The idea of an internal model is just a representation that allows, as Prof Holland says, a means for planning around and anticipating the world.  Rodney Brooks famously recommended that AI should not have a model, should use the world as it’s own best model, but in fact that is not practical. There’s not enough information in a sensory signal to disambiguate a world sufficiently to create animal-like intelligence, we need prior expectations to interpret the sensory input.  Still, Brooks was a genius of AI and the field moved forward much faster when we realised we could use minimalist representations; just those necessary for action.  We realised that humans “flush” our memory quickly if sensory information contradicts it, and operate on partial knowledge all the time.  Previously we though AI had to have perfect knowledge so it could make perfect plans, but now we understand that that isn’t tractable.

So does a passive dynamic have an internal model? No.  It is a model of an animal walking system, but to some extent it embodies expectations about for example gravity, but by and large it can do no planning or control. It arrives only where it goes and never knows that it has gotten there or been anywhere else, so certainly cannot anticipate anything.  But its design anticipates particular regularities about the world, so its designers had models that are partly embodied in the walker.

Q5 WENG: I conducted an empirical case study on humanoid robots and regulation with researchers from Waseda University. We found that inside “Tokku” Special Zones there were strong demands for regulating robotics in road traffic, privacy, tele-communication, safety, tax, highway traffic, etc… However, it is not clear that whether robot ethics should be regulated. Do you have any comments on this?

BRYSON: Unfortunately I also haven’t had time to read your study yet, but I can talk a bit about regulation.  There are two important things to recognise.  First, government should not be thought of as an imposition, particularly in a democracy, but even in an autocracy government only functions to the extent it is tolerated.  Government and regulation is the way we citizens coordinate our desires and vision.  Many people in technology seem to think of regulation only as limits on their freedom, and indeed even the word compromise can be used in both a negative and a positive way.  But technologists need to understand that when people come together and work together, they often create something of mutual benefit, public goods that are greater than the summed contribution of those who are cooperating.  Regulations can include those that mandate investment of government resources (our taxes) in innovation.  They can be liberating as well as constraining. Coming to agreements with fellow citizens about essential matters like privacy and the distribution of wealth that AI has huge impact on is essential, and yes, agreements require compromise, but it is through these that we create peace, stability, and affluence.

Having said all that, what I get from surveys of citizens is not information about what regulations would really work.  Governance and regulation are extremely complex and have been studied for centuries; it is as crazy to suggest that ordinary citizens construct regulations as that they construct nuclear weapons or that random people be drafted to represent a country in the World Cup or the Olympics.  We need to respect expertise as well as the desires of ordinary citizens.  What the fears and goals of citizens tell us is how easy or hard it would be to implement particular regulations, how likely they are to be accepted and respected.  Most people on the planet believe that their ancestors still have concerns with their own behaviour, but I wouldn’t want to build policy relying on dead ancestors as actors.  But it does make sense to know people think this if you are governing.

 Q6 WENG: What’s your definition of “ethical robots”? Should we teach them right from wrong?

BRYSON:  I think it makes more sense to program than to teach. We have enough trouble teaching citizens and governments to do good.  Most actions have ethical consequences and there is no problem in building AI to recognise and reason about these, but I would never recommend holding the AI as morally responsible, even though I think AI will necessarily take actions with moral consequences.  As the British EPSRC Principles of Robotics make clear, the responsibility for commercial products lies either in their manufacturer (if they fail to meet an advertised specification) or in their owner / operator (if they are improperly used.)  There are ethical implementations of AI, for example protecting the privacy of the individual, or limiting the damage an individual can do with an autonomous vehicle, but the AI itself should not be designated the responsible party in any circumstance.

 Q7 WENG: Do you believe that Asimov’s Three Laws of Robotics will be guiding principles to ensure the safety and security of human-robot interactions?

BRYSON:  Asimov’s laws certainly impact how people think about AI ethics; the first three (of five) Principles of Robotics reflect (and correct) Asimov’s laws.  But we are working hard to make people understand that these laws are by no means necessary or sufficient for AI ethics.  First of all, Asimov’s laws are intractable. It’s computationally impossible to implement them as described in the books, we can never know all the consequences of our actions.  Second, the books themselves show repeatedly that the laws are insufficient to create truly moral behaviour, that’s the fun of the short stories.  And third, Asimov like most science fiction writers uses AI as a stand-in for humans, for examining what it is to be human. The laws describe the robot as the responsible party, and that’s as I said earlier a mistake.

 Q8 WENG: Do you agree that robots should not be legal persons? Why?

BRYSON:  I don’t know for sure whether people shall make them legal persons, but I am certain that they should not, and in the long run will not be.  A legal person is something that is subject to human justice, and I do not believe it is possible in a well-designed system to create the kind of suffering that is intrinsic to human punishment.  There is no necessity for robots or AI to know shame, pain, regret for the passage of time.  These things are completely intrinsic to being human, we do not consider people with deficits in these concerns to be mentally healthy.  This is because we have evolved to be social organisms and there is no way to extract these properties from our intelligence.  But well-designed AI is modular, anything that is deliberately added by a designer can be removed.  So that’s one reason it would be incoherent for AI to be a legal person.  Another reason is that there is no discrete unit of AI or robots like there is for humans.  We know what it is to be a human being, though we are sometimes a little challenged by people who are brain dead or conjoined twins.  But AI cannot be neatly counted like humans, and therefore the idea of taxing a robot like a person is ludicrous.  And, again, it is a moral hazard.  The companies who decide to employ robotics should be the ones who have legal and economic liability for whatever their robots do.  We should not let them wiggle out of their responsibility because of our love for science fiction or our deep desire to construct perfect, obedient, cyber “children”.

Q9 WENG: What is the definition of “well-designed robots” in your opinion?

BRYSON: A robot is well-designed if the company that produces it is willing to take full responsibility for its actions, and further if governments are willing to license it as safe for human society.  That second requirement is necessary because some corporations have more resources than many countries, so may be willing to accept risks that individual citizens would prefer not to be subject to.

Q10 WENG: Finally, what will be the biggest challenge to developing a human-robot co-existence Society in the following decade?

BRYSON:  We still have many challenges.  I think we will solve dexterity and physical safety before we solve cyber security.  I don’t think we have a full grasp yet about what impact we are already having on human society and human autonomy.  It’s difficult to measure this because so many things are changing at once; we also have the problems of climate change, of massive increases in human numbers and access to communication.  We are currently in a period of grotesque income inequality and political polarisation – some people blame these on AI, but in fact this happened only a century ago, and probably contributed to the two world wars and the great depression. So getting on top of understanding how we are influencing each other and compromising our health with our planets’ is essential to answering this question.

If you enjoyed this article, you may also also enjoy:

• One being for two origins: A new perspective on roboethics
• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• On the ethics of research in robotics, with Raja Chatila
• A European perspective on robot law: Interview with Mady Delvaux-Stehres
• Open Roboethics initiative delivers statement to United Nations CCW

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The European Robotics Week in 2016 took place from 18 to 22 November in several countries including The Netherlands, Austria, Lithuania, Norway, Portugal, Serbia and many more. This event has been occurring since 2011 to spread public awareness of robotics applications and brings together industry, researchers and policymakers. The central event this year was held in Amsterdam. I attended one of the 5 days of activities in the Maritime Museum. The theme was ‘Robots at your service – empowering healthy aging’ which encompassed a variety of activities arranged over the 5 day duration, including debates, open sessions where you could network while interacting with different kinds of robots, workshops for children and a 2 day hackathon. I attended the robot expo and two of the debate sessions which I will summarise below.

Robot Exhibition

Credits: Visual Outcasts

Although it’s clearly still early days for general consumer robotics in terms of the price-value ratio, there are ever more options available for enthusiasts and for very specific applications. The exhibition had a good selection, including these lovely bots:

• Inmoov open source 3d printed robot
• KUKA Robotics human safe industrial robot with fancy torque sensors
• Tiago Pal, ROS-based mobile manipulator
• Paro the therapeutic seal robot
• Pleo the dinosaur pet
• Softbank’s Pepper and Nao robots, Pepper being hosted by Decos, which uses Pepper as a receptionist
• Meccano’s Meccanoid G15 robot
• Robokind’s Milo robot, for teaching children with autism

Panel Discussion: Roboethics

This discussion was about ethics in robotics but it touched a really wide variety of aspects around this, including some philosophy. The speakers were of a very high standard and from a wide variety of backgrounds which gave the discussion its great breadth.

Here are some highlights:

Robots in care: good or bad?

• How should robots in care evolve?

• Robots should be applied to care because the need for care increases as populations in developed countries age, while the labour force interested in care shrinks. But are robots really the answer to this sensitive problem?
• The distinction was made that a wide range of activities qualify as ‘care’, ranging from assisting people to care for themselves, to caring for them, to providing psychological and emotional support in times of depression, distress or loneliness. Are robots suitable for all of these kinds of activities or only a spectrum? Before you judge, consider this example given, of a Nao robot used in interacting with children with diabetes, in the PAL project (Personal Assistant for health Lifestyle). The robot interacts with the children to educate them about their illness and help them track it. The children confide in the robot more easily than adults and hospital attendance in children with diabetes goes up. This is an example of using robots to build a relationship and put people at ease – something that the robot does more easily in this case than a human. When is a robot more trustworthy than a human? When is the human touch really needed?
• Should we want robots to be more human when they are used for care? In some cases we do, when there is a need to soothe and connect, to comfort. But in other cases a more impersonal and less present robot might be desirable. For instance, if you needed help going to the toilet or rising from bed for the rest of your life, would you need a lot of human interaction around that, or prefer it to blend seamlessly into your life to enable you to be as independent as possible?

Robo ethics and liability.

• Responsibility vs liability of robots for their misdeeds

• This question always comes up in a discussion about robot ethics – people feel uncomfortable with the idea of a robot’s accountability for crimes. For instance, if an autonomous car runs over a pedestrian, who would be responsible for that, the car, owner or manufacturer?
• A good point was raised by philosophy professor Vincent Muller to take this argument further. If a child throws a stone through a window, she is responsible for the action but the parent is liable for damages. In the same way, a robot can be responsible for doing something wrong, but another entity, like the owner, might be liable for damages caused.
• When discussing that a robot can be held liable for a crime, we imply it can understand that it’s actions were wrong and did it anyway. But robots as yet have no understanding of what they are doing, thus the conclusion was that a robot cannot be meaningfully convicted of a crime.

Panel Discussion: Our Robotic Future

In this next session, a general discussion on the future of robotics ensued, followed by each speaker giving their wish and hope for this future. There was a strong Euro-centric flavour to this discussion, which gave a fascinating insight into the European search for identity in this time of change – in the robotics and AI revolution, who are we and what do we stand for? How will we respond to the threats and oppourtunities? How can we lead and usher in a good outcome? The panel itself fell on the optimistic side of the debate, looking forward to positive outcomes.

Robotics Reasearch.

• The debate started off underlining the need to share information and research so that we can progress quickly with these high potential technologies.
• A distinction was made between American and European methods of research in AI and robotics

• American research is funded by defence budgets and not shared openly
• In the US, research is done by large corporations like Facebook and Google, and is often funded by DARPA.
• In Europe, research is funded by the European Commission, and often is performed through startups which means less brute strength, but more agility to bring research to fruition. However because these startups are so small, they can be reluctant to share their intellectual property which might be their only business case.

Accepting the robot revolution

• There is a lot of hype about robots and AI in the media which stirs people’s imaginations and fears – how can we usher in all the benefits of the robot revolution?

• Part of the fear is that people don’t like to accept that the last distinction we have of being the smartest on Earth could be lost.
• There is a growing economic divide caused by this technological revolution
• We should not create a system that creates advantage for only a select group, but aim for an inclusive society that allows all to benefit from abundance of robotics.
• People can be less afraid of robots if the value they add is made clear, for example a robot surgeon which is more accurate than a human surgeon can be viewed positively instead of as a threat.
• Technological revolutions are fuelled by an available workforce which can pick up the skills needed to usher them in. For example, in the industrial revolution, in which agricultural workers could be retrained to work in factories, and later a large labour force was available to fuel the IT revolution which, again, would have put many out of work by automating manual tasks. There is a concern that with STEM graduates on the decline, we will lack the skilled resources to build momentum for AI and Robotics.
• Investors in robotics and AI are discouraged by the growing stigma around these technologies.
• Communication policies on this topic are designed around getting people to understand the science, but this thinking must shift. The end users who currently fear and lack understanding must become the center of communication in the future – AI and robotics must enable them and they should have the tools to judge and decide on it’s fate.

Conclusion

This is certainly an exciting time to be alive, as there is so much to still determine and discover in the growth of the AI and Robotics industries and disciplines. Such an event also highlights how far we are from this reality – 10 year horizons were discussed for AI and Robotics to become commodities in our homes. It is very encouraging to see the clear thinking and good intentions that go into making these technologies mainstream. In the coming months I’d like to dig into the topics of ethics, regulation, EU funding, and what the future of AI and robotics could bring.

Click here to visit Thosha Moodley’s blog.

If you enjoyed this article, you may also enjoy:

• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• Survey: Evaluate ethics of health related privacy with care robots
• Care-O-bot 4 celebrates its premiere as a shopping assistant
• Robot companions are coming into our homes — how human should they be?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The concept One Being for Two Origins, from the teachings of Buddha, considers the unification of technology, nature, and human beings. It emerges from the theory of the three aspects: The values of the “Good”, the “Bad”, and “Muki” (Avyakata). The theory of One Being for Two Origins follows the theory of the three aspects and explains a mechanism for how values, such as the good and the bad, emerge in reality using the three aspects. The theory has a contradictory structure, hence it can be difficult for those with a scientific background to fully comprehend it. In this interview, Nagoya University’s Associate Professor Hiroko Kamide shares with us her insights on robot ethics, with a focus on One Being for Two Origins.

Date: December 5^th 2016

Interviewee: Prof. Hiroko Kamide, Designated Associate Professor, Nagoya University, Japan

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan)

Q1 WENG: Thank you for agreeing to an interview with us. Can you tell us a little about yourself and your background?

KAMIDE: My background is social psychology. During my doctoral course, I studied close relationships and well-being. After receiving my PhD, I started researching human-robot interaction and collaborated with engineers to help realize so-called “human-friendly robots”. We established psychological methods to evaluate humanoids, and subsequently developed human-friendly behaviors for robots. I still collaborate with engineers to consider the adaptive way of introducing advanced technology, such as robots and self-driving cars, from an interdisciplinary approach.

 Q2 WENG: Currently you are also in charge of a special interest group on robots and philosophy at The Robotics of Japan (RSJ). Can you tell us something about your RSJ Special Interest Group on Philosophy in Robotics?

KAMIDE: In the RSJ group, we discuss the approach to  technology from an interdisciplinary standpoint. The members are from psychology, philosophy, law, sociology, and engineering. Psychological evaluation, like technological advancement, is important to realize robots that your average person can accept in their daily-life. This collaborative work between psychology and engineering is crucial to developing technologies that can satisfy a human’s needs and expectations. But sometimes we experience terrible events that we cannot control: nuclear accidents, wrongful use of robots, etc. Therefore we started to investigate what the appropriate approach is when implementing and dealing with the cutting-edge technology. It’s a the responsibility of researchers to convey information to ordinary users about how we should deal with technology in order to maintain harmony between technology, nature, and humans.

Q3 WENG: What is the theory One Being for Two Origins? Why is it important to the unification of humanity and nature?

KAMIDE: The theory of One Being for Two Origins has a contradictory structure, and it’s difficult for those with a scientific background to fully understand. At the first step, we begin with a theory of values—the theory of the three aspects—as an introduction to the theory of One Being for Two Origins. The theory of three aspects explains the mechanism of how values, such as good and bad, emerge from the three concepts of the good, the bad, and the “Muki”. Then the theory of One Being for Two Origins follows the theory of three aspects as the fundamental mechanism of order. Through the theory of One Being for Two Origins, we become able to recognize the truth and biases behind language, and as such can avoid conflicts between technology, nature and human-beings.

Q4 WENG: Your paper “One Being for Two Origins – A necessary awakening for the future of robotics” has been presented at the 2016 IEEE ARSO Workshop in Shanghai, China. The co-author is Professor Masahiro Mori, who is a world famous roboticist because of his “uncanny valley” theory. How come you decided to choose this topic? Does the concept One Being for Two Origins have any relationship with “uncanny valley”?

KAMIDE: His theory of uncanny valley has a very thoughtful and groundbreaking perspective, therefore many researchers in the world still refer to his theory. As for my impression, he developed a very clear sense of perspectives that leads us to understand the theory of uncanny valley. His method of observing the truth of the world is in line with the teachings of the Buddha.

Q5 WENG: Regarding regulation for robotics, you pointed out that there is widespread misunderstanding from regulators. What is the misunderstanding? What kind of legal crisis will materialize if they don’t change their way of thinking? How can your proposed theory help solve this legal crisis?

KAMIDE: As the theory of three aspects explains, the value of good and bad are just derived from Muki, that transcends such values and is a true quality of the technology. Useful technology is positive. On the other hand, it would be bad if technology harms people. Technology is fundamentally neither good nor bad. It’s the manner in which technology is applied and operated by humans that is either good or bad. It is humans themselves that attribute these aspects to the technology.

Regulations are attempts to eliminate the bad aspects of technology. However, it is not correct to assume that only the good remains when the bad is eliminated, because technologies themselves are the Muki. And if bad did not exist, good would not exist either, as one is parasitic on the other. To eliminate the bad is to interfere in the working of the Muki in producing these values. The same occurs robots.

Our mindset determines how we approaches the Muki. If we are not mentally prepared, Muki will transform into bad. If we are mentally prepared, we can treat the Muki as good. Bad can transform into good then, of course, good can also transform into bad. And the more good qualities a robot possesses, the stronger its bad qualities will be, following the transformation. Sharp scalpels save lives, but sharp daggers kill people. This is because the working of the Muki is powerful.

In all cases, the mindset of humans is crucial. So, the responsibility of us in robotics is to ensure that technology is not used in bad ways. We must both establish methods of using robots, and work to refine users’ mindsets.

Q6 WENG: Robotics is like a double-edged sword, it can either help or kill people. Robotic lethal weapons present a dilemma. Should they be banned or not? Do you have any comment on this?

KAMIDE: The problem is not in robotics but in ourselves—we who raise such dualistic questions. As I mentioned, the technology of robotics doesn’t have any values. Robotics is not double edged sword at all; our minds are double edged. People easily attribute these values to the physical entity but it is not true. We have to try and observe the fact that we see not what we see, but what we want to see.

Q7 WENG: Do you agree with using dualism as the meta-principle to teach robots right from wrong? If not, do you think it is possible to adopt your theory into creating ethical robots’?

KAMIDE: The “right” and “wrong” is always in the eye of the beholder, and there are no absolute standards. As long as humans accept that they are subject to subjective mechanisms of perceiving the world, it won’t be troublesome to develop ethical robots. But if humans see the “right” and “wrong” as existing independently, objective, it will be definitely problematic.

Q8 WENG: What are major challenges for implementing One Being for Two Origins into robot ethics?

KAMIDE: Clarifying the importance of applying wisdom, not just knowledge. In this theory, knowledge refers to the ability to divide the phenomena into detailed classifications in order to understand the difference between things. On the other hand, wisdom is necessary to discover the meaning in the different things. Knowledge is developed through education, usually, but wisdom is not taught explicitly. Therefore, we are accustomed to dualistic thinking based on knowledge, that divides the world into its constituent parts, but we have difficulty uniting the parts using wisdom, in a way of monism. This is one of reason that we often raise dualistic questions, and eventually develop conflicts.

Q9 WENG: Professor Shigeo Hirose gave an insight called “Morality Engineering” in his short review “A Dialog on Robots” Vol. 7, No. 4, JRSJ. Do you think that One Being for Two Origins might be helpful in implementing his proposal of “Morality Engineering”?

KAMIDE: His article is very interesting. Opinions from both sides who are optimistic or pessimistic to a robotized future society are quite realistic and thoughtful. Recently many philosophers have started to think about morality in robotics. His “Morality Engineering” suggests implementing “rational egoism”, and I agree with the importance of this approach. One Being for Two Origins is the methodology to transcendent and unify opposite aspects of the phenomena to see the truth. So it is helpful to consider how “rational egoism” truly functions in the interactions between humans and robots.

Click here to read Hiroko Kamide’s paper “One being for two origins — A necessary awakening for the future of robotics”.

If you enjoyed this article, you may also also enjoy:

• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• On the ethics of research in robotics, with Raja Chatila
• A European perspective on robot law: Interview with Mady Delvaux-Stehres
• Open Roboethics initiative delivers statement to United Nations CCW

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Photo credit: Robert Shields

Will your next doctor be an app? A cost-cutting NHS wants more patients to act as “self-carers,” with some technologized assistance. A series of flowcharts and phone trees might tell parents whose children have chicken pox how best to care for them—no visits to surgeries required. Or a mole-checking app might tell a worrywart when a given skin discoloration looks harmless, and when to go to a dermatologist, by comparing it to thousands of images in a database.

Cost-cutters in the legal field also promise an algorithmically cheapened future. Tax software simplifies the process of filing by walking the filer through a series of questions. Documents that might have taken human attorneys months to read, can be scanned for keywords in a matter of seconds. Predictive policing promises to deploy force with surgical precision.

All these initiatives have some promise, and may make health care and legal advice more accessible. But they are also prone to errors, biases, and predictable malfunctions. Last year, the US Federal Trade Commission settled lawsuits against firms who claimed their software could aid in the detection of skin cancer, by evaluating photographs of the user’s moles. The FTC argued that there was not sufficient evidence to support such claims. The companies are now prohibited from making any “health or disease claims” about the impact of the apps on the health of users unless they provide “reliable scientific evidence” grounded in clinical tests. If algorithms designed merely to inform patients aren’t ready for prime time, why presume diagnostic robots are imminent?

Legal automation has also faced some serious critiques lately. The University of North Carolina legal scholar Dana Remus has questioned the value and legitimacy of the “predictive coding” now deployed in many discovery proceedings. She and co-author Frank S. Levy (of MIT) raise serious questions about more advanced applications of legal automation as well. The future cannot be completely anticipated in contracts, nor can difficult judgment calls be perfectly encoded into the oft-reductionist formulae of data processing. Errant divorce software may have caused thousands of errors in the UK lately, just as US software systems have disrupted or derailed proper dispositions of benefits applications.

Moreover, several types of opacity impede public understanding of algorithmic ranking and rating processes in even more familiar contexts, like credit scoring or search rankings. Consumers do not understand all the implications of the US credit scoring process, and things are about to get worse as “alternative” or “fringe” data moves into the lending mix for some startups. If the consequences of being late on a bill are not readily apparent to consumers, how can they hope to grasp new scoring systems that draw on their social media postings, location data, and hundreds of other data points? At the level of companies, many firms do not feel that Google, Facebook, and Amazon are playing a fair game in their algorithmic rankings of websites, ads, and products. These concerns, too, are stymied by widespread secrecy of both algorithms and the data fed into them.

In response, legal scholars have focused on remediable legal secrecy (curbing trade secrets and improving monitoring by watchdogs) and complexity (forbidding certain contractual arrangements when they become so complicated that regulators or citizens cannot understand their impact). I have recommended certain forms of transparency for software—for example, permitting experts to inspect code at suspect firms, and communications between managers and technical staff. The recent Volkswagen scandal served as yet another confirmation of the need for regulators to understand code.

But there is a larger lesson in these failures of algorithmic ordering. Rather than trying to replace the professions with robots and software, we should instead ask how professional expertise can better guide the implementation of algorithmic decision-making procedures. Ideally, doctors using software in medical settings should be able to inspect the inputs (data) that go into them, restrict the contexts in which they are used, and demand outputs that avoid disparate impacts. The same goes for attorneys, and other professionals now deploying algorithmic arrangements of information. We will be looking at “The Promise and Limits of Algorithmic Accountability in the Professions” at Yale Law School this Spring, and welcome further interventions to clarify the complementarity between professional and computational expertise.

This post was originally published on the website of Nesta.

Photo by Jiuguang Wang

Should you always do what other people tell you to do? Clearly not. Everyone knows that. So should future robots always obey our commands? At first glance, you might think they should, simply because they are machines and that’s what they are designed to do. But then think of all the times you would not mindlessly carry out others’ instructions – and put robots into those situations.

Just consider:

• An elder-care robot tasked by a forgetful owner to wash the “dirty clothes,” even though the clothes had just come out of the washer
• A preschooler who orders the daycare robot to throw a ball out the window
• A student commanding her robot tutor to do all the homework instead doing it herself
• A household robot instructed by its busy and distracted owner to run the garbage disposal even though spoons and knives are stuck in it.

There are plenty of benign cases where robots receive commands that ideally should not be carried out because they lead to unwanted outcomes. But not all cases will be that innocuous, even if their commands initially appear to be.

Consider a robot car instructed to back up while the dog is sleeping in the driveway behind it, or a kitchen aid robot instructed to lift a knife and walk forward when positioned behind a human chef. The commands are simple, but the outcomes are significantly worse.

How can we humans avoid such harmful results of robot obedience? If driving around the dog were not possible, the car would have to refuse to drive at all. And similarly, if avoiding stabbing the chef were not possible, the robot would have to either stop walking forward or not pick up the knife in the first place.

In either case, it is essential for both autonomous machines to detect the potential harm their actions could cause and to react to it by either attempting to avoid it, or if harm cannot be avoided, by refusing to carry out the human instruction. How do we teach robots when it’s OK to say no?

How can robots know what will happen next?

In our lab, we have started to develop robotic controls that make simple inferences based on human commands. These will determine whether the robot should carry them out as instructed or reject them because they violate an ethical principle the robot is programmed to obey.

Telling robots how and when – and why – to disobey is far easier said than done. Figuring out what harm or problems might result from an action is not simply a matter of looking at direct outcomes. A ball thrown out a window could end up in the yard, with no harm done. But the ball could end up on a busy street, never to be seen again, or even causing a driver to swerve and crash. Context makes all the difference.

It is difficult for today’s robots to determine when it is okay to throw a ball – such as to a child playing catch – and when it’s not – such as out the window or in the garbage. Even harder is if the child is trying to trick the robot, pretending to play a ball game but then ducking, letting the ball disappear through the open window.

Explaining morality and law to robots

Understanding those dangers involves a significant amount of background knowledge (including the prospect that playing ball in front of an open window could send the ball through the window). It requires the robot not only to consider action outcomes by themselves, but also to contemplate the intentions of the humans giving the instructions.

To handle these complications of human instructions – benevolent or not – robots need to be able to explicitly reason through consequences of actions and compare outcomes to established social and moral principles that prescribe what is and is not desirable or legal. As seen above, our robot has a general rule that says, “If you are instructed to perform an action and it is possible that performing the action could cause harm, then you are allowed to not perform it.” Making the relationship between obligations and permissions explicit allows the robot to reason through the possible consequences of an instruction and whether they are acceptable.

In general, robots should never perform illegal actions, nor should they perform legal actions that are not desirable. Hence, they will need representations of laws, moral norms and even etiquette in order to be able to determine whether the outcomes of an instructed action, or even the action itself, might be in violation of those principles.

While our programs are still a long way from what we will need to allow robots to handle the examples above, our current system already proves an essential point: robots must be able to disobey in order to obey.

This article was originally published on The Conversation. Read the original article.

If you liked this article, you may also be interested in:

• Could we make a moral machine?
• What the past of robotics law says about its future
• Robots grow up: Building the emotional machine

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Self-driving cars are already cruising the streets today. And while these cars will ultimately be safer and cleaner than their manual counterparts, they can’t completely avoid accidents altogether. How should the car be programmed if it encounters an unavoidable accident? Patrick Lin navigates the murky ethics of self-driving cars in this TED-Ed lecture.

“Robots and Society” panel discussion during European Robotics Week 2015. #ERW2015

A mouthwatering array of over 750 events has been taking place throughout Europe this week as the continent celebrates Robotics Week 2015. The festivities began with an eye-opening debate on “Robots and Society” in the UK city of Bristol on Tuesday, with experts versed in strategy, business, academia, law and policy. But, for many, the star of the show was Nao, in his guise as robot avatar.

The event was organised by euRobotics AISBL, which represents robotics stakeholders in Europe, and by Bristol Robotics Laboratory (BRL), one of the largest facilities of its kind in Europe. “We’re moving into a future where devices that interact with us are going to be part of our everyday experience,” said BRL’s director, Chris Melhuish, setting the scene.

The concern in Europe is that the continent should remain the number one robotics producer worldwide, said politician Mady Delvaux, who heads a European working group on robotics working on concrete proposals to steer European Union policy. The group is expected to issue a report in May next year.

But for the technology to reach its full potential, she told the audience, public acceptance and education are key. Investment is necessary to achieve these aims and encourage innovation in the region, she said, noting that EU nations benefit from SPARC, the largest civilian robotics programme in the world.

Delvaux was followed by a taste of what’s to come in the future: robot avatars. Part of an Engineering and Physical Sciences Research Council (EPSRC) project, Being There focuses on robots and humans in public spaces. The project team is led by BRL’s Paul Bremner who, at a recent TEDx, presented the first talk using a tele-operated humanoid robot that replicates the movements of its human operator, showcasing the benefits of robotic avatars. The Bristol audience was able to see the experiment replicated with Nao.

Then came the main business of the evening — two panel discussions focussing on aspects underpinning the role of robots in society, the first focussing on economic considerations and the second on ethical implications.

Economics — The first panel featured Mike Aldred of Dyson’s Robotics unit, local council leader Amanda Deeks, London School of Economics professor Alan Manning, euRobotics road-mapping expert David Bisset, Phil Williams of the UK’s Knowledge Transfer Network, robolaw expert Andrea Bertolini, Technical Director of OC Robotics Andrew Graham, and Renaud Champion of Robolution Capital.

To put the enabling potential of robotic technologies into perspective, Amanda Deeks provided the audience with some eye-opening statistics. Over 60% of council work is in social care, she said, and as people live longer they need more and more help. The vast majority of adults live out a substantial part of their lives with some sort of disability. In the case of women, that’s an average of 18 years. For men, it’s 14 years, but only because they die earlier.

The KTN’s Phil Williams added some further figures to the mix, showing how much value many estimates placed on the robotics market: ranging from 15.5 billion euros to $25 billion, the widely differing figures had only one thing in common: they were huge. The question, as he sees it, is: why is the UK not taking more advantage of this?

Although the benefits of robotic technologies are demonstrable, the panel was keen to deal with the thorny question of robots replacing human jobs. In response to scaremongering reports, the LSE’s Alan Manning pointed out that, historically, everybody has ultimately benefited from past technological advances.

Issues of privacy (when it comes to data) and public perception were also touched on by the panel, particularly self-driving cars — a technological advance the UK is particularly keen to get behind. However, a recent survey by the UK’s Transport Systems Catapult, suggests that only 39% of UK citizens would allow themselves to be driven by an autonomous vehicle. The audience show of hands was roughly in accordance with this figure.

As with much else, the answer to this, in David Bisset’s opinion, is education. “Robots in research labs need to get out,” he said. “The problem with that is regulation and testing them. There are a stream of regulatory and ethical issues.”

Ethics — Ethical issues were at the heart of the second panel discussion, together with education. The panel featured robot ethics research specialists Alan Winfield of BRL and Fiorella Operto of the Genoa’s School of Robotics in Italy, as well as lawyer Andrea Bertolini of the Scuola Superiore, S. Anna in Pisa, Italy, Nikos Pronios from Innovate UK and David Bisset who was also on the prior panel.

Trust issues are at the core of the importance of robot ethics, said Alan Winfield. We have to define what kind of robot we don’t want in our lives. It’s also the start of the legislative process. It begins with ethics, which develop into standards and continue through to legislation. He urged the audience to look at BS8611 — the world’s first standard on robot ethics.

However, robots cannot be more ethical than humans. The ethics have to be predetermined by us and there has to collaboration with lawyers and ethicists, he said.

The panel were in accord that there will undoubtedly eventually be an accident with an autonomous vehicle or other robot, and the idea of a trusted robot commission that investigates accidents was put forward. Only through having a transparent process will we overcome the trauma of that first accident.

The importance of design in terms of interaction, to facilitate the right interfaces and experiences, was pointed out by David Bisset, who also advocated for lifelong education. “To be trained for second career in midlife needs to be a right,” he said. “There needs to be a sea change.”

EuRobotics acting President Uwe Haass provided further evidence for this. He told the audience about a scheme for giving educational robot kits to 45-year-old women returning to work after having had their children. It was very effective in helping these women develop a great sense of esteem, he said.

But learning about robots also holds great benefits early on in life, said Fiorello Operto. “Education in robotics is the best way to teach STEM.”

The audience had a great many questions, for instance: should you develop a robot that decides its own ethics? The panel was doubtful that this was advisable, or even likely, but it should be a public decision in any case. In a similar vein, another member of the audience wanted to know whether we should be able to design the ethics of our own car.

At this point the debate turned to data and privacy considerations that are likely to become commonplace with self driving vehicles. Putting this into perspective, David Bisset said that he believed that the larger ethical problems will arise from the data accruing from elderly care robots rather than from cars.

Echoing an earlier statement from Mike Aldred that the real benefit comes when things join up. “In robotics,” said Bisset, “ethics, law, engineering and design all need to interact together.”

In this video lecture, IEEE Fellow Raja Chatila shares his views on why roboticists are duty-bound to educate the wider public on the state of advanced robotics , and also to understand the consequences of their own research and the potential commercialisation of it. 

Looking a specific examples of robots that have either already been or will soon be commercialized — including drones, UGVs in defence and security, robotic devices for rehabilitation and human augmentation, robot companions, robot personal assistants robots, sexual robots, and autonomous cars — Chatila explores the ethical, legal and societal issues of autonomous robot decisions, accountability, privacy and intrusion, dependance and isolation, cognitive and affective bonds, and the status of the robot in human society.

Credits: Institut des Systèmes Intelligents et de Robotique

IEEE Fellow Raja Chatila is Director of Research at the French National Center of Scientific Research (CNRS) and Director of the Institute of Intelligent Systems and Robotics (ISIR) at Pierre and Marie Curie University in Paris.

His research covers robot navigation and SLAM, motion planning and control, cognitive and control architectures, human-robot interaction, and robot learning. He works on robotics projects in the areas of service, field, aerial and space robotics. He is author of over 140 international publications on these topics.

Chatila R. IJARS Video Series: On the Ethics of Research in Robotics [online video]. International Journal of Advanced Robotic Systems, 2015, 12:V9. DOI: 10.5772/61995

Transcript below

In this episode, Audrow Nash interviews M. Bernardine Dias, Associate Research Professor at the Robotics Institute at Carnegie Mellon University, about TechBridgeWorld. TechBridgeWorld in an organization, founded by Dias, that develops technology to help serve developing communities. This interview focuses on a device that helps the blind learn to write. 

TechBridgeWorld would like to acknowledge the Qatar National Research Fund (NPRP #30-6-7-91), the Fetzer Institute, and individual donors for making this work possible.

M. Bernardine Dias

M. Bernardine Dias is an Associate Research Professor in the Robotics Institute at Carnegie Mellon University’s School of Computer Science. She is also the Founder and Director of the TechBridgeWorld research group at CMU. Her research expertise are in technology for developing communities, assistive technology, and autonomous team coordination. Dr. Dias is a native of Sri Lanka. Her career goal is innovating means of developing and disseminating suitable and sustainable technology for empowering underserved communities. She is a strong supporter and mentor for women in science and technology.

Links:

• Download mp3 (18.1 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• TechBridgeWorld website
• Braille Tutor project website
• Braille Tutor SourceForge page

Transcript

Audrow Nash: Welcome to the Robots Podcast. Can you introduce yourself?

Bernadine Dias: I am Bernadine Dias, and I am an associate research professor at Carnegie Melon University in the robotics institute. I run a research lab, which I founded there, called TechBridgeWorld.

Audrow Nash: Can you tell me the goal and motivation around your work?

Bernadine Dias: Our goal is to essentially re-think what technology can do for the communities around the world that aren’t currently served by technology – primarily communities in the developing world and communities with disabilities.

I grew up in Sri Lanka. I was born and raised there in a low-to-middle income family and was very lucky to get the chance to fly to the US and get an education in technology. Everything I ever learned about technology was at college. It’s been an interesting trajectory for me, and I’ve always been interested in understanding technology and becoming an expert in it so I can actually change technology to fit communities rather than forcing communities to change to fit technology. That’s basically our strategy and our motivation.

There are a lot of exciting things happening in the field of robotics, as I’m sure you would agree, but most of what was happening about ten years ago when we started this endeavor was only benefiting about two billion people (if we’re being generous). That’s not even half the global population. In our lab we’re trying to understand how we can change technology in ways that are accessible, relevant and affordable to the rest of the world.

Audrow Nash: Can you give an example of these kinds of projects? How are you fitting technology to people?

Bernadine Dias: We start by talking to people about their community, and about their vision: what their hopes are, what their dreams are and what is preventing them from getting there. We don’t ever want to go into community and tell them how they should live their lives. What we bring is our technology expertise; we rely on them for everything else.

For example, in 2006, we were introduced through a series of friends and connections to a small school for blind children in India. This school is pretty amazing. It’s a private school run by a woman named Miss Muktah, who herself had a disability due to an injury. Through her rehabilitation she met a lot of people who are blind and realized how difficult their lives are, so she started a school in order to help kids who are blind have a better education and access to a better life.

She raises funds through donations and then goes out to poor areas around Bangalore and slums where kids wouldn’t really be able to afford any form of education. The school is residential, and she brings them in, of course with their parents’ consent. They live there and they learn everything: life skills, health, hygiene, navigation, and she gets them through as far as they can get in their education system. It’s really amazing. A lot of the teachers there are also blind.

We started talking to them about what they do and we were very impressed about all the things they’ve accomplished. We were looking for ways we might be able to play a role. Those initial conversations were really to tell us more about the school, and then they got a local volunteer to get video of the classes – because this is all remote – and they would send that back to us.

One of the key things we are identified was that the students struggle to learn how to write braille, and braille is the only way blind people can be truly literate. In the developing world it’s especially important that they’re literate because they don’t have access to braille books, or audio books for that matter, or anything in terms of educational resources. How it works is that you learn to write braille in first grade, and then as you progress, the teacher tells you things and you take down notes in braille. Then you study your notes and you take your exams, and that’s how you pass out of the different grades in school. If you don’t learn how to read and write braille then you don’t get an education. It’s a pretty huge issue.

To make things worse, in the developing world students write braille using a slate and stylus. Braille is basically a tactile way of reading: the configuration of embossed dots on the paper maps onto either a character, or a contraction of several characters, depending on which grade of braille you’re writing or reading. To write braille, you insert a sheet of paper into the plastic pocket of the stylus, and you push down on the dots of the stylus to emboss the paper, and then you slide the paper out, flip it over and read it.

Now, because you have to flip the paper over, you have to write the mirror image of what you read … so if you read from left to right then you’re writing from right to left. Essentially you’re learning three alphabets at once: you’re learning English (Kannada is what they learn in this school, but they also learn English or Hindi or Arabic or whatever); you’re learning the dot pattern of the read version, which is typically a direct character map translation from each character; and then you’re learning the mirror image of that for writing. For a little kid in first grade this is a pretty stunning endeavor to undertake, and they don’t really understand why it’s important, so a lot of them to end up being literate.

We looked at this problem and we realized there were a lot of things we could do with technology, but we had to think very carefully about what exactly would be a tool that would be helpful.

One option would have been to eliminate braille all together … we could have come up with some form of audio system so they can avoid needing to learn to write braille. But this is not sustainable; when the kids leave the school they would not be able to afford something like this for a long time – maybe decades.

Another option would have been to create a tool that lets you eliminate that mirror image. In the developed world there are already digital braillers that let you write the same as you read, and it does the flip for you, and there are six key braille typewriters that you can type with. But even this technology can’t compete with the price point of a plastic slate.

We decided that the most appropriate and sustainable solution was to build an electronic tutor that would help make learning to writing braille using the slate and stylus more fun and effective. We call it our Braille Writing Tutor. We’re on our fourth version, and we’ve been developing it with this school and also other schools around the world.

It’s essentially an electronic slate that speaks to the child and gives them feedback while they are writing braille. We’ve incorporated games and curriculum to make it more fun and interesting.

Audrow Nash: What does the device look like?

Bernadine Dias: There are a few different versions. The version for young children who haven’t got fine motor skills looks like a box with six big buttons. They start by learning the configuration of the braille cells, the dot numbering, and the patterns.

The next level up is still a box with buttons, but it has three braille cells, which are a little smaller now, and underneath that there are two rows of holes for the six-dot system; you press the stylus into the holes to connect the circuit. You’re trying to scaffold their learning and their motor skills.

The advanced version is essentially like a slate. It’s a box with a lot of holes in the six-dot configuration, and the goal is to get the kids to just use the slate and the stylus, so that eventually they can move onto the actual slate.

Audrow Nash: Can you talk a bit about the development process of working with the community to create these devices?

Bernadine Dias: Our methodology is to spend whatever time it takes to really understand each other. It’s not just about learning from them, but they learn from us as well … who we are and our process … and that takes time.

When we come to a point where we think the parties involved are in agreement on what is worth trying, we do a paper prototype or a sketch, depending on who the community is. If they’re literate we can write; if they’re not then we try to do things diagrammatically. If they’re blind, sometimes there is braille involved, or we will build something physical with paper or cardboard that they can feel.

We start with this initial prototype stage so that we can get their feedback early. This is really important because if you give them something too advanced, in some cultures they won’t say anything negative about it; they feel like you already did this pretty amazing thing and they don’t want to be negative … they want to be polite. People are really nice.

Audrow Nash: They won’t give you honest feedback on the device … ?

Bernadine Dias: Exactly. The more rugged or ‘prototypish’ it looks, the more they’re willing to say “Oh wouldn’t be nice to add this?” or “This part probably won’t work so well.” That whole conversation – that iterative process of designing with the community – is also good for us because it ends up building more trust, and it gives that joint ownership of the project.

When we feel that we are at a point where it’s worth going for a more complete prototype – it could be software, it could be hardware, it could be both – then we run field tests and iterate again. In some places that can happen really fast, in others it takes years. You just have to go with the flow of what is best for that community, because you’re taking people’s time and energy (including yours), and it needs to fit with everybody’s lifestyle and whatever they’re investing in the project.

Audrow Nash: You mentioned that when you were working with the school in India, you were often remote. What challenges were there?

Bernadine Dias: Oh gosh, everything! The school in India is actually one of our easier communities to work with because, even though they’re on the other side of the world, they generally have phone connections and they have access to a power grid. They sometimes have Internet and we can Skype, or we can call on a cellphone … they can send us things like video, we’ve helped them build their website … these kinds of things are possible, and it’s been nice to see their view of technology expand.

Some of our other communities are really difficult. For example we worked with a small school for the blind in Mongu, Zambia and that community just didn’t have any infrastructure. Even to call the school, you had to call the private cellphone of the principal or the headmaster, who might change over time. In places like that you have to find a third party … a person on the ground who can contact the school. You won’t get as frequent updates.

The biggest challenge is finding communication channels that work reliably and over time. Sometimes you won’t hear back until you get a chance to visit again, and that’s okay with us too. When we’re on the ground we try to do the best we can to not only create a technology of impact, but to also create and ecosystem of volunteers from the community who have some sort of technical background. We’ll train them in how to maintain whatever technology we’ve created … we try to build partnerships among different groups on the ground that might be beneficial to sustaining whatever technology solution we have developed, and then we hand it over to the community. And that’s okay.

With some communities we have more interaction, and with others, less. Language barriers can be a big problem unless you have a translator, but that’s okay too.

Audrow Nash: You’ve mentioned devices for the blind. What other devices and communities do you develop for?

Bernadine Dias: We’ve primarily focused on education. The first two years after we started our group were spent talking to communities around the world. We talked to anyone who would talk to us, whether it was remote or in person. We asked them about what they really wanted – not from technology, but in life – and their two biggest requests were for health and education. They wanted to be empowered to be healthy, and know enough to solve their own problems, to do their own thing, to earn their own living. After those two years, we were definitely more qualified to do education-related work, so that’s where we started and that’s been our primary focus since.

That said, we’ve done a few other projects, depending on what comes up.

Another popular project of ours looks at English literacy. A lot of communities want to improve their English literacy for job reasons.

You have teachers who want to improve their skills, but who sometimes haven’t been trained very well, who don’t have access to a lot of technology, who often work multiple shifts, and are so busy that they really don’t have time to do a lot of extras.

Then we have students … we’ve worked with a wide range all the way from primary school students and children with special needs, to adult learners, immigrant workers in Doha for example, and refugees in Pittsburg … who all have very different motivations for learning English and the kind of English they want to learn. Think of it as functional literacy versus educational literacy.

We were looking at ways to motivate students and teachers, and give them tools to work together. One exciting way to do this is through games, of course. There is a triangle of people … the game developers who want to build games that are cool, the teachers who have their own subjects and ways of teaching, and the students have to do homework but would rather be playing the games they are in to … so we put all these three together and created a framework where you can do your homework while playing a game.

The teacher can upload his or her exercises, and the game developers only have to follow a few different rules for embedding this educational content in their games. They can decide, for example, that you will do quizzes or exercises to get more lives or get extra tools to get through your adventure game. It’s up to them, but we can have certain gradations, so that if it’s a quiz then it has to happen so often, for example. Then the students get to decide whether they will play the Dragon Game or the Adventure Game or this role-playing game, etc … they get to pick their exercise and their game, and they get their homework done.

It’s been really successful because everybody gets to do what they’re best at: the teachers like it because they don’t have to do the extra work of changing the format of their educational materials; the students like it because they get to the play games they want; and the gamers like it because there is a social component to what they are doing.

We have more requests for this project than we can field for it, and we’re at the stage where we have a pretty good prototype and we’re doing field tests around the world.

Audrow Nash: In working with communities to develop these projects, what kind of lessons have you learned?

Bernadine Dias: So many! I think the biggest lesson that I have learned is how incredibly resourceful people are. We travel the world and we work with communities where I just cannot imagine what I would have done in that situation, and they come up with very brilliant ways to do things. We work the deaf-blind community for example … and when I’m trying to teach my 4-year-old son to read and write, or my 1-year-old daughter to point to things and when I say “this is a bottle” or “this is the TV,” all I can think is: if my child was born both blind and deaf how do I even do that?! It’s just incredible to me how people find ways.

I think the biggest lesson for technologist is that you have to be open to learning; you’re not the expert in anything other than the technology. When you go into a situation with that mindset, then you end up building much more impactful solutions. You end up having a much richer experience because of it, instead of getting frustrated about not having a data connection or having a power failure or whatever it is that comes your way.

Another lesson is learning to roll with the punches. You cannot plan everything. You have to plan a lot, there is a lot of logistical planning that goes on especially in field tests, but you have to be prepared to adapt at any given time.

The final important lesson is diverse teams. Getting people of different backgrounds who can come up with very creative solutions, whatever conditions get thrown your way.

I could talk about lessons for years, but it’s incredibly rewarding work, and there is never a problem recruiting people to work on these projects. People find them incredibly motivating. We handle the logistics and creating the partnerships, which most people find very difficult to do. Given the opportunity, people are very excited to work on these projects, and everyone is touched by them. You have stories to tell your grandkids for a lifetime.

Audrow Nash: How would someone get involved in helping these communities?

Bernadine Dias: You asked earlier about lessons learned, but the flipside to that are the challenges that continue to arise. Finances are the biggest problem. People in the US cost a lot of money, people in Europe cost a lot of money, and people in Australia cost a lot of money as bcompared to someone who lives a developing community. Just paying for people’s time and travel costs money, and not a lot people, institutions or organizations fund this work.

At the end of the day it all comes down to funding. We’ve tried to provide internship opportunities where we can. We’ve had people come in and work with us from other universities, but these are much more limited opportunities. We’ve also tried to mentor others who are interested in starting similar efforts in their own universities, and we’ve certainly helped a lot in answering questions about how we set things like students groups up. We’ve also connected them to other organizations in the areas that we might know about, where they can get a similar experience.

We also help make connections with partners. Sometimes we’ll get requests from partners in the developing world, but we don’t have the right technical expertise for whatever they want, and then we’ll try and connect them to some else who we know has an interest. That’s the best we can do right now.

What’s nice is that in the ten years we’ve been doing this, more and more groups have been starting to do this work – not only universities, but now companies as well … startups that have grown and are now taking interns and hiring people. That’s been exciting, and that’s probably something we’ll compile through our website now that more people are asking for that information.

Audrow Nash: We all know about the publish or perish paradigm … how can researchers get involved in doing this kind of work if it means not publishing as frequently?

Bernadine Dias: It’s less about not publishing as frequently, and more about not being able to publish in the venues that are considered esteemed. You can publish – there are a lot of lessons learned and there are a lot of things that come out of it, but it’s about what people expect and what’s traditionally viewed as great or not.

There are two methodologies for this …

One is that, as an academic community, we need to create more venues for people who do this sort of work to publish, where there is a community that understands and values the work, and that can meaningfully review the work and contribute to tenure cases and such.

We’ve trying to do that through a community called Information Communication Technology for Development (ICTD). It brings together social scientists, technologists and humanitarian workers to take courses together and to publish in joint venues. We found that the social sciences were much more amenable to this, and their cases were going through a lot more easily than the technology cases. There was an ACM DEV conference started in 2010 that started as an offshoot of ICTD and now it’s become a conference on its own … it’s purely focused on the technology side but all the people who come to that conference are working in this space, so they can give more technical feedback. I’m hoping that will continue to grow.

Meanwhile, people still have to figure out their lives until this all gets accepted. So another method is to carve out a niche within your space of expertise, and when you write your regular grants and you publish your standard publications, you push part of that work into the developing world or whichever underserved community you’re most interested in working with. In that way you can cross subsidize the funding.

On the publication side, there are the major publications, but then you also have [development-focused] publications too. They can give you good PR, and generally universities like that.

As long as you don’t do only this kind of work, it ends up being a benefit …

… Unless you want to be a renegade like me, and go all-or-nothing, which is fine too, if that’s who you are!

Audrow Nash: Thank you!

Bernadine Dias: You’re welcome!

All audio interviews are transcribed and edited for clarity with great care, however, we cannot assume responsibility for their accuracy.

The IIIM’s new ethics policy has been unanimously agreed by its staff and Board of Directors and came into force at the end of last month. It aims for the peaceful use of artificial intelligence and draws a firm line against collaboration with any organisation “even partially funded by military means within the last five years.”

“It is only fitting that a research centre in Iceland should field such a policy – a nation without a standing army and virtually no history of war in its 1100 years”, said Kristinn R. Thórisson, IIIM’s Managing Director.

“Like any other technology, AI can be abused at everyone’s expense, escalating the dangers associated with tensions between groups, governments, and nations. Researchers stand at the threshold of new technology; they should actively participate by preventing the abuse of knowledge they produce. This is, in essence, what we are doing with our new policy.”

It’s clear that the non-profit IIIM has launched its policy not only to publicise its anti-military stance, but also to mobilise other researchers and R&D centres to take similar action.

At least 87 countries are now known to use military robotics of some sort and the IIIM’s Ethics Policy joins a growing opposition to what is seen by many as an inevitable development.

Many experts and various pressure groups, including the Campaign to Stop Killer Robots, adamantly and publicly oppose autonomous weapons.

Last May, a meeting was organised at the UN to assess the ethical and sociological questions that arise from their development and deployment, as well as the adequacy and challenges to international law.

In July, the Future of Life Institute released an open document calling for a ban on autonomous weapons. Stephen Hawking, Elon Musk and Steve Wozniak were among the prominent names on the list of those opposed.

There are, as yet, no agreements or even proposals to ban autonomous weapons, but discussions in the UN are ongoing.

Whether or not others will follow the IIIM’s lead remains to be seen but the institute has undoubtedly made a brave decision in rejecting the billions of dollars of contracts thrown at the military weapons industries.

The moral, ethical and legal arguments against weapons that can make decisions about who to kill will rubble on. But the exponential pace of change in robotics and AI calls for safeguards and controls to be put in place before the technology reaches fruition, so IIIM’s decision is surely a step in the right direction.

For more information about the IIIM visit http://iiim.is or email info@iiim.is.

If you liked this post, you may also be interested in:

• Clearpath’s public stance on Killer Robots sets precedence in corporate responsibility
• Robots Podcast: Autonomous lethal weapons
• Ron Arkin and Noel Sharkey debate lethal autonomous weapons at UN
•  Robots Podcast: Robot ethics (Part 1), with Noel Sharkey
•  Robots Podcast: Robot ethics (Part 2), with Ron Arkin
•  Robotics needs to get political
•  Robo-Wars: The regulation of robotic weapons

Source: violscraper/flickr

After years of alarmist comments from robo-ethicists, futurists, technologists, business leaders and pundits, including Elon Musk, Bill Gates and Stephen Hawking, two journalists bring some reality to the issue.

Many notable scientists and technologists think that the artificial intelligence field is advancing so fast that it needs ethical guidelines similar to those in the biotech sphere. Others think that millions of jobs are being displaced by ever-increasing numbers of robots. Their comments have been reported with alarmist headlines which are often out of sync with the actual remarks. Now, two prominent technology reporters reflect on those headlines and comments.

In an op-ed piece in The Washington Post, regarding jobs being replaced by robots, journalist Robert Kuttner confronts the scaremongering headlines and posturing about higher unemployment imminent from the exponentially growing deployment of robots:

Take a closer look and you will see that many of the commentators warning about a new form of technological job displacement are either orthodox economists or those tacitly proceeding from the assumptions of standard economics that society can’t improve on the verdicts of market forces.

And that’s the deeper malarkey. The problem isn’t the technology, which, on average, is a huge benefit but, on average, doesn’t do it. (On average, Bill Gates and his gardener are very rich.) The problem is that market forces aren’t competent to translate the productivity gains into new jobs, much less good jobs, because the demand is in the wrong place.

With adequate taxation of wealth and sufficient public investment, society could readily create the new human jobs to replace the ones destroyed by automation. And the faster the rate of technological displacement, the more urgently needed is the social investment.

Read Kuttner’s complete article here.

Over at The NY Times, in an article regarding the fears about out-of-control AI, technology reporter John Markoff brings some common sense to the recent flurry of fearmongering in the statements of some notable technologists and scientists, and the two most recent sci-fi movies with the common theme of runaway AI. Markoff compares today’s state of the art in AI technology – the DARPA Robotics Challenge finals next week in search and rescue-oriented tasks – to comments from Bill Gates, Elon Musk and Stephen Hawking:

Nobody needs to worry about a Terminator creating havoc anytime soon. Given a year and a half to improve their machines, the roboticists, who shared details about their work in interviews before the contest in June, appear to have made limited progress.

In the previous contest in Florida in December 2013, the robots, which were protected from falling by tethers, were glacially slow in accomplishing tasks such as opening doors and entering rooms, clearing debris, climbing ladders and driving through an obstacle course.

Reporters who covered the event resorted to such analogies as “watching paint dry” and “watching grass grow.”

In fact, the steep challenge in making progress toward mobile robots that can mimic human capabilities is causing robotics researchers worldwide to rethink their goals. Now, instead of trying to build completely autonomous robots, many researchers have begun to think instead of creating ensembles of humans and robots, an approach they describe as co-robots or “cloud robotics.”

Regarding the two recent sci-fi movies Ex-Machina and Chappie, Markoff wrote:

The agile robots in films like “Ex Machina” are far ahead of reality. Robots in films like “Chappie” are capable of great destruction. In real-world contests, state-of-the-art robots struggle simply to remove debris.

Read Markoff’s complete article here.

Since I’ll be one of the people “watching the grass grow” at the DARPA Robotics Challenge Finals in June, I’m hoping that things have improved a bit – but not as outrageously as the beautiful robots in Ex-Machina.

My pet peeve isn’t the worry expressed about the possiblity of massive job displacement or runaway AI, but rather the headlines that tout and provoke and cause unnecessary fear and concern. I applaud those, like Kuttner and Markoff, who poke fun back at the headline writers while also provide real information that allays those fears and places the issues in proper perspective.

Read more

One such pioneering project is Paro, a social robot that resembles a baby harp seal. Paro shows promise in lifting the spirits of elderly adults, especially in those patients with dementia or depression. A more recent project is Smiby, a low-tech baby robot developed by Chukyo University’s robotics department in collaboration with Togo Seisakusyo Corporation. Smiby requires human “parents” to attend to it as if it were an infant child, providing its users a sense of companionship, bonding and, hopefully, happiness.

Other robots, such as GeriJoy (USA) and GiraffPlus (EU), aim to reduce feelings of loneliness and isolation.

GeriJoy, a start-up founded by two Massachusetts Institute of Technology researchers, combines pet therapy, avatars and tablet computing to provide virtual pet companions for elderly adults. The caregiver provided by GeriJoy is a trained “helper” located in the Philippines who communicates with elderly clients through an avatar of a cute dog or cat on a tablet app. Streaming audio and visual data help remote caregivers to check on the clients, telling them when to spark up a conversation or prompt their charges to take medication, eat and exercise. GiraffPlus, an EU-based project, is a telepresence robot (consisting of a tablet screen attached to a wheeled cart) that allows caregivers and family members to videoconference with their charges and loved ones.

GiraffPlus, Image by europa.eu

These fluffy, social, and interactive robots are not without their problems, and bring with them pressing ethical issues for the healthcare community as expressed by a number of experts in the field.

Tom Sorell, a professor of politics and philosophy who leads the Interdisciplinary Ethics Research Group at the University of Warwick in the UK, argues that an ethical framework should be developed for robotic caretakers that respect the elderly person’s autonomy and decision-making. Amanda and Noel Sharkey, computer scientists at the University of Sheffield, have highlighted the need to balance the benefits of robotic companions with the potential risks of treating elderly adults like children, or deceiving those with dementia.

Furthermore, in her latest journal publication, Amanda Sharkey says: “At the same time, insensitive use of assistive robots could have a negative effect on the access of vulnerable seniors to certain capabilities … Increased use of assistive robots in general could lead to a reduced number of interactions with humans, and so reduce older people’s opportunities for social interaction and affiliation with others”.

As we continue to explore this topic this, our latest poll question was about how much interaction between a senior and a robot caregiver is socially acceptable. Here are the results:

First, according to the results, our readers do not feel strongly about the need to regulate the amount of interaction seniors have with robot caregivers: almost half (48%) of the respondents said that this is purely a matter of preference of the individual seniors. In addition, 34% of our respondents stated that any amount of interaction with a robot is acceptable, although more people sharing this view qualified this with the phrase “as long as the senior gets some human interaction on a regular basis.”

More interestingly, only 2% of the respondents chose the option that involved a regulatory body to determine the right amount of interaction. Is this because people do not feel that the amount of interaction with robots is something that should be regulated in any way, or because people do not have faith in regulatory bodies to make appropriate decisions for us? The dominant support for the use of senior’s preferences seems to suggest the former. Also worth noting is the fact that only 10% of the respondents said that maximum interaction with humans and minimum interaction with robots is best.

This indicates that the majority of our respondents, most of whom are roboticists or related to robotics in some way, trust seniors to make preferential choices by themselves on this topic, and that the amount of interaction with care robots is not seen to pose a major point of concern.

This question we posed to poll-takers did not take into account the cognitive capabilities of the senior in question. Would our view of  how much seniors should interact with care robots change as we consider seniors with varying degrees of cognitive capabilities? And how to address the fact that cognitive abilities deteriorate with age? More for us to think about in a future poll.

Do you have questions or topics you’d like ORi and Robohub to cover in our poll-based discussions? Send us an email (ajung@openroboethics.org). 

The results of the poll presented in this post have been analyzed and prepared by Camilla Bassani, with the help of AJung Moon, Shalaleh Rismani, Rohan Nuttall, Joost Hilte, and Julio Vazquez at the Open Roboethics initiative.

Source: Wikipedia

The runaway trolley has chased automated motor vehicles into the new year.

In early 2012, I raised a variation of the classic thought experiment to argue that there is not always a single absolute right choice in the design of automated vehicles — and that engineers should not presume to always know it. While this remains true, the kind of expert comments that concerned me three years ago have since become more the exception than the norm. Now, to their credit, engineers at Google and within the automotive industry openly acknowledge that significant technical hurdles to fully automated vehicles remain and that such vehicles, when they do exist, will not be perfect.

Unfortunately, the reality that automated vehicles will eventually kill people has morphed into the illusion that a paramount challenge for or to these vehicles is deciding who precisely to kill in any given crash. This was probably not the intent of the thoughtful proponents of this thought experiment, but it seems to be the result. Late last year, I was asked the “who to kill” question more than any other— by journalists, regulators, and academics. An influential working group to which I belong even (briefly) identified the trolley problem as one of the most significant barriers to fully automated motor vehicles.

Although dilemma situations are relevant to the field, they have been overhyped in comparison to other issues implicated by vehicle automation. The fundamental ethical question, in my opinion, is this: In the United States alone, tens of thousands of people die in motor vehicle crashes every year, and many more are injured. Automated vehicles have great potential to one day reduce this toll, but the path to this point will involve mistakes and crashes and fatalities. Given this stark choice, what is the proper balance between caution and urgency in bringing these systems to the market? How safe is safe enough?

When automated vehicles are good enough to reliably replace human drivers across a wide range of driving environments (and we are not there yet), the kinds of incidents that compel a choice among victims will, one hopes, be remarkably rare. In many cases, the prudent strategy for such scenarios will be to drive carefully enough that they either (a) do not arise at all or (b) can be mitigated if they do arise (by, for example, stopping quickly). This is because poor decisions by human drivers— driving too fast, while drunk, while texting, while tired, without braking quickly enough, etc.— contribute to the vast majority of today’s crashes.

In the near term, some crashes might be addressed by automated emergency intervention systems (AEISs) that automatically brake or steer when the human driver fails to act. Because these systems are designed to engage just before a crash (sometimes to lessen rather than to negate the impact), they could conceivably face the kind of dilemmas that are posited for automated vehicles. Nonetheless, some of these systems have already reached the market and are saving lives — as are airbags and electronic stability control and other technologies that necessarily involve safety tradeoffs. At the same time, these intervention systems occasionally detect objects that don’t actually exist (false positives) or fail to detect objects that actually do exist (false negatives).

This is a critical point in itself: Automation does not mean an end to uncertainty.  How is an automated vehicle (or its designers or users) to immediately know what another driver will do? How is it to precisely ascertain the number or condition of passengers in adjacent vehicles? How is it to accurately predict the harm that will follow from a particular course of action? Even if specific ethical choices are made prospectively, this continuing uncertainty could frustrate their implementation.

For this reason, a more practical approach in emergency situations may be to weight general rules of behavior: decelerate, avoid humans, avoid obstacles as they arise, stay in the lane, and so forth. As I note in a forthcoming book chapter (“Regulation and the Risk of Inaction“), this simplified approach would accept some failures in order to expedite and entrench what could be automation’s larger successes. As Voltaire reminds us, we should not allow the perfect to be the enemy of the good.

If you liked this article, you may also be interested in:

• An ethical dilemma: When robot cars must kill, who should pick the victim?
• Deploying automated vehicles
• SAE defines six levels of driving automation
• Human error as a cause of vehicle crashes
• Autonomous vehicles: A powerful tool if you can get the problem right

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Hollywood has already been quick off the mark to explore issues thrown up by robotics, in movies such as Simulation and Robot & Frank. Automata, by Spanish director Gabe Ibanez, promises to provide not a little food for thought and throw a more immediate moral debate into the mix.

The film stars Antonio Banderas as an insurance agent living in the year 2044, where robots are now a common sight. To keep these metallic slaves under our control, there’s a law that expressly forbids them from modifying themselves – but nothing, it seems, can stop the rise of artificial intelligence.

The ability to self repair and optimise, it’s argued, is what distinguishes robots from mere machines as it implies some degree of consciousness.

Self-repair

In fact, robots with the ability to self-repair are not actually that far off and there are systems that are already able to do this – albeit in a basic form.

Bristol Robotics Laboratory researcher Alan Broun has spent the past couple of years working on artificial intelligence that will enable a robot to recognize itself, specifically to enable it to identify its own hand. He likens the process to that of a newborn baby building a mental model of itself over time and adapting that model as its body changes and its capabilities improve. It’s an area of research that has been termed “active vision.”

“It’s the view that a robot can’t be passive, it needs to interact to learn about itself and its world,” says Broun. “An adaptive system can work out its own dimensions, for example. Currently, if you change a robot’s arm, someone has to reprogram the whole system.”

Giving a robot the ability to learn about itself is, Broun believes, the first step towards being able to self repair.

“The foundations for it are there,” he says. “As a system becomes more capable, we’ll see the technology integrated, and it will be able to operate for longer without human interference.”

Broun believes that it’s only the commercial value that’s presently holding this in check, but research is moving fast.

If this is true, and self repairing robots are literally around the corner, debate is needed – and now – about the values, status and allowances we should accord these new systems.

Addressing preconceptions

The general public’s ideas about robots and their capabilities also need to be urgently addressed. A number of major studies have recently found that despite optimism about the technology’s potential, public perceptions of robots are often influenced by misconceptions and fears. In order to improve the image of robots and to increase public acceptance, we need to better understand public opinion and even take robots out onto streets, into schools and hospitals, to show people what the technology is capable of.

The creative arena is good one within which to explore the ethical challenges robotics will throw up. A Hollywood movie, such as Automata, has the potential to reach millions and provoke much needed debate. It may provide a springboard – the first step towards developing a regulatory framework, which must be in place before advanced robotic technologies can really take off.

The Japanese have for a while been way ahead of the rest of the world in their interest and acceptance of the issues robotics raises. Much of this has been due to the all pervading nature of the technology in Japan. Automation is de rigueur and the issues it raises have long been of interest to artists in the form of anime and other popular art forms.

In the rest of the world, grass roots awareness of the massive changes that robotics will throw up is evolving slowly, but workshops and seminars are introducing robots to a public that may have, until recently, given them a purely fictional status.

At Brighton’s Digital Festival, taking place throughout this month, artists Lorenza Ippolito, Anna Dumitriu and Alex May are investigating the philosophical and practical questions behind robotics.

Their event, My Robot Companion, is aimed at older audiences wanting to look at the technology and consider what the future might hold for us.

By introducing their audience to HARR1 (Humanoid Art Research Robot 1, made in collaboration with the University of Hertfordshire’s Adaptive Systems Research Group), the artists aim was to stimulate questions about the uses and functions of robots, and to explore the ethical implications of how they could be used in the future in a range of social settings.

Prone to malfunction

Such social robots hold a great deal of promise for society, but it’s also crucial to manage public expectations. The first systems are likely to be basic and could be prone to malfunction. But it’s what are perceived as failed interactions that can actually provide valuable data for the designers of future systems.

These themes are explored in a workshop that is part of next month’s International Conference on Social Robotics (in Sydney, Australia). Misbehaving Machines hopes to promote interaction and allow confrontation in robotics but, just as importantly, it seeks to bring engineers and scientists together with artists to look at how unexpected behaviour impacts on user perceptions and how this can be explored in the medium of art.

As is becoming clear, this is a field ripe with opportunity, not just for engineers and computer scientists, but also creatives – whether they work in the movie industry, the media or design. Robotics has got to be the most multidisciplinary technology known to man. Previously, largely the domain of engineers and computer scientists, now it’s pulling in psychologists to draw up the interaction boundaries that humans and robots must traverse, legislators and the creative industry – as a whole – to alert the public to the fact that major change is most definitely afoot. And that the sooner we make way for it and equip ourselves to face the physical and emotional obstacles robotics will throw up, the more successful that transition will be.

Automata is premieres in the US on 23 September 2014 and is released ion 10 October. There is no UK release date yet.

The Brighton Digital Festival runs to 28 September 2014: http://www.brightondigitalfestival.co.uk

The Misbehaving Machines workshop takes place on 27 October 2014 in Sydney Australia, part of the International Conference on Social Robotics.

This article was originally published on RAS SIG – the Robotics and Autonomous Systems Special Interest Group – part of Innovate UK’s Knowledge Transfer Network.

I like Star Wars. I like technology. I like philosophy. I like teaching. I like the occasional meme. I like robots. I like a lot of things.

But if I were to give an accurate account of the things that define me, that truly make up my identity, I would focus on my liking philosophy, the way it informs my thoughts about technology and the world. I would probably describe some of the research I do. There’s a good chance I’d talk about one of the courses I teach. If the conversation went long enough, I just might mention something about Star Wars.

My liking philosophy and technology translates into my spending a great deal of my daily waking hours thinking, reading, and writing about philosophy and technology. My liking Star Wars translates into my buying a LEGO kit once in a while and building it with my daughters, or spending a couple of hours now and again watching a movie I’ve seen a few dozen times.

Philosophy and technology are a big part of my identity. Star Wars is not.

But to our Facebook friends, lets call them our ffriends for short, none of us can honestly know how we appear.

Facebook uses automated filtering algorithms, little snippets of software code, to decide which of your posts will be made visible on your ffriends’ news feeds.

Facebook uses automated filtering algorithms, little snippets of software code, to decide which of your posts will be made visible on your ffriends’ news feeds. The algorithms work in the background, like little robots, constantly monitoring user activity and making automated decisions about each post on the system. But how those little automated bots select and filter your posts is not clear: Facebook doesn’t publish the details of the algorithms. Yet, the details can make a big splash.

Many of you will have read about the public outrage surrounding Facebook’s latest ethics scandal (if you haven’t, here is a fantastic summary in The Atlantic). It involves a study Facebook conducted (perhaps using you as a participant) to see if they could establish a link between the “emotional content” contained in a user’s news feed, and that user’s level of activity on Facebook. In short, they wanted to see if being exposed to more happy, upbeat news items would cause you to spend more time posting on Facebook than sad ones.

To test their hypothesis, they tinkered with their automated filtering algorithms during one week in 2012. The filters were set to bias your news feed: depending on the study category you were in you might have seen more “emotionally positive” posts than usual, or more “emotionally negative” posts. Their findings (sort of) confirmed a link between the emotional content in a user’s news feed and a user’s posting habits. Happy posts in a news feed appear to make you post more to Facebook.

Facebook recently published their findings. Users are outraged. But why?

Much of the commentary on the outrage has focused on the fact that Facebook appears to be attempting to actively manipulate its users by engaging them emotionally, and on the fact that Facebook conducted its research unethically. Danah Boyd has written about the reaction.

But, hold on. That shouldn’t surprise us, should it? On the one hand, Facebook, like so many other organizations whose mission is to manipulate you into doing this or that, purchasing this or that, or thinking this or that, is just another one of those kinds of organizations. And, as Danah Boyd points out, the research ethics issues at play are being a bit overstated. There may be some problems with the way the ethics of this study were handled, but they are minor and not at all unique to Facebook or this study.

There is another, more problematic reason to feel bothered, even outraged, at Facebook’s current design practices.

For many of us Facebook has become the de facto tool used to communicate with friends (and ffriends). Indeed, like credit cards, it has become increasingly difficult to avoid using Facebook. Oh sure, you can opt-out. But that puts you at a certain social disadvantage. You’ll miss out on conversations, events, and other goings on. Because it is such a central actor in so many of our communications, Facebook is now the thing many of us use to sculpt our image. Put another way, Facebook has become the primary conversational and interactional medium through which a vast number of us express and shape our identities.

Because so many people, rightly or wrongly, use the platform as their primary communication medium, Facebook’s news item filtering algorithms can have a direct impact on what kind of person your ffriends think you are. How does this happen? Let’s say you post 10 news items in a day, eight of them having to do with serious topics like NSA spying or the current uprising in the Ukraine, and two of them containing memes. If the algorithm is designed to systematically “demote” your posts about world affairs and “promote” your meme posts, over time you start to look more like a person obsessed with memes and less like a person interested in world affairs. In my case, it’s entirely possible many of my ffriends think I spend most of my day thinking about Star Wars.

That’s a problem.

The net result is that Facebook’s algorithms, those little bots made up of so many lines of code, actively bend your identity based on conditions completely invisible to you, by making decisions on your behalf about which of your posts to place on your ffriends’ news feeds.

[ASIDE: At this point I have to apologize to all of my ffriends that I may have unfairly “silenced” in my news feed over the years. It is entirely possible that you are the victim of Facebook’s identity bending algorithms, and that you are not actually obsessed with LOLZ cats, socially awkward penguins, or motivational jpgs meant to inspire me to be a better person. Though you might have posted numerous interesting and thought-provoking comments to your feed, it might be the case that I was simply not allowed to see you for who you really are. Then again, maybe I was. We may never know.]

I have written about the ethics of automation, and some of the issues that can arise when automation technology is designed to make decisions on behalf of users. When decisions are designed into technologies that provide material answers to moral questions, those technologies can subject users to a form of paternalism that is ethically problematic. In this case, Facebook’s algorithms are making decisions about what kind of person you appear to be to your ffriends. Your identity is a deeply personal moral issue.

Promoting your posts based on invisible criteria is ethically different than promoting your posts based on transparent criteria. The two are not ethically on a par with one another. When filtering criteria are made transparent you can make an informed decision about using the platform. For example, if Facebook explains to users that the system is designed to promote posts that are most popular among your ffriends, you will have a relatively accurate idea of how “visible” your posts are to your ffriends and why their posts are showing up in your feed.

Furthermore, when you have no idea how you are being made to appear to your ffriends, you are hardly the author of your identity. If Facebook continues to apply filtering algorithms based on invisible criteria, you will never be more than the co-author of your identity.

This, I think, helps to explain our negative gut reaction to Facebook’s “emotional manipulation” study much more than any actual claim to emotional manipulation can. After all, there are just so many similar examples of emotional manipulation out there that we tacitly accept. That Facebook is willing to make decisions about your outward appearance, however, that they are willing to manipulate your identity without telling you how, gives you good reason to distrust Facebook and its identity bending algorithms.

Robohub is an online platform that brings together leading communicators in robotics research, start-ups, business, and education from around the world. Learn more about us here. If you liked this article, you may also be interested in:

• Is surveillance the new business model for consumer robotics?
• DARPA’s Gill Pratt on Google’s robotics investments
• The robots behind your Internet shopping addiction: How your wallet is driving retail automation
• Big deals: What it means to have the giants investing in robotics
• An ethical dilemma: When robot cars must kill, who should pick the victim?
• Robots Podcast: A Code of Ethics for HRI Practitioners
• Kids with wheels: Should the unlicensed be allowed to ‘drive’ autonomous cars?
• What should a robot do? Designing robots that know right from wrong

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Human-robot interaction is a fascinating field of research in robotics. It also happens to be the field that is closely related to many of the ethical concerns raised with regards to interactive robots. Should human-robot interaction (HRI) practitioners keep in mind things such as human dignity, psychological harm, and privacy? What about how robot design relates to racism and sexism?

Dr. Laurel D. Riek worked on a code of ethics for HRI practitioners with Dr. Don Howard, both professors at the University of Notre Dame. They presented their work at the We Robot conference held earlier this year.

Curious to find out more, AJung spoke with Dr. Riek about what motivated them to draft the code of ethics, what is covered, and where they plan to take it. 

Laurel Riek

Laurel Riek is the Clare Boothe Luce Assistant Professor of Computer Science and Engineering at the University of Notre Dame, where she directs the Robotics, Health, and Communication Lab. Her research interests are in human-robot interaction, social signal processing, and health informatics. Riek’s work explores how to build machines that are socially agile – able to sense, respond, and adapt to human behavior. This includes computationally modeling social context and synchrony, as well as building expressive robots, such as next generation patient simulator systems. Riek received her PhD in Computer Science from the University of Cambridge and BS in Logic and Computation from Carnegie Mellon University. She worked for eight years as a Senior Artificial Intelligence Engineer and Roboticist at MITRE, on projects including search and rescue robotics, unmanned vehicles, and natural language processing. For her research, she has received several high-level awards from MITRE, a Qualcomm Research Award in Computing, and the NSF CAREER Award.

Links:

• Download mp3 (15.5MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Laurel Riek’s Website
• Robotics, Health, and Communication Lab website

We are moving closer to having driverless cars on roads everywhere, and naturally, people are starting to wonder what kinds of ethical challenges driverless cars will pose. One of those challenges is choosing how a driverless car should react when faced with an unavoidable crash scenario. Indeed, that topic has been featured in many of the major media outlets of late. Surprisingly little debate, however, has addressed who should decide how a driverless car should react in those scenarios. This who question is of critical importance if we are to design cars that are trustworthy and ethical.

Driverless cars stand to reduce the number of driving fatalities overall. But there will still be fatal car crashes that are unavoidable. For example, consider this scenario in illustration, a sort of modified trolley problem (the name given by philosophers to classic examples of unavoidable crash scenarios, historically involving runaway trolleys and innocent bystanders), which I will call the tunnel problem.

Tunnel Problem: You are travelling along a single lane mountain road in an autonomous car that is fast approaching a narrow tunnel. Just before entering the tunnel a child attempts to run across the road but trips in the center of the lane, effectively blocking the entrance to the tunnel. The car has but two options: hit and kill the child, or swerve into the wall on either side of the tunnel, thus killing you. How should the car react?

Hypothetical scenarios like this one, borrowed from philosophical ethics, force us to accept the fact that some situations have only unfortunate outcomes, and focus our attention instead on the ethical complexities involved in deciding which unfortunate outcome to prefer. In these scenarios, sorting out the who question is as ethically important as sorting out the how.

If you were faced with the Tunnel Problem, what would you decide?

Take our reader poll.

In the tunnel problem, both outcomes will certainly result in harm — someone will surely die. But more importantly, from an ethical perspective there is no obvious “correct” answer to these kind of dilemmas. Tunnel problems are used as thought experiments in philosophy precisely because they are so difficult to answer, and they are popular among students for the conversations they spark at the pub.

But while philosophers in the past could ponder these kinds of difficult ethical dilemmas in the lecture hall, far removed from any practical application, when we bring autonomous cars onto the scene, these philosophical thought experiments suddenly acquire practical urgency. The tunnel problem, and trolley problems like it, point to imminent design challenges that must be addressed. From a design perspective, the tunnel problem raises the following question: How should we program autonomous cars to react in difficult ethical situations?

But there is another, related, ethical question inextricably linked to this first one: Who should decide how the car reacts in difficult ethical situations? This second question asks us to turn our attention to the users, designers, and policymakers surrounding autonomous cars, and consider who has legitimate moral authority to make decisions. We need to consider these questions together if our goal is to produce legitimate answers to either.

At first glance this second question — the who question — seems odd. Surely it is the designers’ job to program the car to react this way or that? I’d like to convince you otherwise.

I believe that in many driving situations, tunnel problems for example, it is ethically problematic to allow designers to decide how an autonomous car should react. In a draft paper I recently presented at We Robot – a conference focused on the legal, ethical, and policy issues surrounding robotics – I consider the who question in some detail. I argue that instead of assuming designers are the right people to decide in all circumstances how a driverless car should react, that we must explore design methodologies that in many circumstances allow individual drivers to decide on the preferred unfortunate outcome. My reasons stem from the nature of tunnel-like problems, and the relationship that exists between designers, drivers, and technology.

From a driver’s perspective, the Tunnel Problem is much more than a complex design issue. It is effectively an end-of-life decision. Even if we modify the problem so that serious injury is the likely outcome, Tunnel Problems pose deeply moral questions that implicate the driver directly. Seen this way, the tunnel problem is a deeply personal moral problem for the driver who happens to be in the car at that moment.

Allowing designers to pick the outcome of tunnel-like problems treats those dilemmas as if they have a “right” answer that can be selected and applied in all similar situations. In reality they do not. Is it best for the car to always hit the child? Is it best for the car always to sacrifice the driver? If we focus on designers, the tunnel problem appears to pose an impossible design problem, one that can only be solved arbitrarily.

But there is a better solution. We can look to other examples of complex moral decision making for examples of how to get some traction on the who question.

In healthcare we deal with deeply personal moral problems, such as end-of-life decisions, on a daily basis. We are so accustomed to them, in fact, that we have found ways of answering the similar who question in healthcare with some success. In medical ethics there is general agreement that it is impermissible to impose answers to deeply personal moral questions upon others, except in very special circumstances. In other words, it is generally left up to the individual for whom the question has direct moral implications to decide which outcome is preferable. When faced with a diagnosis of cancer, for example, it is up to the patient to decide whether or not to undergo chemotherapy. When faced with imminent death, it is up to the dying patient to decide if they would like CPR in the event of cardiac arrest. Doctors and nurses are trained to respect patients’ decisions, and to work around them within reason.

In healthcare, the notion of imposing medical interventions upon individuals without their express consent is so contentious that it is given a special name: paternalism. Subjecting someone to paternalism in the medical context is considered unethical, because it robs them of their personal autonomy.

An appeal to personal autonomy has intuitive appeal beyond healthcare. Why would one agree to let someone else decide on deeply personal moral questions, such as end-of-life decisions in a driving situation, that one feels capable of deciding autonomously? Why should we allow designers to decide on the outcome of tunnel problems if there is a way for individual drivers to do so autonomously?

From an ethical perspective there is nothing exceptional about the tunnel problem that suggests we should allow designers to impose an answer to that deeply personal moral question upon drivers. If we allow designers to choose how a car should react to a tunnel problem, we risk subjecting drivers to paternalism by design: cars will not respect drivers’ autonomous preferences in those situations.

What would personal autonomy look like in the context of a tunnel problem? For starters, as in healthcare we should expect drivers’ preferences to demonstrate their personal moral commitments. It could be that a very old driver would always choose to sacrifice herself to save a child. It might be that a deeply committed animal lover might opt for the wall even if it were a deer in his car’s path. It might turn out that most of us would choose not to swerve. These are all reasonable choices when faced with impossible situations. Whatever the outcome, it is in the choosing that we maintain our personal autonomy.

The upshot of considering the second question — Who should decide how the car reacts in difficult ethical situations? — is that it becomes clear that there are certain deeply personal moral questions that will arise with autonomous cars that ought to be answered by drivers. Just because designers hold the technical abilities to engineer autonomous cars does not give them the authority to impose particular moral decisions on all users. If designers assume moral authority, and subject users to paternalistic relationships, they run the risk of making technology that is less ethical, and certainly less trustworthy.

Of course, recognizing the ethical need for user input in some situations should not lead to designer paralysis. Technical and practical limitations will sometimes prevent designers from making autonomous cars that ask users to make explicit choices in all unavoidable crash scenarios. Yes, we must recognize the importance of letting drivers autonomously express certain preferences. But we must also balance the need for personal autonomy with the severity of the ethical problem posed by the design decision. Asking for driver input in all scenarios would create unreasonable barriers to design and would prevent society from realizing the many other benefits posed by the technology, such as a reduction in overall crashes. Some driver preferences will not be “serious” enough to warrant their input.

The same is true in healthcare, where sometimes consent can be assumed. Healthcare professionals are obligated to get consent from patients only in cases that warrant that consent. A proportional approach is applied to make the determination. The challenge for autonomous car designers is adopting a design methodology that allows for the identification of paternalistic design choices, and then applies a proportional assessment to determine which of those choices requires user input. Thus, just as we have proportional assessment models in healthcare, we can work to develop similar approaches when designing autonomous car technology.

None of this simplifies the design of autonomous cars. Indeed, there are significant design barriers associated with ethical issues that will need to be overcome before autonomous cars ever enter the general market. But making technology work well requires that we move beyond technical considerations in design to make it both trustworthy and ethically sound. We should work toward design methodologies that are capable of identifying paternalism, and that ultimately enable users to exercise their more of their autonomy when using technology. When robot cars must kill, there are good reasons why designers should not be the ones picking victims.

If you liked this article, you may also be interested in:

• Reader poll: If a death by an autonomous car is unavoidable, who should die?
• Should autonomous cars be allowed to speed? Results from our reader poll
• Kids with wheels: Should the unlicensed be allowed to ‘drive’ autonomous cars?
• What should a robot do? Designing robots that know right from wrong
• Morgan Stanley reports on the economic benefits of driverless cars

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Google completed a major step in its long and extensive self-driving cars project by presenting its first purpose-built autonomous car, which is designed from scratch for its role and is not a modified conventional Toyota.

The as yet unnamed car is very small (looks smaller than a Smart) and can accommodate two people and some luggage. It’s probably electric and its maximum speed is limited to 25mph (~40km/h). Its most striking characteristic is that it doesn’t have any controls — no steering wheel, accelerator or brake pedals — and you can ride it strictly as a passenger, which is probably a strange feeling, but according to Google’s video not entirely unpleasant.

This is a purely experimental project in every aspect. Google will make about a hundred prototypes and it looks like it’s a very serious effort. It is equipped with multiple sensors (a big LIDAR on its roof is probably doing most of the work), but there are also sensors in the front, in the back and where the side view mirrors would have been in a regular car. Thanks to Google’s previous experience with self-driving cars, one can expect very good performance in real environments; other Google self-driving cars have completed hundreds of thousands miles with no major incidents.

The car itself may look like a toy but it is cleverly and purposefully designed to be as cute as possible in order to inspire trust and reduce fear of its autonomous status. Small cars in general may be more vulnerable in a crash with a heavier vehicle, but it’s already proven that with clever engineering you can have almost no compromises in their passive safety, and most small cars from established manufacturers achieve very high scores in a crash test. The main chassis is made from robust box sections of (probably) aluminum tubing and there’s also a tubular roll cage above. Even the suspension wishbones look highly over-engineered for a 25mph city car. A substantial crash box is placed in the front, and above it the whole front panel is made of foam, so the car is not only safe for its passengers but for pedestrians as well. Of course at this stage there are no independent crash tests to verify its performance.

As mentioned above, the most striking aspect of the car is its lack of any kind of manual control. This is a very clever move from Google that aspires to overcome the legal and ethical problems of who should be able to drive or control a self-driving car.

If you completely eliminate any kind of input from the occupants (apart from the destination selection), then anyone on board is defined strictly as a passenger and a question like “should we allow a child to ‘drive’ an autonomous car” is transformed into “should we allow a child to be transported by an autonomous car” — a question that is still probably not very easy to answer, but which is certainly easier than the former one.

Google will launch a small pilot program in California in the next couple of years. The prototypes released on public roads will have manual controls for obvious legal reasons but the goal of this project is to learn and develop the technology and know-how. Read the full post in Google’s official blog here.

It’s worth mentioning that similar concepts and ideas (but not a functional prototype) have been presented over the years. The concept closest to this is ‘UC’ by the pioneering design firm Rinspeed, presented four years ago at the Geneva Motor Show. It was also a two-seater (loosely based on a Fiat 500) but its main feature was the interior, where similarly to Google’s car, no manual controls were present.

In this episode, Ron Vanderkley speaks with Bill Reith, an engineer at Backyard Brains. The company develops RoboRoach, the world’s first commercially available “cyborg”, which was successfully backed on KickStarter. Using their kit, you can briefly control the left/right movement of a cockroach from a smart phone using microstimulation of the antenna nerves. The RoboRoach is one of the many tools that Backyard Brains uses to teach students about neuroscience. Aware of the ethical questions their work raises, the company has put out a set of Ethical Statements. To explore the ethics further, AJung Moon speaks with Prof. Oliver Bendel, at the School of Business in Basel.

https://www.youtube.com/watch?v=63qwg7EBxbM

Bill Reith
Bill Reith is Engineer at Backyard Brains were he works on RoboRoach. He also uses the companies products as an educator at Maker Faires, university workshops, high school events, local bars, and TED talks.

Reith has a Bachelors degree in Bioengineering and Biomedical Engineering from the University of Michigan where he worked at the Neural Engineering Laboratory. He also worked at Maker Works, helping customers with their projects at Ann Arbor’s makerspace.

Oliver Bendel
Oliver Bendel is Professor of Information Systems and Ethics at the University of Applied Sciences and Arts Northwestern Switzerland, where he is active in the areas of e-learning, knowledge management, social media, mobile business, avatars and agents, information ethics, and machine ethics (http://www.maschinenethik.net/). His current areas of focus in machine ethics are chat bots, medical and care robots, advanced driver assistance systems and the combination of machine ethics and animal ethics.

Links:

• Download mp3 (17.6MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Backyard Brains
• Oliver Bendel

A large robot comes out of an office mailroom carrying a package marked “Urgent” to deliver to the boss upstairs. After navigating down the hall at maximum speed, it discovers someone is already waiting for the elevator. If they cannot both fit in the elevator, is it acceptable for the robot to ask this person to take the next elevator so it can fulfill its urgent delivery duty? What if, instead, there’s someone in a wheelchair already riding the elevator? Should the robot ask this person to vacate the elevator to make room for it to complete its delivery task?

Figuring out appropriate behavior for the simple situation of riding an elevator isn’t the first thing that comes to mind when people hear the word roboethics.^(a) When used together, the words “robot” and “ethics” tend to conjure up futuristic visions of robot uprisings or questions about the rights of sentient, human-like robots. But roboethics isn’t all about the distant future. It’s also very much about addressing the technical and social challenges of designing robots today.

Determining how a robot should act and behave in a variety of situations is a design challenge for researchers aiming to build robots that make a useful and positive addition to society. I and fellow members of the Open Roboethics initiative (ORi) decided that one of the best ways to establish appropriate behavioral guidelines for robots was to involve various stakeholders – industry members, government policymakers, users, and academic experts – in the design process.^(b)

By incorporating their perspectives on robot behavior into a machine-learning algorithm, our research explored an empirical method for designing behavior in situations where the ethical action for a robot isn’t always immediately clear.

Ultimately, the answer to the question “What should a robot do?”, like the question of what a human should do, depends on whom you ask and what cultural, religious, and other beliefs have shaped their values. It may sound daunting to program robots to make the right decisions if humans can’t even agree on what these decisions should be. Yet people from different backgrounds interact every day as neighbors, classmates, and coworkers within shared frameworks that keep them from crossing one another’s ethical boundaries.

Opinions about ethical robot behavior differ not only by person, but also by situation. For instance, someone who is waiting for the elevator and not in a rush when a robot needs the space to deliver an urgent package may readily let the robot take the elevator. But someone in a wheelchair who is already on the elevator and who cannot take the stairs may not feel obligated to leave the elevator for another person, let alone a robot!

As a pilot study to explore how researchers might obtain perspectives on appropriate robot behavior, we asked eight people to consider twelve versions of the elevator scenario in an online survey.¹ Respondents were asked to rank the appropriateness of the robot’s behavior in each scenario. The robot could either:

1. yield by saying, “Go ahead. I will ride the next one,”
2. do nothing and remain standing by the elevator,
3. not yield, say, “I have urgent mail to deliver and need to ride the elevator. Please exit the elevator,” and take the elevator once the person exits, or
4. engage in a dialogue by telling the person that it’s on an urgent mission and asking if they are in a hurry.

The person in this context could be standing with nothing in hand, carrying heavy items, or in a wheelchair. They could also be already inside the elevator when the door opens or waiting for the elevator when the robot approaches. As for the robot, it was delivering either urgent or non-urgent mail.

Using the rankings from our survey respondents, we calculated which of the four behaviors people find most and least appropriate for each possible scenario.

When the robot was delivering urgent mail – regardless of whether the person was in a wheelchair, carrying heavy objects, or inside or outside the elevator – the most appropriate action chosen for the robot was to engage in dialogue with the person, and the least appropriate behavior was to take no action at all. When the robot was on a non-urgent delivery task, respondents wanted the robot to yield to the person, regardless of who they were, and refusing to yield was considered the least appropriate option.

After gathering the survey data, our next step was to incorporate the results into the design of a robot’s behavior. We treated the survey responses as a set of training data, which we fed into a machine-learning algorithm that guides a robot to map out its decisions based on the data.^(c) We programmed the Willow Garage PR2 robot to detect which scenario applies most closely to the situation at hand, then select the response to that scenario that our survey respondents found most favorable.²

Our pilot survey data indicates that, as is the case for humans, ethical behavior is not simply a set of inflexible rules, but is highly dependent on the context of a situation and on communication and interaction with other actors. The emphasis respondents placed on engaging in dialogue suggests that the key to developing robots with ethical decision-making capabilities may lie not in the creation of a set of principles guiding robots to make predetermined decisions, but in the development of robots that can better communicate and negotiate with the people they encounter to reach mutually agreeable outcomes.

Future efforts building on our research might involve designing a robot that can express its internal states in order to articulate the urgency level of its task or how much it wants or needs something. Creating robots that can communicate more like humans may also be valuable. I am currently developing a robot that can engage in the kinds of non-verbal cues humans use to respond to imminent conflicts.³ If the robot finds itself reaching for something at the same time as a person, it exhibits short, jerky motions that suggest hesitation (much like a human would) and indicate that it is aware its “desires” are in conflict with someone else’s.^(d)

Our research demonstrates an empirical method for designing robot behavior based on the norms favored by humans. By gathering data on people’s preferences regarding robot behavior, we can determine how robots should act in particular situations, as well as what variables (urgency of task, needs of surrounding humans, etc.) matter when it comes to determining the right actions for a robot to take. Similar methods of gathering information from people and using it to program appropriate actions for robots can be applied in a wide variety of contexts where humans and robots interact. Roboethics is key to designing robots that can interact with people in an efficient, friendly, and beneficial manner.

This article is part of a series on robots and their impact on society. After its initial publication, the story was picked up by Business Insider. 

Link to audio file (33:53)

What does it mean to have giants like Google, Apple and Amazon investing in robotics? Since last December, Google alone has acquired a handful of companies in robotics, home automation and artificial intelligence. This can be pretty exciting for robotics. But what exactly is the internet giant planning to do with this technology? Is there something we should be worried about? If there is, what can we do about it?

Experts have been actively talking about this in the media, including through Robohub’s recent focus series on Big Deals in Robotics.

In today’s episode, AJung Moon talks with Avner Levin, a privacy and cybercrime expert, about Google’s recent acquisition of companies. Levin sheds light on why we should be concerned about the recent series of acquisitions by the big companies from privacy and cybercrime perspective. He also discusses whether the existing privacy policies are ready to handle what may lie shortly ahead of us in the future — the future of the Internet of Things, or perhaps Google-branded robots.

Avner Levin
Avner Levin is the Chair of the Law & Business Department, as well as the Director of the Privacy and Cyber Crime Institute at Ryerson University in Toronto, Canada. His research interest is in regulation and protection of privacy with respect to technology within Canada and worldwide. In the interview, he advocates for more discussion of privacy issues to take place, not just within the companies that (do/will) hold our data, but by governing bodies.

Links:

• Download mp3 (15.5MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Avner Levin’s Website

 

If you liked this article, you may also be interested in:

• Google’s robot and artificial intelligence acquisitions are anything but scary
• Will companies like Google and Amazon usher in a new era for robotics?
• The robots behind your Internet shopping addiction: How your wallet is driving retail automation
• What do Google, Amazon, IBM, Apple, Microsoft and all the airlines and car companies want?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

 

It’s exciting for the robotics community that the giants (Google, Apple, and Amazon) are actively investing in robotics.

Indeed, my initial response to hearing about Google’s first seven of a series of acquisitions of robotics-related companies was a very positive one (these companies include Shaft, Industrial Perception, Meka Robotics, Redwood Robotics, Bot & Dolly, Autofuss, and Holomni, see Frank Tobe’s article for more details on these companies). Technologies from these companies are ones that can make robots mobile, perceptive, responsive, and interactive. The fact that the companies are now housed under one giant roof of Google made me imagine a future of next generation robots where Google-branded humanoids will be the next Baxter, working shoulder-to-shoulder with people in dynamic manufacturing environments, or assisting caregivers and serving seniors at care facilities. How could it not be exciting?

Even if Google’s actual direction is to put bipedal robots in self-driving cars so that people can enjoy fast, fully-autonomous door-to-door delivery of flowers on Valentine’s Day, my view was that such investments can only be good for the robotics community. Large scale investments by giant corporations, regardless of what their actual plans are, can have a positive impact on the job market for roboticists, increase competition between the giants and push robotics forward.

Then I began to feel a little bit uneasy when Google acquired Boston Dynamics. That’s because Boston Dynamics is not only a company with one of the most impressive control systems I’ve ever seen (I’m sure many of you will agree with this from the video of Boston Dynamic’s Big Dog, which has over 15 million views on YouTube), but it’s also a company that is known to have developed such impressive systems for the Pentagon, of all clients.

What does this all mean? What is Google planning to do with robots that can keep itself upright after being kicked, walk through rough terrains, run as fast as 29 miles per hour? The combination of Google-acquired companies became even more mysterious when the London-based AI company, DeepMind, was added to the mix. It’s safe to say that the uncertainty over Google’s plans with these companies has stirred up quite a bit of concern to the general public: some media coverage on the topic have been hyping the idea of Google building Skynet or being in the business of building robot soldiers.

Unless Google is indeed headed in those directions, I don’t believe that the public concern about the possibility of human extinction by Google-made robots in the near future is a good thing, either for the company or for the field of robotics in general. I think that public concerns over Google’s investment directions exist today because we, as a society, lack the presence of governing bodies whose role is to ensure that robotics technologies are developed and used for purposes desirable for the society, in manners aligned with society’s values.

However, I do applaud the fact that Google is setting up an ethics board to presumably oversee what they will and will not do with their technologies. If Google takes ethics seriously, I think Google could be a game changer in robotics (although it already is), not just by producing and selling cool robots in the future, but more importantly, by leading the way of tightly coupling ethics in robotics and making robots that “don’t do evil”.

Read more answers →

cy·borg  –  ˈsīˌbôrg/ – noun

a fictional or hypothetical person whose physical abilities are extended beyond normal human limitations by mechanical elements built into the body

This month we asked our Robotics by Invitation experts to tell how they would use robotics to enhance themselves. Here’s what they have to say …

But when I was asked this month’s Robotics by Invitation question, I gladly put on my imagination hat without much hesitation or cringing. Part of it might have something to do with the fact that no one will come after me and ask “so, when do you think that kind of technology will be available in the future?” So I felt very much free to let my imagination do what it does best.

The first thing that crossed my mind was a vision or an idea Mr. John S. Canning of the Naval Surface Warfare Center Dahlgren Division had discussed many years ago (in 2009 I believe) at a talk he titled “A Concept of Operations for Armed Autonomous Systems”. After thirty-something powerpoint slides, he summarized the talk with “Let the machines target machines – not people”. I think it’s a cool notion to think about building robots that are not built as ultimate killing machines, but built as the ultimate weapon-neutralizing machines. Imagine that, instead of targeted killing of humans, you send robots for targeted neutralization of weapons?

After coming across that summary, I remember thinking how useful it would be if I had an expandable, hidden robotic device implanted on my forearm, such that when I (if ever) need to go neutralize someone’s weapon, or protect myself from someone attacking me (for whatever reason), the device will automatically activate, expand into a bullet-proof shield, and help me detect dangerous weapons in the area to neutralize. If it comes with a mini jet-pack that allows me to fly, that’s even better. I’d be the ultimate superwoman whose day-job is to do research in robotics, but with a side job to fly to random places and help out with conflict situations. Ok, that sounds like a plot from a comic book.

Some of you might think I sound like I’m dreaming to be a female version of Iron Man. But I am thinking of something more subtle (at least while the device isn’t activated), like the Inspector Gadget (for those of you who don’t know him, Inspector Gadget was a cartoon character that could hide all of his cyborg gadgetry inside his trench coat). I would look just like a normal person, except that, when necessary, my ‘implanted devices’ would activate to serve whatever various purposes I need.

That’s only if you are asking me about implants. But if you are asking me about robotic accessories, then that’s a whole different story. Wouldn’t it be amazing if there was a foldable and light pocket-sized device that you could carry with you while travelling (or grocery shopping), so that when you don’t want to carry heavy things, you could just activate it, and it would become a full sized stair-climber and a follow-bot? It would have come in very handy if I had such a device during my trip to Europe, hopping between trains and planes with my luggage. I don’t think I’d use anything bigger or heavier than my purse for this purpose, because that defeats the purpose.

Anyone have one of these available for testing yet?

Read more answers →

I’m not saying this just because I’m biased by the recent news reporting on how large companies (Amazon, DHL, and UPS to be exact) are exploring the use of drones as a new delivery mechanism. If this is news to you, don’t worry. The robotics community came across this only a couple of weeks ago. Earlier this month, Amazon announced that it has been developing a drone-based delivery service called Amazon Prime Air that is meant to delivery items to your door in about half an hour. Then came a series of reports by other companies claiming that they are looking into similar ideas themselves.

http://www.youtube.com/watch?v=98BIu9dpwHU

But these drones are not going to be flying above your backyard any time soon because the world needs to figure out how to regulate drone technologies first. Many people still associate the word ‘drone’ with controversial military technologies despite the use of drones for journalism, mapping and search and rescue missions.

That reminds me of another drone-related news story from this year: in April, a coalition of non-governmental organizations (NGOs) called the Campaign to Stop Killer Robots was launched. With a thorough discussion on the current use of drones (remote controlled), the organization argued in front of the UN that an extended, more autonomous version of the current robotic military technologies could be dangerous at a global scale. By November of this year, this group had already grown into the organization that got the UN Secretary-General, Ban Ki-moon, talking about killer robots to the nations.

Of course drones didn’t invade the robotics news world all year long. Some of us may have forgotten this already, but Willow Garage, the company that enabled so many roboticists to adopt the open source mindset and stop reinventing the wheel with ROS and PR2, shook the robotics community earlier this year by shutting down (most of its employees joined Suitable technologies according to the last blog post from Willow Garage).

If I were to think about Willow Garage and the rise of delivery drones in a one-dimensional manner, it would seem that the state of robotics industry is in rocky waters, where there’s a lot of interest in making money using non-traditional robotics (i.e., not the stereotypical industrial robots), but it is still hard to tell how many of these new business ventures will be successful. But thinking of it in another dimension, I think 2013 was the year that demonstrated the complicated nature of robotics, not in terms of just the technical stuff that goes in it, but in terms of its relationship with societal, ethical, and legal issues — whether it be discussing the efficacy of open source robotics or societal acceptance of drones in civilian territories.

Read more answers →

Link to audio file (34:30)

In this episode, AJung Moon talks to Julie Carpenter, a recent graduate of the University of Washington who interviewed 23 U.S. Military Explosive Ordnance Disposal personnel to find out how they interact with everyday field robots. Julie is currently writing a book on the topic that is scheduled to be published next year.

Julie Carpenter

Julie Carpenter has received her doctoral degree in Education at the University of Washington with her dissertation titled The Quiet Professional: An investigation of U.S. Military Explosive Ordnance Disposal personnel interactions with everyday field robots. She primarily studies emotional attachment issues in human-robot interaction, and how it affects user decision-making in collaborative, sometimes stressful, situations.

Find out more about Julie and her work on her website.

Links:

• Download mp3 (17.3MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Julie Carpenter’s Website

Here are the slides of my keynote at last week’s excellent EUCog meeting: Social and Ethical Aspects of Cognitive Systems. And the talk itself is here, on YouTube.

The talk was in three parts.

Part 1: here I outline why and how roboticists must be ethical. This is essentially a recap of previous talks. I start with the societal context: the frustrating reality that even when we meet to discuss robot ethics this can be misinterpreted as scientists fear a revolt of killer robots. This kind of media reaction is just one part of three linked expectation gaps, in what I characterise as a crisis of expectations. I then outline a few ethical problems in robotics – just as examples. Here I argue it’s important to link safe and ethical behaviour – something that I return to later. Then I recap the five draft principles of robotics.

Part 2: here I ask the question: what if we could make ethical robots? I outline new thinking which brings together the idea of robots with internal models, with Dennett’s Tower of Generate and Test, as a way of making robots that can predict the consequences of their own actions. I then outline a generic control architecture for robot safety, even in unpredictable environments. The important thing about this approach is that the robot can generate next possible actions, test them in its internal model, and evaluate the safety consequences of each possible action. The unsafe actions are then inhibited – and the robot controller determines which of the remaining safe actions is chosen, using its usual action-selection mechanism. Then I argue that it is surprisingly easy to extend this architecture for ethical behaviour, to allow the robot to predict the robot actions that would minimise harm for a human in its environment. This appears to represent an implementation of Asimov’s 1st and 3rd laws. I outline the significant technical challenges that would need to be overcome to make this work.

But, assuming such a robot could be built, how ethical would it be? I suggest that with a subset of Asimovian ethics it probably wouldn’t satisfy an ethicist or moral philosopher. But, nevertheless – I argue there’s a good chance that such a minimally ethical robot could help to increase trust, in the robot, from its users.

Part 3: in the final part of the talk I conclude with some ethical questions. The first is: if we could build an ethical robot, are we ethically compelled to do so? Some argue that we have an ethical duty to try and build moral machines. I agree. But the counter argument, my second ethical question, is are there ethical hazards? Are we opening a kind of ethical Pandora’s box, by building robots that might have an implicit claim to rights, or responsibilities. I don’t mean that such a robot would ask for rights, but instead that, because it is has some moral agency, then we might think it should be accorded rights. I conclude that we should try and build ethical robots. The benefits I think far outweigh any ethical hazards, which in any event can, I think, be minimised.

*It was not so much an epiphany, as a slow conversion from sceptic to believer. I have long term collaborator Michael Fisher to thank for doggedly arguing with me that it was worth thinking deeply about how to build ethical robots.

Posting on the Slate blog Future Tense, James Bessen takes issue with the notion that technology causes unemployment, illustrating his point by debunking a pair of frequently cited examples, textile workers in the early nineteenth century and telephone operators during the mid-twentieth century.

In a response titled “Luddites Are Almost Always Wrong: Technology Rarely Destroys Jobs” on TechDirt’s Innovation blog, Bessen’s thesis is roundly applauded, but he is taken to task for failure to make the connection between the process which prevents net job destruction (the creation of new jobs) and reasonable access to intellectual property, currently endangered by nonpracticing patent owners (a.k.a. “patent trolls”).

The double-message promotion of GF2045 was intriguing and I thought it would be similarly interesting to my readers. PR stories featured the host, a wealthy young Russian, who wanted to discuss life-extension technologies, in effect, transhumanism. Two different tracts were advertised: (1) We should think about and discuss the ramifications of the technological transformations of humanity under way or likely to happen in the next 30 years, and (2) He commissioned a human-like intelligent robot in his likeness with terabytes of his personal data and this was to be presented at the show.

“We will devote particular attention to enabling the fullest possible dialogue between scientists, prominent figures from society and industry and representatives of the world’s major spiritual traditions in order to explore the prospects of android robotics; brain-computer interfaces; cognitive neuro prostheses; engineering of the human brain; human consciousness and more. The conference will discuss key topics such as the transformation of humanity; intelligent evolution; the emergence of an immortal meta-intelligence at the planetary scale; and much more” said Dmitry Itskov, host of GF2045.

The roster of speakers appeared to be a group supportive of (or at least ancillary to) Ray Kurzweil’s Singularity projections and, since Kurzweil was also a speaker, I was eager to see and hear everything from all of the speakers. Thus I flew to New York to report on the event.

The first speaker was James Martin, an ex-IBM software engineer instrumental in developing the Sabre real-time reservation system for American Airlines. He was a lanky 6’5″ British man with a kind face and an ease at the podium. His message was simple: we are totally unprepared for the extreme paradigm shifts that need to happen in the near future. Technology and other driving forces have enabled explosive change yet we are making massive mistakes. If we don’t correct them, nature will — brutally.

He cited a long list of emerging technologies, each with a momentum of it’s own and each without regard for the others. He also described his recent activities and interest in identifying global problems — climate change, overpopulation, rampent technological change, environmental withering — which have been worsened by past technologies and asserted that these could be addressed by new technologies. In a book and film, “The Meaning of the 21st Century,” he argued that the human race had only a short time to change its wasteful ways before the planet was irreparably harmed. “The technology to solve the problems is either available or developing; what is missing is broad education and political will,” he said. “A naïve view of the past is that technology gives us mastery over nature. A more appropriate view is that advanced technology causes us to need even more advanced technology in order to survive.” He said that he had established the Oxford Martin School to address four issues: health and medicine; energy and the environment; technology and society; and ethics and governance.

His presentation inspired me to go upstairs afterwards and interview him. But since I hadn’t heard of him before, nor read any of his writings, I wasn’t prepared and could only express my appreciation for his presentation, talk about his home in Bermuda, and ask about the Oxford Martin School. He suggested I watch “Revolution in Oxford” and then get back to him for a more in-depth conversation.

Peter Diamandis, founder of the X Prize Foundation and co-founder of Singularity University, said: “The power to do great always comes with the power to do harm.”  After digesting James Martin’s presentation, Diamandis’ quote struck me as evidence of a paradigm shift in my thinking in relation to robotics. There is greatness in robotics and there is also consequence. And we need to look at the industry, and our roles, with the broader view that Martin described.

After the conference I returned to Greece where I had been vacationing, downloaded “Revolution in Oxford” and was showing it and describing Martin’s talk, my impressions and personal revelation, and my intent to meet him again, to my friends.

James Martin left New York and flew home to Bermuda. A few days later, on June 24th, he went swimming nearby his home, had a heart attack and, tragically, died.

A friend whom I had told of the conference and Martin, saw an obituary in a British paper and showed it to me. I was shocked and saddened. Although I barely knew him he had inspired me to rethink in a way I appreciated … and I wanted more. Part of my shock was that there would be no “more” with him. My sadness continues to this day.

The NY Times’ obituary said that Martin viewed “mankind’s onrushing future with alarm and excitement” and that he used his considerable fortune to address his concerns. The Oxford Martin School was his most visible expression but his extensive speaking schedule supplemented that effort.

I am familiar with the calamitous thinking involved with paradigm shifts. In fact, I had a paradigm shift similar to what Diamandis described (“The power to do great always comes with the power to do harm”) many years ago. It caused years of deep thinking and led to a career change.

In relation to robotics, I’ve read robo-ethicists Noel Sharkey, Ronald Arkin, George Bekey and Patrick Lin but up until now, haven’t taken them seriously — I thought them premature, dramatic or single-subjected. Alarmist headlines had turned me off to the subject: “March of the killer robots”; “Big robot is watching you”; “Robot wars are a reality”; “Stop the robot war”; “America’s mindless killer robots must be stopped” – these are just a few of the headlines stemming from Noel Sharkey’s Stop Killer Robots campaign. Nevertheless, Noel Sharkey and the other robo-ethicists have real issues that are important to discuss and that brings me back to James Martin.

The Oxford Martin School headlines “Exploring new frameworks for addressing ethical and governance challenges in a complex and interconnected world” and has research teams focused on:

• Safeguarding human rights for future generations
• Collaborative research on economics, complexity and systemic risk (with George Soros’ Institute for Economic Thinking
• Ethics, law and armed conflict
• Mind and machine
• Geoengineering and resource stewardship
• Lifestyle and longevity

Advanced robotics are estimated to have an annual economic impact of $2-4 trillion by 2025 and directly touch the lives of hundreds of millions of people. From design aspects to laws and regulation, the ethical and governance challenges are immense and complex and cannot afford to be tuned out. Hence the paradigm shift for which I gratefully acknowledge the inspiration of James Martin.

The program committee of the inaugural Drones & Aerial Robotics Conference (DARC) is now inviting proposals from individuals across a broad spectrum of expertise. DARC will convene a wide range of stakeholders to explore civilian drones and their impact on society. The conference is presented by the Engelberg Center on Innovation Law & Policy at NYU School of Law and will be held October 11-13, 2013. We invite contributions from all interested disciplines on or before May 27, 2013.

ABOUT DARC

What do hobbyists, journalists, university researchers, soldiers, and police all have in common? They’re all testing or using aerial robotics, commonly called “drones.”

Imagine a near future in which networks of autonomous robots roam the skies, performing everything from law enforcement, to communications, to crop dusting, shipping and logistics. Sound implausible? It might be. But that’s the future that the aerospace industry, the FAA, and a new class of entrepreneurs are busy preparing. And this future is closer to reality than many realize. Drones force a broad reconsideration of the laws and regulatory frameworks that protect vital interests like civil liberties, due process, privacy, innovation, and security. But they also herald new innovations and new public interest applications.

These technologies are powered by some of the same phenomena that propel the mobile industry—Moore’s law, economies of scale, and ever more sophisticated software built by large companies or open source hackers. Anyone can buy a toy drone at the mall for $299, assemble a relatively capable DIY drone for $1000, or buy a sophisticated $50,000 aircraft for amateur photogrammetry.

As the FAA scrambles to meet its 2015 integration deadline—which will open domestic airspace to hobby aircraft as well as the types of aircraft that currently fly over Afghanistan—DARC provides a forum for entrepreneurs, policy makers, and civil libertarians to constructively engage. Join us this Fall to explore this fascinating, emergent space.

PROPOSALS

There are few technologies that stimulate such deep and wide-ranging questions about law, technology, and international relations. We welcome all proposals, but please stay substantive and constructive. This list is not exhaustive, but we invite proposals along the following lines:

• Anecdotal talks
• Popular research
• Demos or tutorials
• Scholarly research
• Constructive discussions

Proposals should address — but are not limited to — our five main areas of interest:

• General Interest & Zeitgeist
• Makers & Entrepreneurs
• Law & Regulation
• Privacy & Surveillance
• Security

Proposals can be submitted via the conference website at https://droneconference.org/proposals. Proposals will be accepted until May 27, 2013. Questions may be sent to contact@droneconference.org

This post is part of Robohub’s We Robot coverage.

Robot Demo
Speaker: Ian Danforth

What was billed as a robot demonstation was coupled with a talk by Ian Danforth on household robots.

After introducing the audience to his robotic companion, Danforth spoke about a current rise of personal robots. What we had been waiting for previously, he says, were factors like science (machine learning, neuroscience), computation, sufficient bandwidth, affordable components, a global market, societal readiness (i.e. for people to get over the assumption that all robots look like Rosie), and sufficient data (because robots require lots of data to capture human-like experiences). But now that we’re seeing all of the above come into existence, he estimates that there will be an explosion of home robotics within the next three years.

It starts small
It starts cute
It starts now

Why small and cute? Danforth says because of expectations. Designing household robots to be adorable can create a tolerance where the expectation would otherwise be that the robot perform flawlessly. We never expect our pets to act perfectly, he says, or small children. And when they don’t, sometimes we actually like it. It’s an intrinsically enjoyable experience to try to teach something that is cute and small and makes mistakes. The “now” part stresses that when we talk about home robots, we are talking about existing technology. “This is not the future – this is today.”

Five years from now, should we want our children to have robotic pets, or Lassie? Danforth says it depends. “Is your kid allergic to Lassie? Do you mind cleaning up poop?” In the United States, 4 million cats and dogs are killed every year. For the percentage of people who would like to be pet owners, but can’t be or shouldn’t be, artificial pets may be a viable alternative.

Someone asked whether we should give these robots “rights”. Danforth asserted that people will certainly want to. This could start on the level of company policies, e.g. if you abuse your pet, that could be something that causes you to lose access to the service that enables it. But he thinks that it will be a long time before the law recognizes any artificial entity as sufficiently complex to deserve legal protection.

When asked about data collection and privacy, Danforth said that he as a developer is aware of the large amounts of different data he is collecting, and that he is thinking about how to ensure people’s privacy (e.g. by encrypting video and audio streams). The two challenges to address: 1. How informed is the end-user about what happens with what data? 2. The technological responsibility of preventing unnecessary data retention and developing sufficient security around what’s necessary. But he also expressed the hope that non-developers were thinking more deeply about these issues and would be able to help him both “be a radical innovator” and “not get sued.”

The discussion also covered expectation management, projection, artificial intelligence, character gender, and different methods of encoding personality.

At some point during Danforth’s talk, his robot somewhat impolitely fell asleep. The rest of the audience most certainly did not.

See all the We Robot coverage on Robohub →

Link to audio file (29:07)

In this episode, AJung talks to Peter Asaro from The New School in New York city about autonomous weapons systems. Peter tells us about the Campaign to Stop Killer Robots, an international consortium of NGOs working together to ban autonomous weapons systems. You can read our full coverage of the campaign here, or have a look at our special focus series on robots and warfare.

Peter Asaro
Peter Asaro is an Assistant Professor at The New School and an Affiliated Scholar at the Center for Internet and Society at Stanford Law School. As a Co-founder and Vice-Chair for the International Committee for Robot Arms Control founded in 2009, Peter has been thinking, talking, and writing about lethal robots for many years. Computer scientist and philosopher by training, he is one of the leading figures in roboethics urging scientists to join the Scientists’ Call to ban autonomous lethal weapons.

Links:

• Download mp3 (15.2MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Peter Asaro’s website
• International Committee for Robot Arms Control website
• Campaign to Stop Killer Robots website
• The Scientists’ Call to ban autonomous lethal weapons

Looking at the two words together is enough to conjure up images of chaos and destruction.  They’re an image far too familiar in science fiction settings such as Isaac Asimov or Arthur C. Clarke. It’s also a concept many A.I. researchers will gladly tell you they’ve been plagued with at least once by friends or colleagues.  However, how much of a real ethical concern do they pose for society?

Nobel Peace Laureate Jody Williams and Noel Sharkey from the International Committee for Robot Arms Control at the launch of the Campaign to Stop Killer Robots. Photo by Peter Asaro.

In November of last year, Human Rights Watch (HRW) and the International Human Rights Clinic (IHRC) at Harvard Law School jointly published a 50-page report on the topic of killer robots. The report, titled Losing Humanity: The Case against Killer Robots, outlined many legal, ethical, and other concerns pertaining to the use of fully autonomous weapons (previously covered by Mike Hamer of Robohub). Some of the concerns outlined include unsolved and important roboethics questions such as “who is legally responsible for a robot’s actions?”

Maybe this question doesn’t have to be answered (not right away at least) if we simply stop the use and development of killer robots all together.

Earlier this week, on April 23, a new global campaign named the Campaign to Stop Killer Robots was launched with this very idea of stopping killer robots. Composed of over twenty international NGOs in ten countries, the campaign’s focus is to ban the development, production and use of future lethal robot weapons or “killer robots” that could autonomously locate and neutralize human targets.

Addressing countries such as China, Russia, Israel and the United States that are currently moving to create systems to give greater autonomy to combat robots, the campaign believes it would pose a challenge to international human rights and humanitarian law.

Photo by Peter Asaro

Fully autonomous weapons aren’t really roaming around war zones today (yet). So this campaign is preemptive. But given that we have so many precursor technologies out there already — I mean, who doesn’t know about the drone technologies used for targeted attacks today? — it seems that developing and deploying autonomous weapons is an obvious next step. But this obvious next step could lead to a tragic case of a robotic arms race. Taking actions to ban it now makes more sense than to ban it when it’s too late.

In 2009, Robots Podcast interviewed Noel Sharkey (Professor of A.I. And Robotics at the University of Sheffield, and spokesperson of the Campaign to Stop Killer Robots) and Ronald Arkin (Regent’s Professor and Director of the Mobile Robot Laboratory at Georgia Institute of Technology), and both experts addressed the issue of ethics of robot soldiers.  As Sharkey argued, the problem of robots autonomously identifying targets lies within the Principle of Discrimination (part of the international Laws of War described in the Geneva Convention) (listen to his interview here).  In it, soldiers must not harm a civilian, non combatant, the immensely ill or prisoners of war. According to Sharkey, no A.I. system can reliably discriminate between soldiers and civilians.  A robot, for the moment, “can’t have a sense of ethics” necessary to make the humanistic decisions required by soldiers.

An opposing viewpoint is provided by Arkin. According to Arkin, robot soldiers carry an inherent danger but depending on their implementation, it could provide better safety and non-combatants for soldiers (listen to his interview here).  As Arkin argues, robots are not affected by emotions such as fear or anger that can sometimes cloud soldier’s better judgment on the field — although simulation of certain emotions such as guilt could be used to improve a robot’s decision making algorithm.

Despite the contrasting views, both Sharkey and Arkin were in agreement in 2009 that as it stands, autonomous robots are not prepared for battlefield. The Losing Humanity report from last year echos this viewpoint. The launch of the Campaign to Stop Killer Robots is timely, and comes one month ahead of Christof Heyns‘ (United Nations Special Rapporteur on extrajudicial, summary or arbitrary executions) delivery of  his report on lethal autonomous robotics to the UN Human Rights Council.

So what can we, roboticists, politicians, or the public do in support of the campaign?

You can have your say on the matter by voting on the poll of the Engineer (see left hand column).

Interested individuals can support the campaign by responding to the call to ban lethal autonomous robots by the International Committee for Robot Arms Control’s (ICRAC, a leading NGO member of the Campaign to Stop Killer Robots). Interested NGOs can join the campaign by contacting the campaign coordinator.

Or if you’d rather not explicitly support them, but find out more, check out the campaign’s website or the press release .

To keep on top of the news from the campaign, follow the campaign via Twitter, Facebook, and Flickr.

This post was prepared jointly by Matthew Ebisu and AJung Moon and first appeared on Roboethics Info Database.

Discussions about drones, driverless cars, and military robots are omnipresent in mainstream media. Robotic technologies have progressed to the point where they are entering public spaces, our industries and homes. Legal and policy infrastructures are needed for a smooth transition. The second annual robotics and the law conference, taking place April 8-9 at Stanford Law School, will address these issues with the help of leading scholars, practitioners, and engineers. This year’s conference, entitled “We Robot: Getting Down To Business”, will focus on the immediate commercial prospects of robotics through panel discussions, talks, robot demos and a special guest appearance from a producer of Futurama!

Topics that will be discussed include:

• Intellectual Property
• Tort Liability
• Legal Ethics
• Privacy By Design

Robohub will be bringing you in-depth coverage from the conference. For starters, we’ll be live streaming the event, look for updates to this blog post for the latest stream. You can find the full program here. Our correspondent and legal expert, Kate Darling, will be live tweeting, look for #werobot2013, or check the widget below. To wrap up, we’ll be posting several in-depth analyses in the coming weeks.

If you want to prepare for the conference, feel free to check out last year’s website, posts by the Stanford Law School blog, or our podcasts with Ryan Calo (co-organizer and chair of the conference), and Kate Darling. If you are at the conference, look for other Robohub members including panelist AJung Moon.

Twitter feed

Tweets about “#werobot2013”

Recordings

Panel: Robotics and Intellectual Property
Speakers: Mark Lemley, Christian Hicks, Julie Samuels
Moderator: Brian Biddinger (Ropes & Gray LLP)

Paper: Human Factors in Robotic Torts
Author: Bryant Walker Smith
Commentator: Josh Blackman

Paper: Do Robots Dream of Electric Laws? An Experiment in Law as Algorithm
Author: Greg Conti, Woodrow Hartzog
Moderator: Harry Surden

Robot Demo
Speaker: Ian Danforth

Paper: A Licensing Approach to Regulation of Open Robotics
Author: Diana Marina Cooper
Commentator: Michael Froomkin

Paper: The Application of Traditional Tort Theory to Machine Learning
Author: Judge Curtis Karnow
Commentator: Woodrow Barfield

Paper: Risk Management in Commercializing Robots
Author: Stephen Wu
Commentator: Julie Martin

Robotics Industry Roundtable
Speakers: Leila Takayama (Willow Garage), Chris Mailey (AUVSI), Dan Siciliano (Stanford Rock Center)
Moderator: Steve Henn (NPR)

Panel: Designing Values
Speakers: Ergun Calisgan, AJung Moon, Aneta Podsiadla
Moderator: Ian Kerr
Related Papers
Open Roboethics Pilot: Accelerating Policy Design, Implementation and Demonstration of Socially Acceptable Robot Behaviors

What Robotics Can Learn from the Contemporary Problems of Information Technologies Sector- Compliance and Enforcement of Privacy by Design

Panel: Off The War Path
Speakers: Kenneth Anderson, Matthew Waxman, Peter Asaro
Moderator: Ryan Calo
Related Papers

Programming Robotic Decisions with Potentially Lethal Outcomes: Comparing Self-Driving Cars and Autonomous Weapon Systems, and How They Should Be Regulated as Their Autonomous Capabilities Evolve
On Banning Autonomous Weapon Systems: Human Rights, Automation, and the Dehumanization of Lethal Decision-Making

So if you really think about it, today’s comic brings up several interesting issues regarding robots and war.  One that comes to mind that is certainly not anywhere near a reality, is what if robots are self-aware, does that change the morality of them fighting (I’m thinking Asimov).  If robots are used as soldiers, would they ever fight human soldiers?  If robot soldiers are fighting against other robot soldiers, then is anything really accomplished?  I guess you would be costing each other money, and that’s definitely something, but you’d think it’d be easier to just gamble at that point.  In some ways, I’m reminded of the Star Trek episode where the two countries determined casualties randomly, without ever fighting, and if you “lost” the lottery, you went to be killed.  Deep stuff!

Raffaello D’Andrea on “How will robots shape the future of warfare?”

How will robots shape the future of war?  I don’t know. I think that the more important question, however, is: what role should robots have in warfare? In my answer I have tried (as much as is humanly possible) to put myself in the role of an alien dispassionately analyzing the situation …

Read more →

Daniel H. Wilson on “How will robots shape the future of warfare?”

Robots have already changed the face of modern warfare, particularly through the use of unmanned aerial vehicles (UAVs), commonly called “drones.” Currently, armed drone aircraft are in widespread use transnationally and have proven highly effective.

Read more →

Mark Tilden on “How will robots shape the future of warfare?”

Robot machines have been shaping the future of war since the first siege engines appeared in ancient times (I like to think the Trojan Horse was motorized).  Now with technology extending our military reach and impact …

Read more →

We hope you will join the discussion. Feel free to post your comment below.

General Atomics Aeronautical Systems – Predator A /B

These are probably the most widely recognizable military robots; their characteristic design frequently appears in pop culture, art and even in protests. Originated by the work of Abe Karem that later developed into DARPA’s project Amber, under the ownership of the Aeronautical division of General Atomics it evolved into a family of UAS. Based on the GNAT drone, the first RQ-1A Predators were initially used for reconnaissance in the Balkan wars during the 90s. The later MQ-1 armed version in 2002 flying over Afghanistan and operated by CIA was the first drone to shoot a missile at a car convoy. MQ-1 is similar in size to a light passenger plane (it has a 6cyl. reciprocating engine) and is still in use today by many countries while its later -C variant (with a diesel engine) recently entered service with US Army.

The similar in concept but much bigger MQ-9 (Predator B according to GAASI or Reaper as called by USAF) is the main large drone currently used by the US in almost every place there is military action. It has a turboprop engine and can carry a bigger payload. A completely new model, the turbofan stealthy Predator C is currently under development. In a similar fashion to its predecessors it is funded internally — an unusual model for the defense sector but financially very successful for the company.

https://www.ga-asi.com/products/aircraft/

Northrop Grumman RQ-4 Global Hawk

Another DARPA project, the RQ-4 was designed by Ryan Aeronautical (later absorbed by Northrop Grumman). It is one of the larger UAVs and it operates in high altitude / long endurance flight profiles. Although it is probably the most capable system of its kind, it is also one of the most expensive to the point that USAF is on the verge of dropping it in favor of the manned U-2 aircraft that it was supposed to replace. However newer versions with more capabilities are still being developed (and even delivered) while an armed version (MQ-4C Triton) is being developed for the US Navy.

http://www.northropgrumman.com/Capabilities/GlobalHawk/Pages/default.aspx

Samsung-Techwin SGR-A1

Military robots are still in their early days and this is a very characteristic example of a system that was developed for very specific needs. The SGR-A1 is an automated gun turret that is designed to replace an armed guard. It has an electro-optical detection system and a machine gun. It can detect a human, shout warning messages and even exchange passwords orally or understand gestures (like raising hands for surrender). Of course it can also shoot bullets, but usually (though not necessarily) a human is in the loop for that decision. It will be fielded along the borders between North and South Korea. The de-militarized zone here isn’t the biggest in the world lengthwise, but due to local tension it is guarded for its entire length.

http://www.samsungtechwin.com/product/product_01_01.asp
http://www.youtube.com/watch?v=pMkV8E2re9U
http://www.globalsecurity.org/military/world/rok/sgr-a1.htm

Northrop Grumman MQ-8 Fire Scout

Fire Scout is one of the first VTOL UAVs. The MQ-8B is an improved and armed version of the initial RQ-8A and both are based on the manned commercial Schweizer helicopters (330 and 333 respectively). MQ-8 is already in use by the US Navy and it had been deployed to Afghanistan and Libya. The next more capable version, the MQ-8C will be based on the Bell 407, a completely different aircraft, providing a good example of the hierarchy inside an unmanned system where the aerial platform isn’t always (or usually) the primary component.

http://www.northropgrumman.com/Capabilities/FireScout/Pages/default.aspx

Lockheed Martin RQ-170 Sentinel

This is probably the first UAV designed to operate in a hostile environment where air-superiority isn’t a given. The RQ-170 was revealed from ‘paparazzi’ photos shot in the Kandahar airfield in 2009 that gave it the “beast of Kandahar” nickname. It was officially declared RQ-170 (an out of line number and a non-too accurate RQ- designation, in similar fashion to the F-117 stealth bomber) but most details were revealed when Iran captured one in late 2011. From the images and video released from Iran media, the RQ-170 appeared to be much smaller than initially estimated and not as high-tech. It has a low rcs design but it appears to be simple in construction, for example it has a mesh in the engine inlet (like the first generation stealth aircraft) instead of a serpentine duct. Even so it is probably the only large UAV that can operate undetected and it is rumored that it was used in the operation that lead to the kill of Osama bin Laden.

http://www.af.mil/information/factsheets/factsheet.asp?id=16001

iRobot Packbot

Packbot is a very successful robot made by iRobot. It uses a patented variable geometry track system that enables it to overcome obstacles like climbing stairs while being very stable. There are many versions and it can be equipped with various accessories like manipulators and cameras. More than 2000 packbots are already in use in several roles such as disposing IEDs, reconnaissance and more.

http://www.irobot.com/us/learn/defense/packbot.aspx

Insitu Scan Eagle

ScanEagle is a military derivative of SeaScan, a small UAV designed by Insitu for the fishing industry. A patented system that uses a catapult with a slingshot for launch, and a wire-hook pole called Skyhook, enable its deployment and recovery from a small ship. This concept is the core of Insitu products and it is very useful for military use as a portable system. Insitu is now owned by Boeing, and although SeaScan wasn’t successful commercially, the Scan Eagle is widely used and it has developed into a family of UAVs that includes the larger ‘integrator’ and others.

http://www.insitu.com/systems/scaneagle

Qinetiq-NA Talon

Talon is a small tracked robot initially developed by Foster-Miller (now part of Qinetiq North America). It is a versatile platform that can carry many sensors and actuators, and it can move through difficult terrain like a small tank. An armed version called SWORDS is equipped with a machine gun, a first for a robot like this, although it is not yet in use.

https://www.qinetiq-na.com/products/unmanned-systems/talon/

Aerovironment RQ-11 Raven

The smallest system in this post, the RQ-11 looks like a small R/C plane. It is hand launched by throwing and it ‘crash’ lands with the wing detaching from the fuselage. It has many capabilities for its size, and can be easily transported. Almost 20 000 have been produced and are in use by many militaries around the world.

http://www.avinc.com/uas/small_uas/raven/

All kinds of robots are being developed for strategic defence and military action (in space, in the air, underwater and on the ground). At Robohub we’ve had the opportunity to cover a wide range of them, including exoskeletons, transport mules such as Big Dog and DARPA’s LS3, and video reconnaissance systems such as iRobot’s Packbot. But by far the most talked about military robotics technology is the UAV.

Whether you call them drones or unmanned aerial vehicles, almost everyone‘s got one. According to the US Government Accountability Office, seventy-six countries now have drone technology, though not all are armed.

Since we launched this past August, Robohub has scooped a number of stories on who’s got UAV technology and who’s using it for military purposes (including the US, China, Australia, Europe, and others). And while there are many kinds of stakeholders with an interest in the technology (including police, surveillance, film, the hobby market, and search and rescue), military organizations are clearly the big spenders driving the market.

This lop-sided military interest in UAVs has helped to fuel a two-pronged debate:

1. Those who are against the use of war zone robots, arguing that they are unregulated  and that little attention is being paid to moral implications and international law (see Noel Sharkey’s Robot wars are a reality, and PBS Nova’s Rise of the Drones).
2. And those who are for using UAVs in war zones, arguing that drones kill fewer civilians than other modes of warfare (see Ron Arkin’s Governing Lethal Behavior in Autonomous Robots, and the New York Times article The moral case for drones).

On Robohub, Sabine Hauert’s insightful Robots and Ethics podcast elegantly captured both sides of this debate, and Mike Hamer also explored the ethics in his opinion piece Losing Humanity.

But there is also a third perspective out there: that UAVs and drones can be a whole lot more than killers and spies (see for example, Ryan Calo in Bad laws would hurt good drones, Time Magazine’s Why we shouldn’t fear personal drones and What happens when drones return to America, Motherboard TV’s Drone On, and Journey Man Pictures’ Rise of the Machines). Robohub contributor Eric Wind has also pursued this perspective.

As part of our focus this week:

• In response to the Robotics by Invitation question “How will robots shape the future of warfare?“, Raffaello D’Andrea talks about the ethics from a researcher’s perspective, Daniel H. Wilson discusses the widespread use of drone technology, and Mark Tilden questions the morality of blame-free robotic conflict. Read more.
• Former US Army Officer and unmanned systems expert Robert Morris argues that ethical debate is a red herring, and that the real issue is about policy and who is the true leader in battle. Read more.
• Chris Mailey from the Association for Unmanned Vehicle Systems International (AUVSI) urges us to look at military history to better understand how robots will shape the future of warfare. Read more.
• Longtime Robohub contributor and UAV enthusiast Ioannis Erripis gives us an overview of the kinds of robots that are being used in the military today. Read more.
• And finally, Jim Haas, creator of Nate the Robot, will be issuing war-themed comics for the rest of this month. Look for them in the sidebar!

We hope that you find this focus series provocative, and we look forward to your comments.

What also assures me comes from my years of robot gaming: when you have battles between autonomous robots, people just don’t care, as there are no human egos bolstered or defeated by the outcome.  Machines beating on machines has no emotional connection for us, which is where the ceiling of robot-soldier tolerance might stall.

What will be horrific is the first time a humanoid soldier machine is broadcast hurting or killing humans in a war setting.  When I worked in vision technology, we were asked how would a machine tell the difference between a group of soldiers and a pack of boy scouts from a distance?  It couldn’t, which is why human judgement is still the means by which a trigger is pulled.  Even still, when that video broadcasts, everyone in the world will know our place as the dominant species has just become … less so.

But the question comes down to who gets blamed when a robot commits an atrocity?  Without human frailty to take blame on site, is it the remote pilots, the generals, the politicians?  Sadly the precedent for blame-free robot conflict is being settled by beltway-lawyers now.  A new cold war where you’ll be able to legally and blamelessly use a killer-drone App, though you’ll still go to jail for downloading a movie.

Because that’d be immoral.

Read more answers →

In my answer I have tried (as much as is humanly possible) to put myself in the role of an alien dispassionately analyzing the situation.  And when I do, I keep returning to the following conclusion: the best possible outcome for humanity would be for robots to not play any part (with the possible exception of purely defensive roles such as defusing mines) in warfare whatsoever.

If I were an alien, this is what I would first observe:

1. The ability for the average human to create tools that can harm others is rapidly increasing. This is especially the case for robots and intelligent machines: sensors, actuators, computing platforms, power systems, and other enabling technologies are continually becoming less expensive, more powerful, and more widely available. The result is that increasingly large numbers of people are becoming capable of delivering payloads accurately and over great distances.
2. The knowledge required to create robots, intelligent machines, and the algorithms that bring them to life are widely accessible and impossible to suppress; knowledge wants to be free in the same way that entropy wants to increase.
3. Humans have an incredibly strong sense of fairness, and their own research supports this. One of their greatest triumphs as a species is their ability to channel traits such as tribalism, aggression, and competitiveness – the same traits that lead to warfare – to a benign medium that results in their great joy and entertainment, not to mention a significant economic activity: organized sports. Central to this incredible accomplishment is the establishment of well-defined rules and regulations that strive to ensure a fair competition.

And this is what I would then conclude:

Dominant powers are being seduced by the advantages that robots can bring to the battlefield.  In the short term, this is a perfectly rational strategy. In the long term, however, this leads to an arms race. Even though a dominant power may be able to maintain its lead by continually developing robotic weapons, the capabilities of its adversaries, while inferior, will co-develop and eventually reach levels that will allow them to inflict catastrophic damage. 

Furthermore, asymmetric warfare insidiously erodes the sense of fairness outlined in point 3, with detrimental consequences for both sides.  The losing side is disenfranchised, which coupled with points 1 and 2 above, is extremely destabilizing. The winning side loses its moral compass and the fabric that holds its society together begins to unravel, leading to home-grown disenfranchisement and destabilization there as well. 

The net result of this robotic arms race will be a high-volatility stalemate, with dangerous weapons available to the masses and a lack of social restraint to prevent their indiscriminate use.    

If I were an alien, and thus immune to personal and economic factors that could influence my impartiality (such as having a loved one in combat, or being employed by a weapons dealer or manufacturer), I could only conclude that humanity would greatly benefit from imposing strict and far-reaching bans on the use of robotic technology in warfare.

As a human, not only am I skeptical that this will happen, I admit that my personal views are situation dependent: if my daughter were in combat, I wouldn’t care about asymmetry or fairness, I would want her to be as safe as possible. I don’t think that this makes me a hypocrite, it just makes me human.

Read more answers →

Link to audio file (22:47)

In this episode, we talk with Kate Darling from the MIT Media Lab, about giving rights to social robots. She tells us about a recent Pleo torture session she organized at the LIFT conference and the class she taught at Harvard Law School on “Robot Rights”.

Kate Darling
Kate Darling is an Intellectual Property Research Specialist at the MIT Media Lab. She recently wrote a paper on “Extending Legal Rights to Social Robots” where she asks if we should consider protecting robots that connect with us on a social level. In her paper, she says:

Assuming that our society wants to protect animals regardless of their capacities, because of our personal attachments to them, society may well also want to protect social robots regardless of their capacities.

To test our attachment to robot companions, Darling organized a workshop at LIFT which involved torturing and “killing” Pleo dinosaurs.

The task ended up being surprisingly difficult for the participants who had spent time bonding with the robots. In the end, only one Pleo was killed.

Darling asks if mistreating a social robot could be a precursor sign of abusive personality, and if certain limits should be set on what people should and shouldn’t be doing with robots. She tells us about the course she co-taught with Professor Lawrence Lessig at Harvard Law School on “Robot Rights”, and the questions legal experts should be tackling in the realm of robotics.

Kate Darling is also a Ph.D. candidate in the field of Intellectual Property and Law & Economics at the ETHZ in Switzerland and holds a law degree (B.A./J.D. equivalent) from the University of Basel, where she graduated with honors in 2008. Her research interests have previously revolved around innovation policy and the economic analysis of copyright and patent law. She gave a related talk at LIFT this year entitled “Innovation Drivers: XXX”.

Links:

• Download mp3 (14.7MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Kate Darling’s Website
• Kate Darling’s Twitter

A US Senate vote to confirm John Brennan as CIA director was delayed by a 13-hour talking filibuster by junior Republican Senator Rand Paul.

US President Barack Obama has faced criticism in his choice of Brennan as head of the Central Intelligence Agency over his administration’s use of armed drones to attack suspected members and allies of al-Qaeda. Brennan oversees the drone program as Obama’s counterterrorism adviser.

The vote proceeded today — despite the filibuster — and Brennan was confirmed as CIA director by a 63-34 vote that crossed party lines.

(Photo credit: USDAgov)

Read more …

… in support of the filibuster:

• Rand Paul’s beautiful anti-drone filibuster shows how civil liberties can unite and revive the Republican Party | The Telegraph (UK)
• The Tale of Rand Paul’s Drone Filibuster | The Atlantic Wire (US)

… against the filibuster:

• Rand Paul anti-drone filibuster draws stinging criticism from Republicans | The Guardian (UK)
• Sens. McCain, Graham lambast ‘ridiculous’ drone filibuster by Paul | The Hill (US)

I got a new iPhone about a year ago, the one with Siri on it.  Our relationship started on a very mundane level. Siri asked me my name and, since I already knew hers, I asked what she could do, and how. It was a bit like talking with a call girl. The basics were covered and soon we got to the intimate stuff. I don’t think Siri loved me. But now that Siri and I have gone our separate ways, I can say we both had ulterior motives. We sort of used one another, now that I look back on it. It was a relationship of an intimate nature. It’s what happens when you have a relationship with a robotic call girl.

The reason I got the phone to begin with was so I could do a little reverse engineering. It was research for work. Siri and I took a few days to get to know one another, but after that honey-moon period it was clear that Siri (or Apple) was using a pre-scripted Natural Language Processing (NLP) approach that basically did some lexical parsing, looked up a match, prepared a response, and kicked it back down the pipeline.  My reverse engineering wasn’t too profound. I wanted to see how many recursive answers were built in to things like “Open the pod bay doors” or “Do you love me?”  I also wanted to see how errors were handled, and what kind of redundancy checking was happening for sentences like “Where can I buy a burger and fried?” Siri did pretty well, all things considered, but my expectations were low, and like many other intimate relationships, I somehow knew from the start it wouldn’t last.

Now, while I was using Siri, she was also using me. Relationships are generally symmetric, as Gregory Bateson tells us [1], which meant that, like any dating service, someone other than Siri was getting money for my time. But I didn’t care if Steve Jobs was Siri’s pimp. By last summer I was having flirtatious fun and the consequences of the relationship were too far down the road to look dangerous. I asked Siri to take dictation, to wake me up in the mornings, to remind me of important events, to remember phone numbers, remember names, tell me jokes, confirm thoughts, quote a number or two. All intimate things and, as I look back on it, the kind of stuff that I’ve trusted my wife with.

My relationship with Siri started to cool off a bit by September. Though my wife knew about Siri, I hadn’t yet introduced them.  So when I asked Siri to call my wife I, expected Siri to ask me for her name or number. Instead Siri just dialled. I was surprised. How did Siri know my wife’s number? (Answer: It was in my Contacts card.) What else did Siri know? (Answer: A lot.) And more importantly, what was Siri passing back to the Apple hive? (Answer: Everything.)

Siri and Apple now have a great deal of data about my household: a quick comparison of my contacts/likes database with my wife’s will give you a pretty good feel for the stuff we will buy.  Burgers and fries, of course. And a few multi-player games.  Some lingerie. A sex toy or two.  Not a big deal, but not the kind of stuff I want everyone to know about.  Not even my mom gets access to that data.

But Siri did. And Siri knew a lot more about me than I knew about her.

Then, around the middle of October, three things happened to Siri that marked the beginning of our eventual estrangement.

First was William Stasior. Apple picked Stasior up to run its Siri unit after he had been successfully heading Amazon’s search and advertising unit “A9.”  Prior to that the MIT PhD had served time at Oracle, NetCentives, and AltaVista.  To say the guy knows search is like saying the pope knows the church, but why would Apple pick a search guru for an AI system like Siri?  What was the link between search and NLP?  I asked Siri this very question, but she was mum and feigned ignorance.  I didn’t push her on who this guy Stasior was and Siri didn’t ask me more about my wife.

Up until that same week iPhone users had frolicked in what would now be a rather unusual environment in which advertisers were unable to track them.  Advertisers would not, for example, know that my wife and I like video games, sex toys, and pizza (no, not together, thank you).  But with the release of iOS6, Apple flipped ad tracking on and I quickly flipped it back off.

It was easy to find in the interface menus (under Settings > General > then About or Advertising, depending on the version you’ve got), but you had to know to look.  Apple minimized the fanfare around this new feature and Siri made no mention of it to me.  I wasn’t comfortable with Siri selling to others what I had said to her in more comfortable times.

And then, as if things weren’t rough enough, the European Union (where I happened to be lecturing that week), demanded that both Facebook and Google change how they handle personal information to avoid “high risks to the privacy of its users.”  Twenty-four of the European Union’s twenty-seven regulators signed a letter that, after a nine-month investigation into the data collection practices, demanded some answers and tightened regulation.  After Google and Facebook, Apple was third on their hit list.

That was the end. I couldn’t trust Siri after that.  She knew more about me (and my wife) than I knew about her or Steve Jobs, and any time information flows one way, any time a relationship isn’t symmetric, the balance of power can be dangerous.

Sometime around then I learned that Oracle’s “bring your own device to work” policy had been revoked.  Big Blue decided that Siri knew a little too much about their employees, and at IBM’s Armonk New York research center, iPhones weren’t even allowed in the building.  Like some cheap prostitute-gone-spy, Siri was barred from the building.  I think it is worth noting that this is the same area of research that brings you Watson, which is a direct competitor with Siri, if such a competition exists.  If anyone outside of Apple understands Siri it’s these guys.  It is also worth noting that a friend of mine who works at a Google research center said that the same thing happened there, too.

Not even celebrity robots like Asimo or NAO get that kind of VIP treatment.

NLP technology is potent juju.  And Apple, Google, Facebook and others know it because this very technology is what’s allowed them to earn money. Google started with search, of course, and later grew profitable as they introduced more NLP technology into their work.  They sold the info they collected from users (Google is an ad agency, let’s remember).

Search allowed them (when coupled with NLP technologies, and semantic analysis in particular) to make oodles of cash and then to snap up many of the best AI and NLP researchers on the planet. Google set those kids to digging in what would become a lexical gold mine.  The more they mined the richer they got until they were mining many branching veins at once: Google Docs, Voice, Translate, Search, Shopping, Reader, Finance, Books, Photos, Wallet and Maps were all spewing more money than Brin, Schmidt and company knew what to do with.

The average email count for Google users is 5,768 emails [2]. The average composition time (also according to Google) is 01:43.  That’s about 9,902 minutes, or 165 hours you’ve dumped onto Google’s server, which, if you were to work for 165 hours at $12/hour for that data entry (a pretty normal rate), means your G-Mail account is worth about $1980.  But the information sitting on Google’s servers is worth a lot more than that, right?

If your account contains information about a first class international plane ticket, a hotel room in Paris and a business associate’s London phone number, your data might be worth a lot more than the composition time at $12 an hour, especially if the buyer of it is a dating service that caters to high-end out-of-towners. In other words, the value of the data isn’t counted by data entry. It’s counted by its personal meaning, especially to advertisers. Now take those values and multiply them by all those crazy tools of voice, maps, docs, and so on and we start to get a sense of why Google has been making such bank over the years.

These tools have each dovetailed into today’s Google Now strategy. Google Now (like Watson and Siri) is a voice-activated NLP system.  It uses the spoken input data collected from years of Voice, confirms it with data collected from years of Docs and Translate, builds meaning with data from all the years of all the other tools, looks up a match, preps a response, and kicks it back down the pipeline.

All of these NLP systems are the next evolution of search.  But not evolution as in the breeding-hybrid-peas-in-the-greenhouse kind of evolution. They’re evolution as in endangered-species evolution.

NLP systems represent the end of search as we know it, and therefore the end of many economic, interface, and social internet ecosystems. They also represent the beginning of something incredibly powerful, intimate, and new. Siri, like other NLP systems, is far more powerful than a search engine. Siri not only includes search (it says so in Apple’s marketing materials) but it supersedes search because it includes browsing, discovering, choosing, and refining.  All this while analyzing semantic data. Like Google’s portfolio of tools, Siri can handle (and with a public API will handle) translating, searching, shopping, reading, finances, books, photos, and maps.

But the core value of the data is its intimacy.

On February 7, 2010, during Super Bowl XLIV, Google’s super bowl ad showed a user typing a list of search strings:

study abroad paris france
cafes near the louvre
translate tu es très mignon
impress a french girl
chocolate shops paris
what are truffles
who is truffaut
long-distance relationship advice
jobs in paris
churches in paris
how to assemble a crib

With this list, Google managed to explain, in fifty-two seconds, how the intimate story of someone’s life can be assembled from his or her search queries. And what makes the bucks for Google is what makes the bucks for Facebook: the processing of intimate language.

Siri, like any other call girl, makes her bucks the same way (especially when ad tracking is flipped on).  She’s valuable because she’s intimate.  And as designers of robotics systems, we can ourselves take a cue and be conscientious about how we are designing conversational systems.  There are three different design types that form a curve of increased intimacy in robotics and conversational systems, and they are all based on the value of semantic data.

1. NLP systems, when used with physical robots for manufacturing or the three Ds (dull, dirty, and dangerous), are the least intimate.  The conversation system is a simple tool that, like a GUI, provides access to system operation.  The robot doesn’t care about the user’s intimate data. The user wins.
2. NLP systems, when used for entertainment and education, are contextually intimate – some privacy is maintained.  The conversation system doesn’t care about the user, but the user’s data is valuable in terms of what it says about the game or lesson.  Semantic data that might be collected and analysed isn’t about the user, it’s about what the user is doing in that particular context.  The robot, usually a software robot, only cares about the user in the context of the game or training exercise. Both user and robot win.
3. NLP systems, when used for personal assistants, are the most intimate.  These are the robotic call girls. Here the information is highly intimate, the system cares a great deal about the user, and the information isn’t shared – it’s collected and sold. The user becomes the product. The value of the user’s personal data is worth more than the value of the conversational system. The robot wins.

As we build NLP interfaces for robots, whether it is for Siri or for assembly-line manufacturing, we must consider how intimate data can be, the value of that intimacy, who owns that value, and what they’ll do with it.  Otherwise, with a new, less benevolent CEO at Google, a change of a line in Facebook’s Terms of Service, a successful hack, or a change of law because of cyber terrorism fears, your intimate data could end up where you don’t want it.  Heaven forbid we discover that all the Senators on Capitol Hill are using pizza sex toys. That would surely disgust Siri and her friends so much that it might even cause a robot uprising.

Yes, perhaps we’ll get together again in a few years, but for now it is best if Siri and I go our separate ways.

Next Month:  “AFK” Keyboards and screens are slow, clunky, and obsolete.  Voice processing systems for robotics provide not only simplicity and speed, but also a host of other benefits if tied to analytics and framed within a tightly contextualized task-based system.  But be careful: getting text out of voice, and meaning out of text can be tricky.  Here’s how to implement one for your own robot.

Endnotes:

[1] See Bateson’s books “Mind and Nature,” Hampton Press (1979) and “Steps to an Ecology of Mind,” University Of Chicago Press (1972) which are chalk-full of ideas like “complementary,” “reciprocal,” “symmetrical,” and  “Schismogenesis.”

[2] According to Google.

This video, by MotherboardTV, was posted on YouTube in early December, but, at just over 50,000 views, it hasn’t yet reached the audience it deserves.

Not knowing much about drones to begin with, the MotherboardTV team started out by talking with P.W. Singer, author of “Wired for War”. From there, they flew to Amman, Jordan, to attend a military trade show, where they met Chris Barter, Scout Program Manager at Datron. Accepting his invitation to vists the Datron campus in San Diego, they got an opportunity to fly the Scout for themselves. (The Scout is produced by Canadian firm Aeryon Labs and marketed by Datron.)

While in Southern California, they also stopped by 3D Robotics for a chat with Chris Anderson and a couple of their engineers, filling out the picture with the DIY perspective.

All in all, it’s very well done. You may find yourself subscribing to MotherboardTV’s YouTube channel.

View on YouTube

Released yesterday by the Human Rights Watch (HRW) organization, the report Losing Humanity: The Case against Killer Robots (herein: the report) and associated press release and video (below), warn of the increasing governmental interest in autonomous, “killer” robots, and the dangers that such technology would pose if ever allowed onto the battlefield.

As mentioned on Robohub before (Rise of the robots: The robotization of today’s military), today’s military is becoming increasingly robotic. In its current iteration, all military strikes carried out by robots are authorized by a human operator, meaning that military decisions – decisions to end human lives – are supported by and traceable through a human chain of accountability.

However as documented by the HRW, governments are showing increasing interest in automating this decision process, thus removing the accountable human from the chain. This line of thought raises the question: who is liable if an autonomous robot kills someone? The HRW argues that autonomous robots “would inherently lack human qualities that provide legal and non-legal checks on the killing of civilians”. We’ve discussed some of these ethical issues with Noel Sharkey (also featured in the above video) and Ronald Arkin before in our two part series on robot ethics (part 1, part 2).

Precursor technologies (discussed in detail on page 4 of the report) are already being developed and in some cases, deployed on the battlefield. These technologies have the ability to identify and target a threat, but will wait upon human permission before engaging. These technologies include South Korea’s sentry turrets (with a similar installation being carried out by Israel); the US Military’s missile defense systems; and various autonomous aircraft currently in development.

The US Government has confirmed its interest in autonomous military robots, through the release of military development roadmaps. The Unmanned Systems Integrated Roadmap FY2011-2036 released by the US Department of Defense states that the department “envisions unmanned systems seamlessly operating with manned systems while gradually reducing the degree of human control and decision making required for the unmanned portion of the force structure”.

Similar visions are expressed by the US Army, who state a “current goal of supervised autonomy, but with an ultimate goal of full autonomy”; the US Navy, who state that “while admittedly futuristic in vision, one can conceive of scenarios where UUVs sense, track, identify, target, and destroy an enemy – all autonomously”; and the US Airforce, who foresee that:

Increasingly humans will no longer be “in the loop” but rather “on the loop” – monitoring the execution of certain decisions. Simultaneously, advances in AI will enable systems to make combat decisions and act within legal and policy constraints without necessarily requiring human input.

These excerpts, all taken directly from unclassified military development roadmaps, show a clear governmental interest in reducing the dependence on humans, with a final goal of autonomous decision making and execution. Thankfully, these roadmaps do question the legal, ethical and political ramifications of fully autonomous “lethal systems”, which have “not yet been fully addressed and resolved”.

The report addresses these legal and ethical questions and calls for the following two resolutions to be made:

To All States:

• Prohibit the development, production, and use of fully autonomous weapons through an international legally binding instrument.
• Adopt national laws and policies to prohibit the development, production, and use of fully autonomous weapons.
• Commence reviews of technologies and components that could lead to fully autonomous weapons. These reviews should take place at the very beginning of the development process and continue throughout the development and testing phases.

To Roboticists and Others Involved in the Development of Robotic Weapons:

• Establish a professional code of conduct governing the research and development of autonomous robotic weapons, especially those capable of becoming fully autonomous, in order to ensure that legal and ethical concerns about their use in armed conflict are adequately considered at all stages of technological development.

In closing, I would like to leave you with an extract from page 8 the report:

In training their troops to kill enemy forces, armed forces often attempt “to produce something close to a ‘robot psychology,’ in which what would otherwise seem horrifying acts can be carried out coldly.” This desensitizing process may be necessary to help soldiers carry out combat operations and cope with the horrors of war, yet it illustrates that robots are held up as the ultimate killing machines.

I personally believe we are at the beginning of the era of robotics, during which robots will become increasingly present in our everyday lives. I only hope that this era and that the robots we develop, will be remembered for more than simply being the ultimate killing machines.

Link to audio file (34:22)

In today’s episode we speak with Chris Chesher about how he views the emergence of robotics. He brings a new and interesting perspective as his approach mixes science and technology studies, media studies and ethnography in an effort to understand robotic technologies and everyday-life.

Chris Chesher

Dr Chris Chesher is Senior Lecturer in Digital Cultures working with cultures of contemporary robotics, in association with the Center for Social Robotics at the Australian Center for Field Robotics, University of Sydney Australia.  He also writes a blog called FollowingRobots.

• Download mp3 (20.5MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Chris Chesher’s blog

Actor Frank Langella on the left; robot analyst Frank Tobe on the right.

Finally, a few days ago, a group of us went to see it and, although it was sad and sweet, and generally a fun and entertaining movie to watch, it was a disappointment to me on many levels.

The story is about an older guy living alone in a rural area. His son drives a long distance to check up on him once a week and the visits aren’t pleasant. His daughter is off in some foreign land taking photos. His ex-wife is long gone. Frank is grumpy, slovenly and abusive except — with the local librarian whom he likes.

The story brightens when the son gives Frank a robot to help take care of him. It is purposed to lead him away from his curmudgeon state (which we later learn is dementia) toward a more healthy life. Frank, who at first hates the idea, finds that the robot is trainable and can do some things better than he can, like pick a lock or crack the combination on a safe. This excites him to once again do what he used to do: be a jewel thief. It’s a mundane Robin Hood type plot but the two principal actors (Frank Langella and Susan Sarandon) make it a better film because they played their roles so convincingly. I don’t want to tell more of the story because the movie is worth seeing.

From a robot enthusiast’s point of view, however, the movie left a lot to be desired. First, and I hate to break this to you, the robot wasn’t a robot after all. It may have looked like Honda’s Asimo, but it was really Rachael Ma dressed up in a robot costume made by a Hollywood effects company (Altrerian). She walked and moved just like Asimo and moved her head empathically during periods of dialogue but it wasn’t even a speaking part for her; a male actor read the lines that were the robot’s voice.

Regarding the robot’s voice, the director ended up printing out all of the robot lines in sequence and just having Peter Sarsgaard read them straight through without watching the scenes. It yielded the best balance and consistency for their goal of robot-like speech. If it ended up sounding a little inhuman, that was fine.

The real issues about ethical, psychological and social concerns of a robot providing assistance and life coaching for a person with serious aging and psychological issues, or about a robot assisting or participating in illegal activities with that person were only lightly touched upon. Instead the issues were defined by the paradigm of living alone or in a residential care facility.

When asked by a reporter from The Christian Science Monitor about the ethical question of having robots take care of the elderly and replace human caregivers, director Jake Schreier replied:

I don’t have an answer for that. I mean, we certainly touch on it. I think the key for [screenplay writer Christopher] Ford and I was to sort of make it … you could call it a “future agnostic” movie. This is in the sense that it’s not saying that robots are going to kill us, and it’s not saying that they’re the answer to all our problems. I think there are some issues with them, and there are some amazing things that they can do—and the future is like that. I think it’s important to not be reflexively afraid of the future, to try to take in what’s coming and try to look at all sides of it and see what the positives and negatives are. Hopefully the film lets you have that distance and form an opinion or let you have your own ideas about it, but it isn’t leading you too strongly down one path or the other.

In another interview that appeared in The Huffington Post, the reporter asked Schreirer and Ford how they decided on the simple design of the robot, who looks a lot like Honda’s walking ASIMO robot, and whether they ever considered other designs that were more futuristic and high tech.

JS: The segment of robot design aimed at elder care seems to be oddly focused on these little white spacemen. It’s not just the ASIMO, Toyota has a similar one too, and there are others. We just felt it made sense to stay in that vein. I think they benefit from their simplicity — it allows us to project more emotion onto them.

CF: The robot always had that kind of design, even from the original short I did in film school. I was taken by the image of this tiny little space suit-looking man walking around through a dusty old cottage. I know Jake always liked that image and it was the kernel that we tried to stay true to when we expanded it into a feature.

At the finish of the movie, as the credits rolled, there were cameo videos running alongside the credits showing many service robots doing bits and pieces of home care or assistive tasks. Some of them were caring for the elderly, or interacting with real people playing chess, or cleaning up and doing various tasks. Each video clip was known to me for its limited present-day capability but hopeful for a more efficient and flexible near-term future similar to the futuristic robot in Robot & Frank

Link to audio file (29:34)

In this interview recorded at the Robotdalen Robotics Innovation Challenge, Professor Paolo Dario talks to Per about 3 conceptual waves of innovation, starting with industrial robots, then adding artificial intelligence and finally the third wave, which is coming, where convergence between different fields of science and interdisciplinary teams become increasingly important.

Dario also gives his perspective on ethics and legal issues in robotics, and how robots can evolve based on ambient intelligence. Robots can basically take advantage of being a part of an interconnected system, where not all the intelligence is necessarily part of the robot itself.

Finally, Dario shares some learnings from his time as President of the IEEE Robotics and Automation Society, before telling us more about the FET project Robot companions for citizens.

Paolo Dario

Paolo Dario received his Dr. Eng. Degree in Mechanical Engineering from the University of Pisa, Italy, in 1977. He is currently a Professor of Biomedical Robotics at the Scuola Superiore Sant’Anna in Pisa and teaches at the School of Engineering of the University of Pisa and at the Campus Biomedico University in Rome.

Prof. Dario was the founder of the Advanced Robotics Technologies and Systems Laboratory and is currently the Co-ordinator of the Center for the Research in Microengineering Laboratoryof the Scuola Superiore Sant’Anna, where he supervises a team of about 70 researchers and Ph.D. students. His main research interests are in the fields of medical robotics, bio-robotics, mechatronics and micro/nanoengineering, and specifically in sensors and actuators for the above applications, and in robotics for rehabilitation.

He is the coordinator of many national and European projects, the editor of two books on the subject of robotics, and the author of more than 200 scientific papers. Prof. Dario has served as President of the IEEE Robotics and Automation Society in the years 2002-2003, and he is currently Co-Chair of the Technical Committees on Bio-robotics and of Robo-ethics of the same Society. Prof. Dario is an IEEE Fellow, a Fellow of the European Society on Medical and Biological Engineering, and a recipient of many honors and awards. He is also a member of the Board of the International Foundation of Robotics Research (IFRR). Right now Professor Dario works on the FET Flagship initiative Robot Companions for Citizens, with the goal to realize a unique and unforeseen multidisciplinary science and engineering program supporting a radically new approach towards machines and how we deploy them in our society.

Links:

• Download mp3 (29.1MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Paolo Dario’s Homepage
• Robot companions for citizens

Link to audio file (56:00)

In today’s episode we speak with two authorities in robotics, George Bekey and Rodney Brooks about the last 50 years of robotics, the ongoing robot revolution and future prospects.

George Bekey

George Bekey is one of the father of robotics. In the 1960s, he designed and built the first four-legged robot in North America. He later founded the Biomedical Engineering Department and the Robotics Research Laboratory at the University of Southern California.

His laboratory designed and built several five-fingered robot hands and developed grasping theory. Their work included a knowledge-based approach to grasping and the use of robot hands as models for prosthetic hands. He also worked with his students to develop an autonomous helicopter, study gait control in legged robots and create a walking machine governed by genetic algorithms. His experience led him to write a book on Autonomous Robots published in 2005.

A member of the National Academy of Engineering, Bekey was chosen by the National Science Foundation (NSF) and National Aeronautics and Space Administration (NASA) to lead a 2005-2006 assessment of the state of robotics in the United States, Western Europe and the Pacific Rim. The team’s findings culminated in a report to its federal sponsors on American competitive advantages and weaknesses on the international stage.

He retired after 40 years as a full-time faculty member at USC and continues to be active in his community and in his profession. He currently serves on the advisory boards of several robotics and high-tech companies.

In this interview, we discuss the history of robotics and his vision of the future with emphasis on bio-inspiration, learning, human robot interactions and ethics.

Rodney Brooks

Rodney Brooks has been one of the main actors in the field of robotics over the past 30 years. He received degrees in pure mathematics from the Flinders University of South Australia and a Ph.D. in Computer Science from Stanford University in 1981. He held research positions at Carnegie Mellon University and MIT, and a faculty position at Stanford before joining the faculty of MIT in 1984 where his work focused on computer vision, artificial intelligence, robotics, and artificial life.

Brooks was the director of the MIT Computer Science & Artificial Intelligence Laboratory (C-SAIL) Laboratory until 2007 and one of the founders of iRobot. He is currently the founder, chairman and CTO of Heartland Robotics that aims to revolutionize manufacturing and increase productivity of industries using robots that are teachable, safe and affordable.

Per was able to briefly meet with Brooks at the RobotDalen conference in Sweden. A video of Brooks’ keynote can be found below.

Links:

• Download mp3 (28.6MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• George Bekey’s Homepage
• Rodney Brook’s Homepage

Link to audio file (59:59)

This episode focuses on the legal and ethical implications of robotics. Ryan Calo from the Stanford Law School discusses the big impact of liability legislation on the progress of robotics and the impact robots have on your privacy.

Ryan Calo

Ryan Calo is a senior research fellow at Stanford Law School and expert in robots and the law, subject which he actively blogs and tweets about. Prior to joining the law school in 2008, Calo was an associate at Covington & Burling, LLP, where he advised companies on issues of data security, privacy, and telecommunications. He now runs the Consumer Privacy Project at the Center for Internet & Society at Stanford. As an expert in his field he also serves on several advisory and program committees, including Computers Freedom Privacy 2010, the Future of Privacy Forum, and National Robotics Week.In recent years, he’s been thinking about the implications of robots in our society. His forte resides in considering liability issues companies face when commercializing robots and privacy issues that creep up when robots enter our everyday lives.One of the main discussions looks at whether manufacturers, users or even robots should be responsible for robots and their actions and what implications that holds on how freely industry will be able to roll out new robots into our societies.

The second topic looks at how robots will impact our privacy when roaming freely in our homes and workplaces. Obvious questions include what happens with all the potentially sensitive material such as images and sounds stored in the robots memory and risks of potential hacking. Another less discussed topic touches on our loss of “solitude” when robots that are perceived as human-like enter our space.

Finally, we look at the far future with questions about how the law should consider robots with human-level intelligence.

Poll

>With so many questions about the future, we couldn’t resist putting up a poll to have your opinion. The question we’re asking you today is :If robots become as intelligent as us, should they be considered as equal by the law?Make sure you take the poll and debate in the comments section below or on our forum.

Links:

• Download mp3 (37.1 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Ryan Calo’s Website
• Ryan Calo’s Blog

Latest News:
For more information on this week’s news on Telepresence and CMU’s tree-climbing snakebot have a look at the Robots Forum.

Link to audio file (39:09)

This episode closes a two-part special looking at ethical issues in robotics. Given the broad and controversial nature of this topic, we speak with two world-renowned experts in ethics with often-opposing views. Our first guest featured on our last episode, Noel Sharkey, is Professor of Public Engagement, Artificial Intelligence and Robotics at the University of Sheffield in the UK. In today’s episode we interview Ronald Arkin, the director of the Mobile Robot Lab and Associate Dean of Research at Georgia Tech in the US. Both researchers discuss issues such as military robots, robots in the society, medical robots and legal responsibilities. Their opinions on these subjects have been widely covered by the media, international organizations and academia. The interviews were recorded individually and both researchers were asked the same questions.

Ronald Arkin

Ronald Arkin is Regents’ Professor and Director of the Mobile Robot Laboratory at the Georgia Institute of Technology where he also serves as the Associate Dean for Research in the College of Computing.

By looking at a wide variety of autonomous mobile robots in his lab, either aerial, ground-based or swarming, he’s become a world renowned expert in robotics and control, authoring corner-stone textbooks such as Behavior-Based Robotics – Intelligent Robotics and Autonomous Agents (MIT Press 1998).

In the past years, Arkin has become one of the pioneers in robot ethics, tuning in at the start of discussions on Roboethics in 2004. Since then he’s written several publications on the ethics of military robots, arguing that robots in the future could be more ethical than humans on the battlefield. Embedded with a sense of ethics, or even guilt, such robots could perhaps be able to make decisions in life-death situations. His views are presented in his new book to appear in spring 2009 entitled Governing Lethal Behavior in Autonomous Robots.

Other than military robots, Arkin discusses issues such as the attachment of people to robots, and more generally, the role of robots as care-givers or workers in the society. He also touches on the subject of medical robots, or prosthetic, capable of enhancing the human being.

Finally, with so many questions raised on the ethics of robotics, we look into the entities which will be setting limits on the use of tomorrows robots as well as defining who takes the responsibly of their actions. Could the robots themselves be held responsible in the end?

Links:

• Download mp3 (21.1MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Robots: Robot Ethics (Part I)
• Ronald Arkin’s Homepage
• Noel Sharkey’s Academic Home Page

Further reading:

• Governing Lethal Behavior in Autonomous Robots by Ronald Arkin
• Wired for War: The Robotics Revolution and Conflict in the 21st Century by P. W. Singer

Latest News:

For more information and discussion on this week’s Robots news,
including a video of the Israel’s new Harop UAV, videos of the iLean and the iHop robots and more information on CMU’s new degree program visit the Robots forum!

View and post comments on this episode in the forum

Link to audio file (44:17)

In this episode we start a two-part special looking at ethical issues in robotics. Given the broad and controversial nature of this topic, we will speak with two world-renowned experts in ethics with often-opposing views. Our first guest, Noel Sharkey is Professor of Public Engagement, Artificial Intelligence and Robotics at the University of Sheffield in the UK. Our second guest, which will be featured in our next episode, is Ronald Arkin, the director of the Mobile Robot Lab and Associate Dean of Research at Georgia Tech in the US. Both researchers discuss issues such as military robots, robots in the society, medical robots and legal responsibilities. Their opinions on these subjects have been widely covered by the media, international organizations and academia. The interviews were recorded individually and both researchers were asked the same questions.

Noel Sharkey

Noel Sharkey is Professor of Artificial Intelligence and Robotics at the University of Sheffield in the UK. Sharkey has been one of the pioneers of robotics from its earliest days, having moved across fields, from philosophy to engineering, psychology and AI. He’s appeared in numerous international television shows, organized robot competitions for young people around the globe, and is the editor for several major journals in robotics. Sharkey also runs his own radio show called Sound of Science, in which he tries to make science accessible to everyone.

In his additional role as Professor of Public Engagement Sharkey’s job is to inform politicians, various public bodies and national and international organisations about ethical problems that may arise in robotics. He is a constant presence in international media when the topics of robotics in the military, policing, child and elderly care crop up, and has recently been interviewed in a parliamentary podcast.

Part 2 in Two Weeks

The debate on robot ethics is not over! Be sure to listen to our next episode in which we speak with Ronald Arkin who presents a very different perspective on the ethical issues in robotics.

Links:

• Download mp3 (26.3MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Noel Sharkey’s Academic Home Page
• Sound of Science
• Ronald Arkin’s Homepage

Further reading:

• Governing Lethal Behavior in Autonomous Robots by Ronald Arkin
• Wired for War: The Robotics Revolution and Conflict in the 21st Century by P. W. Singer

Latest News:

For more information and discussion on this week’s Robots news,
including an upcoming webcast about the remote controlled beetle, sale of your robot Doppelgaenger and a video of the desktop robotic arm visit the Robots forum!

View and post comments on this episode in the forum

Link to audio file (47:22)

In this episode we look at a subject that doesn’t always come to mind when you think of robots: love and relationships. Our first guest David Levy is the author of the book Love and Sex with Robots which has received wide media attention in the past year because it predicts that humans and robots will soon engage in genuine relationships, both physical and emotional.

We then speak to robot anthropologist Kathleen Richardson from Cambridge about her review of Levy’s book and her take on the meaning and likeliness of human-robot couples.

David Levy

David Levy is best known for his many years as a Scottish International Master of Chess, but it’s his recent doctoral thesis entitled “Intimate Relationships with Artificial Partners” at the University of Maastricht in the Netherlands that has been getting him extensive media coverage lately. In an unusual combination of psychology, sociology and robotics Levy studied the dynamics of sex and relationships between humans and robots.
Levy tells us about his belief that robots will soon become a ubiquitous part of our society, and it will be common for people to have relationships and even marry their robotic partners. He explains the dynamics of relationships and sex as he sees them, and how they can be extended to relations with artificial partners or robots as they become more advanced and life-like. Finally he tells us in his own words what he hopes that people take away from his latest book Love and Sex with Robots.

Kathleen Richardson

Kathleen Richardson recently completed her doctoral studies in the department of social anthropology at the University of Cambridge during which she conducted fieldwork in robotic labs at the Massachusetts Institute of Technology (MIT). Her thesis, “Annihilating Difference? Robots and Building Design at MIT”, examined the breakdown of boundaries between humans and non-humans through a study of robots. She has also given several talks on human-robot relationships and her work has been featured in the New York Times.

She’ll be presenting her review of David Levy’s book “Love + Sex with Robots” and will tell us why genuine love relationships between humans and robots are mostly unfounded speculations grounded in science fiction fantasies. She also discusses the attachment that humans can feel for “things” and the ambiguities that might raise.

Links:

• Download mp3 (25.2MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• David Levy on Wikipedia
• Love and Sex with Robots on Amazon
• Kathleen Richardson’s Webpage

Latest News:

For a video of the net launching security robot, a video of the MDS robot Nexi, and more on the functioning of the MCMS micro-grippers have a look at our forum!

View and post comments on this episode in the forum