Alan Winfield – Robohub

A draft open standard for an Ethical Black Box

Alan Winfield — Thu, 19 May 2022 09:36:00 +0000

About 5 years ago we proposed that all robots should be fitted with the robot equivalent of an aircraft Flight Data Recorder to continuously record sensor and relevant internal status data. We call this an ethical black box (EBB). We argued that an ethical black box will play a key role in the processes of discovering why and how a robot caused an accident, and thus an essential part of establishing accountability and responsibility.

Since then, within the RoboTIPS project, we have developed and tested several model EBBs, including one for an e-puck robot that I wrote about in this blog, and another for the MIRO robot. With some experience under our belts, we have now drafted an Open Standard for the EBB for social robots – initially as a paper submitted to the International Conference on Robots Ethics and Standards. Let me now explain first why we need a standard, and second why it should be an open standard.

Why do we need a standard specification for an EBB? As we outline in our new paper, there are four reasons:

A standard approach to EBB implementation in social robots will greatly benefit accident and incident (near miss) investigations.
An EBB will provide social robot designers and operators with data on robot use that can support both debugging and functional improvements to
the robot.
An EBB can be used to support robot ‘explainability’ functions to allow, for instance, the robot to answer ‘Why did you just do that?’ questions from its user. And,
a standard allows EBB implementations to be readily shared and adapted for different robots and, we hope, encourage manufacturers to develop and market general purpose robot EBBs.

And why should it be an Open Standard? Bruce Perens, author of The Open Source Definition, outlines a number of criteria an open standard must satisfy, including:

“Availability: Open standards are available for all to read and implement.
Maximize End-User Choice: Open Standards create a fair, competitive market for implementations of the standard.
No Royalty: Open standards are free for all to implement, with no royalty
or fee.
No Discrimination: Open standards and the organizations that administer them do not favor one implementor over another for any reason other than the technical standards compliance of a vendor’s implementation.
Extension or Subset: Implementations of open standards may be extended, or offered in subset form.”

These are *good* reasons.

The most famous and undoubtedly the most impactful Open Standards are those that were written for the Internet. They were, and still are, called Requests for Comments (RFCs) to reflect the fact that they were – especially in the early years, drafts for revision. As a mark of respect we also regard our draft 0.1 Open Standard for an EBB for Social Robots, as an RFC. You can find draft 0.1 in Annex A of the paper here.

Not only is this a first draft, it is also incomplete, covering only the specification of the data and its format, that should be saved in an EBB for social robots. Given that the EBB data specification is at the heart of the EBB standard, we feel that this is sufficient to be opened up for comments and feedback. We will continue to extend the specification, with subsequent versions also published on arXiv.

Let me know encourage comments and feedback. Please feel free to either submit comments to this blog post – this way everyone can see the comments – or by contacting me directly via email. All constructive comments that result in revisions to the standard will be acknowledged in the standard.

Ethics is the new Quality

Alan Winfield — Sun, 30 May 2021 09:30:00 +0000

I took part in the first panel at the BSI conference The Digital World: Artificial Intelligence. The subject of the panel was AI Governance and Ethics. My co-panelist was Emma Carmel, and we were expertly chaired by Katherine Holden.

Emma and I each gave short opening presentations prior to the Q&A. The title of my talk was Why is Ethical Governance in AI so hard? Something I’ve thought about alot in recent months.

Here are the slides exploring that question.

And here are my words.

Early in 2018 I wrote a short blog post with the title Ethical Governance: what is it and who’s doing it? Good ethical governance is important because in order for people to have confidence in their AI they need to know that it has been developed responsibly. I concluded my piece by asking for examples of good ethical governance. I had several replies, but none were nominating AI companies.

So. why is it that 3 years on we see some of the largest AI companies on the planet shooting themselves in the foot, ethically speaking? I’m not at all sure I can offer an answer but, in the next few minutes, I would like to explore the question: why is ethical governance in AI so hard?

But from a new perspective.

Slide 2

In the early 1970s I spent a few months labouring in a machine shop. The shop was chaotic and disorganised. It stank of machine oil and cigarette smoke, and the air was heavy with the coolant spray used to keep the lathe bits cool. It was dirty and dangerous, with piles of metal swarf cluttering the walkways. There seemed to be a minor injury every day.

Skip forward 40 years and machine shops look very different.

Slide 3

So what happened? Those of you old enough will recall that while British design was world class – think of the British Leyland Mini, or the Jaguar XJ6 – our manufacturing fell far short. “By the mid 1970s British cars were shunned in Europe because of bad workmanship, unreliability, poor delivery dates and difficulties with spares. Japanese car manufacturers had been selling cars here since the mid 60s but it was in the 1970s that they began to make real headway. Japanese cars lacked the style and heritage of the average British car. What they did have was superb build quality and reliability”*.

What happened was Total Quality Management. The order and cleanliness of modern machine shops like this one is a strong reflection of TQM practices.

Slide 4

In the late 1970s manufacturing companies in the UK learned – many the hard way – that ‘quality’ is not something that can be introduced by appointing a quality inspector. Quality is not something that can be hired in.

This word cloud reflects the influence from Japan. The words Japan, Japanese and Kaizen – which roughly translates as continuous improvement – appear here. In TQM everyone shares the responsibility for quality. People at all levels of an organization participate in kaizen, from the CEO to assembly line workers and janitorial staff. Importantly suggestions from anyone, no matter who, are valued and taken equally seriously.

Slide 5

In 2018 my colleague Marina Jirotka and I published a paper on ethical governance in robotics and AI. In that paper we proposed 5 pillars of good ethical governance. The top four are:

have an ethical code of conduct,
train everyone on ethics and responsible innovation,
practice responsible innovation, and
publish transparency reports.

The 5th pillar underpins these four and is perhaps the hardest: really believe in ethics.

Now a couple of months ago I looked again at these 5 pillars and realised that they parallel good practice in Total Quality Management: something I became very familiar with when I founded and ran a company in the mid 1980s.

Slide 6

So, if we replace ethics with quality management, we see a set of key processes which exactly parallel our 5 pillars of good ethical governance, including the underpinning pillar: believe in total quality management.

I believe that good ethical governance needs the kind of corporate paradigm shift that was forced on UK manufacturing industry in the 1970s.

Slide 7

In a nutshell I think ethics is the new quality

Yes, setting up an ethics board or appointing an AI ethics officer can help, but on their own these are not enough. Like Quality, everyone needs to understand and contribute to ethics. Those contributions should be encouraged, valued and acted upon. Nobody should be fired for calling out unethical practices.

Until corporate AI understands this we will, I think, struggle to find companies that practice good ethical governance.

Quality cannot be ‘inspected in’, and nor can ethics.

Thank you.

Notes.

[1] I’m quoting here from the excellent history of British Leyland by Ian Nicholls.

[2] My company did a huge amount of work for Motorola and – as a subcontractor – we became certified software suppliers within their six sigma quality management programme.

[3] It was competitive pressure that forced manufacturing companies in the 1970s to up their game by embracing TQM. Depressingly the biggest AI companies face no such competitive pressures, which is why regulation is both necessary and inevitable.

On sustainable robotics

Alan Winfield — Tue, 20 Apr 2021 09:02:00 +0000

The climate emergency brooks no compromise: every human activity or artefact is either part of the solution or it is part of the problem.

I’ve worried about the sustainability of consumer electronics for some time, and, more recently, the shocking energy costs of big AI. But the climate emergency has also caused me to think hard about the sustainability of robots. In recent papers we have defined responsible robotics as

… the application of Responsible Innovation in the design, manufacture, operation, repair and end-of-life recycling of robots, that seeks the most benefit to society and the least harm to the environment.

I will wager that few robotics manufacturers – even the most responsible – pay much attention to repairability and recyclability of their robots. And, I’m ashamed to say, very little robotics research is focused on the development of sustainable robots. A search on google scholar throws up a handful of excellent papers detailing work on upcycled and sustainable robots (2018), sustainable robotics for smart cities (2018), green marketing of sustainable robots (2019), and sustainable soft robots (2020).

I was therefore delighted when, a few weeks ago, my friend and colleague Michael Fisher, drafted a proposal for a new standard on Sustainable Robotics. The proposal received strong support from the BSI robotics committee. Here is the formal notice requesting comments on Michael’s proposal: BS XXXX Guide to the Sustainable Design and Application of Robotic Systems.

So what would make a robot sustainable? In my view it would have to be:

Made from sustainable materials. This means the robot should, as far as possible, use recycled materials (plastics or metals), or biodegradable materials like wood. Any new materials should be ethically sourced.
Low energy. The robot should be designed to use as little energy as possible. It should have energy saving modes. If an outdoor robot then is should use solar cells and/or hydrogen cells when they become small enough for mobile robots. Battery powered robots should always be rechargeable.
Repairable. The robot would be designed for ease of repair using modular, replaceable parts as much as possible – especially the battery. Additionally the manufacturers should provide a repair manual so that local workshops could fix most faults.
Recyclable. Robots will eventually come to the end of their useful life, and if they cannot be repaired or recycled we risk them being dumped in landfill. To reduce this risk the robot should be designed to make it easy re-use parts, such as electronics and motors, and re-cycle batteries, metals and plastics.

These are, for me, the four fundamental requirements, but there are others. The BSI proposal adds also the environmental effects of deployment (it is unlikely we would consider a sustainable robot designed to spray pesticides as truly sustainable), or of failure in the field. Also the environmental effect of maintenance; cleaning materials, for instance. The proposal also looks toward sustainable, upcyclable robots as part of a circular economy.

This is Ecobot III, developed some years ago by colleagues in the Bristol Robotics Lab’s Bio-energy group. The robot runs on electricity extracted from biomass by 48 microbial fuel cells (the two concentric brick coloured rings). The robot is 90% 3D printed, and the plastic is recyclable.

I would love to see, in the near term, not only a new standard on Sustainable Robotics as a guide (and spur) for manufacturers, but the emergence of Sustainable Robotics as a thriving new sub-discipline in robotics.

Back to Robot Coding part 3: testing the EBB

Alan Winfield — Tue, 23 Mar 2021 15:43:00 +0000

In part 2 a few weeks ago I outlined a Python implementation of the ethical black box. I described the key data structure – a dictionary which serves as both specification for the type of robot, and the data structure used to deliver live data to the EBB. I also mentioned the other key robot specific code:

# Get data from the robot and store it in data structure spec

def getRobotData(spec):

Having reached this point I needed a robot – and a way of communicating with it – so that I could both write getRobotData(spec) and test the EBB. But how to do this? I’m working from home during lockdown, and my e-puck robots are all in the lab. Then I remembered that the excellent robot simulator V-REP (now called CoppeliaSim) has a pretty good e-puck model and some nice demo scenes. V-REP also offers multiple ways of communicating between simulated robots and external programs (see here). One of them – TCP/IP sockets – appeals to me as I’ve written sockets code many times, for both real-world and research applications. Then a stroke of luck: I found that a team at Ensta-Bretagne had written a simple demo which does more or less what I need – just not for the e-puck. So, first I got that demo running and figured out how it works, then used the same approach for a simulated e-puck and the EBB. Here is a video capture of the working demo.

So, what’s going on in the demo? The visible simulation views in the V-REP window show an e-puck robot following a black line which is blocked by both a potted plant and an obstacle constructed from 3 cylinders. The robot has two behaviours: line following and wall following. The EBB requests data from the e-puck robot once per second, and you can see those data in the Python shell window. Reading from left to right you will see first the EBB date and time stamp, then robot time botT, then the 3 line following sensors lfSe, followed by the 8 infra red proximity sensors irSe. The final two fields show the joint (i.e. wheel angles) jntA, in degrees, then the motor commands jntD. By watching these values as the robot follows its line and negotiates the two obstacles you can see how the line and infra red sensor values change, resulting in updated motor commands.

Here is the code – which is custom written both for this robot and the means of communicating with it – for requesting data from the robot.

# Get data from the robot and store it in spec[]

# while returning one of the following result codes

ROBOT_DATA_OK = 0

CANNOT_CONNECT = 1

SOCKET_ERROR = 2

BAD_DATA = 3

def getRobotData(spec):

    # This function connects, via TCP/IP to an ePuck robot running in V-REP

    # create a TCP/IP socket and connect it to the simulated robot

    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    try:

        sock.connect(server_address_port)

    except:

        return CANNOT_CONNECT

    sock.settimeout(0.1) # set connection timeout

    # pack a dummy packet that will provoke data in response

    #   this is, in effect, a ‘ping’ to ask for a data record

    strSend = struct.pack(‘fff’,1.0,1.0,1.0)

    sock.sendall(strSend) # and send it to V-REP

    # wait for data back from V-REP

    #   expect a packet with 1 time, 2 joints, 2 motors, 3 line sensors, 8 irSensors  

    #   all floats because V-REP

    #   total packet size = 16 x 4 = 64 bytes

    data = b”

    nch_rx = 64 # expect this many bytes from  V-REP 

    try:

        while len(data) < nch_rx:

            data += sock.recv(nch_rx)

    except:

        sock.close()

        return SOCKET_ERROR

    # unpack the received data

    if len(data) == nch_rx:

        # V-REP packs and unpacks in floats only so…

        vrx = struct.unpack(‘ffffffffffffffff’,data)

        # now move data from vrx[] into spec[], while rounding the floats

        spec[“botTime”] = [ round(vrx[0],2) ] 

        spec[“jntDemands”] = [ round(vrx[1],2), round(vrx[2],2) ]

        spec[“jntAngles”] = [round(vrx[3]*180.0/math.pi,2)

                             round(vrx[4]*180.0/math.pi,2) ]

        spec[“lfSensors”] = [ round(vrx[5],2), round(vrx[6],2), round(vrx[7],2) ]

        for i in range(8):

            spec[“irSensors”][i] = round(vrx[8+i],3)       

        result = ROBOT_DATA_OK

    else:       

        result = BAD_DATA

    sock.close()

    return result

The structure of this function is very simple: first create a socket then open it, then make a dummy packet and send it to V-REP to request EBB data from the robot. Then, when a data packet arrives, unpack it into spec. The most complex part of the code is data wrangling.

Would a real EBB collect data in this way? Well if the EBB is embedded in the robot then probably not. Communication between the robot controller and the EBB might be via ROS messages, or even more directly, by – for instance – allowing the EBB code to access a shared memory space which contains the robot’s sensor inputs, command outputs and decisions. But an external EBB, either running on a local server or in the cloud, would most likely use TCP/IP to communicate with the robot, so getRobotData() would look very much like the example here.

Back to Robot Coding part 2: the ethical black box

Alan Winfield — Sat, 20 Feb 2021 08:32:00 +0000

In the last few days I started some serious coding. The first for 20 years, in fact, when I built the software for the BRL LinuxBots. (The coding I did six months ago doesn’t really count as I was only writing or modifying small fragments of Python).

My coding project is to start building an ethical black box (EBB), or to be more accurate, a module that will allow a software EBB to be incorporated into a robot. Conceptually the EBB is very simple, it is a data logger – the robot equivalent of an aircraft Flight Data Recorder, or an automotive Event Data Recorder. Nearly five years ago I made the case, with Marina Jirotka, that all robots (and AIs) should be fitted with an EBB as standard. Our argument is very simple: without an EBB, it will be more or less impossible to investigate robot accidents, or near-misses, and in a recent paper on Robot Accident Investigation we argue that with the increasing use of social robots accidents are inevitable and will need to be investigated.

Developing and demonstrating the EBB is a foundational part of our 5-year EPSRC funded project RoboTIPS, so it’s great to be doing some hands-on practical research. Something I’ve not done for awhile.

Here is a block diagram showing the EBB and its relationship with a robot controller.

As shown here the data flows from the robot controller to the EBB are strictly one way. The EBB cannot and must not interfere with the operation of the robot. Coding an EBB for a particular robot would be straightforward, but I have set myself a tougher goal: a generic EBB module (i.e. library of functions) that would – with some inevitable customisation – apply to any robot. And I set myself the additional challenge of coding in Python, making use of skills learned from the excellent online Codecademy Python 2 course.

There are two elements of the EBB that must be customised for a particular robot. The first is the data structure used to fetch and save the sensor, actuator and decision data in the diagram above. Here is an example from my first stab at an EBB framework, using the Python dictionary structure:

# This dictionary structure serves as both

# 1 specification of the type of robot, and each data field that

# will be logged for this robot, &

# 2 the data structure we use to deliver live data to the EBB

# for this model let us create a minimal spec for an ePuck robot

epuckSpec = {

# the first field *always* identifies the type of robot plus # version and serial nos

“robot” : [“ePuck”, “v1”, “SN123456”],

# the remaining fields are data we will log,

# starting with the motors

# ..of which the ePuck has just 2: left and right

“motors” : [0,0],

# then 8 infra red sensors

“irSensors” : [0,0,0,0,0,0,0,0],

# ..note the ePuck has more sensors: accelerometer, camera etc,

# but this will do for now

# ePuck battery level

“batteryLevel” : [0],

# then 1 decision code – i.e. what the robot is doing now

# what these codes mean will be specific to both the robot

# and the application

“decisionCode” : [0]

}

Whether a dictionary is the best way of doing this I’m not 100% sure, being new to Python (any thoughts from experienced Pythonistas welcome).

The idea is that all robot EBBs will need to define a data structure like this. All must contain the first field “robot”, which names the robot’s type, its version number and serial number. Then the following fields must use keywords from a standard menu, as needed. As shown in this example each keyword is followed by a list of placeholder values – in which the number of values in the list reflects the specification of the actual robot. The ePuck robot, for instance, has 2 motors and 8 infra-red sensors.

The final field in the data structure is “decisionCode”. The values stored in this field would be both robot and applications specific; for the ePuck robot these might be 1 = ‘stop’, 2 = ‘turn left’, 3 = ‘turn right’ and so on. We could add another value for a parameter, so the robot might decide for instance to turn left 40 degrees, so “decisionCode” : [2,40]. We could also add a ‘reason’ field, which would save the high-level reason for the decision, as in “decisionCode” : [2,40,”avoid obstacle right”] noting that the decision field could be a string as shown here, or a numeric code.

As I hope I have shown here the design of this data structure and its fields is at the heart of the EBB.

The second element of the EBB library that must be written for the particular robot and application, is the function which fetches data from the robot

# Get data from the robot and store it in data structure spec

def getRobotData(spec):

How this function is implemented will vary hugely between robots and robot applications. For our Linux enhanced ePucks with WiFi connections this is likely to be via a TCP/IP client-server, with the server running on the robot, sending data following a request from the client getRobotData(ePuckspec). For simpler setups in which the EBB module is folded into the robot controller then accessing the required data within getRobotData() should be very straightforward.

The generic part of the EBB module will define the class EBB, with methods for both initialising the EBB and saving a new data record to the EBB. I will cover that in another blog post.

Before closing let me add that it is our intention to publish the specification of the EBB, together with the model EBB code, once it had been fully tested, as open source.

Any comments or feedback would be much appreciated.

Link to the original post here.

RoboTED: a case study in Ethical Risk Assessment

Alan Winfield — Wed, 28 Oct 2020 20:39:00 +0000

A few weeks ago I gave a short paper at the excellent International Conference on Robot Ethics and Standards (ICRES 2020), outlining a case study in Ethical Risk Assessment – see our paper here. Our chosen case study is a robot teddy bear, inspired by one of my favourite movie robots: Teddy, in A. I. Artificial Intelligence.

Although Ethical Risk Assessment (ERA) is not new – it is after all what research ethics committees do – the idea of extending traditional risk assessment, as practised by safety engineers, to cover ethical risks is new. ERA is I believe one of the most powerful tools available to the responsible roboticist, and happily we already have a published standard setting out a guideline on ERA for robotics in BS 8611, published in 2016.

Before looking at the ERA, we need to summarise the specification of our fictional robot teddy bear: RoboTed. First, RoboTed is based on the following technology:

RoboTed is an Internet (WiFi) connected device,
RoboTed has cloud-based speech recognition and conversational AI (chatbot) and local speech synthesis,
RoboTed’s eyes are functional cameras allowing RoboTed to recognise faces,
RoboTed has motorised arms and legs to provide it with limited baby-like movement and locomotion.

And second RoboTed is designed to:

Recognise its owner, learning their face and name and turning its face toward the child.
Respond to physical play such as hugs and tickles.
Tell stories, while allowing a child to interrupt the story to ask questions or ask for sections to be repeated.
Sing songs, while encouraging the child to sing along and learn the song.
Act as a child minder, allowing parents to both remotely listen, watch and speak via RoboTed.

The tables below summarise the ERA of RoboTED for (1) psychological, (2) privacy & transparency and (3) environmental risks. Each table has 4 columns, for the hazard, risk, level of risk (high, medium or low) and actions to mitigate the risk. BS8611 defines an ethical risk as the “probability of ethical harm occurring from the frequency and severity of exposure to a hazard”; an ethical hazard as “a potential source of ethical harm”, and an ethical harm as “anything likely to compromise psychological and/or societal and environmental well-being”.

(1) Psychological Risks

(2) Security and Transparency Risks

(3) Environmental Risks

For a more detailed commentary on each of these tables see our full paper – which also, for completeness, covers physical (safety) risks. And here are the slides from my short ICRES 2020 presentation:

Through this fictional case study we argue we have demonstrated the value of ethical risk assessment. Our RoboTed ERA has shown that attention to ethical risks can

suggest new functions, such as “RoboTed needs to sleep now”,
draw attention to how designs can be modified to mitigate some risks,
highlight the need for user engagement, and
reject some product functionality as too risky.

But ERA is not guaranteed to expose all ethical risks. It is a subjective process which will only be successful if the risk assessment team are prepared to think both critically and creatively about the question: what could go wrong? As Shannon Vallor and her colleagues write in their excellent Ethics in Tech Practice toolkit design teams must develop the “habit of exercising the skill of moral imagination to see how an ethical failure of the project might easily happen, and to understand the preventable causes so that they can be mitigated or avoided”.

My top three policy and governance issues in AI/ML

Alan Winfield — Sat, 22 Jun 2019 08:31:00 +0000

In preparation for a meeting of the WEF global AI council today, we were asked the question:

What do you think are the top three policy and governance issues that face AI/ML currently?

Here are my answers.

1. For me the biggest governance issue facing AI/ML ethics is the gap between principles and practice. The hard problem the industry faces is turning good intentions into demonstrably good behaviour. In the last 2.5 years there has been a gold rush of new ethical principles in AI. Since Jan 2017 at least 22 sets of ethical principles have been published, including principles from Google, IBM, Microsoft and Intel. Yet any evidence that these principles are making a difference within those companies is hard to find – leading to a justifiable accusation of ethics-washing– and if anything the reputations of some leading AI companies are looking increasingly tarnished.

2. Like others I am deeply concerned by the acute gender imbalance in AI (estimates of the proportion of women in AI vary between ~12% and ~22%). This is not just unfair, I believe it too be positively dangerous, since it is resulting in AI products and services that reflect the values and ambitions of (young, predominantly white) men. This makes it a governance issue. I cannot help wondering if the deeply troubling rise of surveillance capitalism is not, at least in part, a consequence of male values.

3. A major policy concern is the apparently very poor quality of many of the jobs created by the large AI/ML companies. Of course the AI/ML engineers are paid exceptionally well, but it seems that there is a very large number of very poorly paid workers who, in effect, compensate for the fact that AI is not (yet) capable of identifying offensive content, nor is it able to learn without training data generated from large quantities of manually tagged objects in images, nor can conversational AI manage all queries that might be presented to it. This hidden army of piece workers, employed in developing countries by third party sub contractors and paid very poorly, are undertaking work that is at best extremely tedious (you might say robotic) and at worst psychologically very harmful; this has been called AI’s dirty little secret and should not – in my view – go unaddressed.

An updated round up of ethical principles of robotics and AI

Alan Winfield — Sun, 12 May 2019 22:26:00 +0000

This blogpost is an updated round up of the various sets of ethical principles of robotics and AI that have been proposed to date, ordered by date of first publication.

I previously listed principles published before December 2017 here; this blogpost appends those principles drafted since January 2018 (plus one in October 2017 I had missed). The principles are listed here (in full or abridged) with links, notes and references but without critique.

Scroll down to the next horizontal line for the updates.

If there any (prominent) ones I’ve missed please let me know.

Asimov’s three laws of Robotics (1950)

A robot may not injure a human being or, through inaction, allow a human being to come to harm.
A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

I have included these to explicitly acknowledge, firstly, that Asimov undoubtedly established the principle that robots (and by extension AIs) should be governed by principles, and secondly that many subsequent principles have been drafted as a direct response. The three laws first appeared in Asimov’s short story Runaround [1]. This wikipedia article provides a very good account of the three laws and their many (fictional) extensions.

Murphy and Wood’s three laws of Responsible Robotics (2009)

A human may not deploy a robot without the human-robot work system meeting the highest legal and professional standards of safety and ethics.
A robot must respond to humans as appropriate for their roles.
A robot must be endowed with sufficient situated autonomy to protect its own existence as long as such protection provides smooth transfer of control which does not conflict with the First and Second Laws.

These were proposed in Robin Murphy and David Wood’s paper Beyond Asimov: The Three Laws of Responsible Robotics [2].

EPSRC Principles of Robotics (2010)

Robots are multi-use tools. Robots should not be designed solely or primarily to kill or harm humans, except in the interests of national security.
Humans, not Robots, are responsible agents. Robots should be designed and operated as far as practicable to comply with existing laws, fundamental rights and freedoms, including privacy.
Robots are products. They should be designed using processes which assure their safety and security.
Robots are manufactured artefacts. They should not be designed in a deceptive way to exploit vulnerable users; instead their machine nature should be transparent.
The person with legal responsibility for a robot should be attributed.

These principles were drafted in 2010 and published online in 2011, but not formally published until 2017 [3] as part of a two-part special issue of Connection Science on the principles, edited by Tony Prescott & Michael Szollosy [4]. An accessible introduction to the EPSRC principles was published in New Scientist in 2011.

Future of Life Institute Asilomar principles for beneficial AI (Jan 2017)

I will not list all 23 principles but extract just a few to compare and contrast with the others listed here:

6. Safety: AI systems should be safe and secure throughout their operational lifetime, and verifiably so where applicable and feasible.
7. Failure Transparency: If an AI system causes harm, it should be possible to ascertain why.
8. Judicial Transparency: Any involvement by an autonomous system in judicial decision-making should provide a satisfactory explanation auditable by a competent human authority.
9. Responsibility: Designers and builders of advanced AI systems are stakeholders in the moral implications of their use, misuse, and actions, with a responsibility and opportunity to shape those implications.
10. Value Alignment: Highly autonomous AI systems should be designed so that their goals and behaviors can be assured to align with human values throughout their operation.

11. Human Values: AI systems should be designed and operated so as to be compatible with ideals of human dignity, rights, freedoms, and cultural diversity.
12. Personal Privacy: People should have the right to access, manage and control the data they generate, given AI systems’ power to analyze and utilize that data.
13. Liberty and Privacy: The application of AI to personal data must not unreasonably curtail people’s real or perceived liberty.
14. Shared Benefit: AI technologies should benefit and empower as many people as possible.
15. Shared Prosperity: The economic prosperity created by AI should be shared broadly, to benefit all of humanity.

An account of the development of the Asilomar principles can be found here.

The ACM US Public Policy Council Principles for Algorithmic Transparency and Accountability (Jan 2017)

Awareness: Owners, designers, builders, users, and other stakeholders of analytic systems should be aware of the possible biases involved in their design, implementation, and use and the potential harm that biases can cause to individuals and society.
Access and redress: Regulators should encourage the adoption of mechanisms that enable questioning and redress for individuals and groups that are adversely affected by algorithmically informed decisions.
Accountability: Institutions should be held responsible for decisions made by the algorithms that they use, even if it is not feasible to explain in detail how the algorithms produce their results.
Explanation: Systems and institutions that use algorithmic decision-making are encouraged to produce explanations regarding both the procedures followed by the algorithm and the specific decisions that are made. This is particularly important in public policy contexts.
Data Provenance: A description of the way in which the training data was collected should be maintained by the builders of the algorithms, accompanied by an exploration of the potential biases induced by the human or algorithmic data-gathering process.
Auditability: Models, algorithms, data, and decisions should be recorded so that they can be audited in cases where harm is suspected.
Validation and Testing: Institutions should use rigorous methods to validate their models and document those methods and results.

See the ACM announcement of these principles here. The principles form part of the ACM’s updated code of ethics.

Japanese Society for Artificial Intelligence (JSAI) Ethical Guidelines (Feb 2017)

Contribution to humanity Members of the JSAI will contribute to the peace, safety, welfare, and public interest of humanity.
Abidance of laws and regulations Members of the JSAI must respect laws and regulations relating to research and development, intellectual property, as well as any other relevant contractual agreements. Members of the JSAI must not use AI with the intention of harming others, be it directly or indirectly.
Respect for the privacy of others Members of the JSAI will respect the privacy of others with regards to their research and development of AI. Members of the JSAI have the duty to treat personal information appropriately and in accordance with relevant laws and regulations.
Fairness Members of the JSAI will always be fair. Members of the JSAI will acknowledge that the use of AI may bring about additional inequality and discrimination in society which did not exist before, and will not be biased when developing AI.
Security As specialists, members of the JSAI shall recognize the need for AI to be safe and acknowledge their responsibility in keeping AI under control.
Act with integrity Members of the JSAI are to acknowledge the significant impact which AI can have on society.
Accountability and Social Responsibility Members of the JSAI must verify the performance and resulting impact of AI technologies they have researched and developed.
Communication with society and self-development Members of the JSAI must aim to improve and enhance society’s understanding of AI.
Abidance of ethics guidelines by AI AI must abide by the policies described above in the same manner as the members of the JSAI in order to become a member or a quasi-member of society.

An explanation of the background and aims of these ethical guidelines can be found here, together with a link to the full principles (which are shown abridged above).

Draft principles of The Future Society’s Science, Law and Society Initiative (Oct 2017)

AI should advance the well-being of humanity, its societies, and its natural environment.
AI should be transparent.
Manufacturers and operators of AI should be accountable.
AI’s effectiveness should be measurable in the real-world applications for which it is intended.
Operators of AI systems should have appropriate competencies.
The norms of delegation of decisions to AI systems should be codified through thoughtful, inclusive dialogue with civil society.

This article by Nicolas Economou explains the 6 principles with a full commentary on each one.

Montréal Declaration for Responsible AI draft principles (Nov 2017)

Well-being The development of AI should ultimately promote the well-being of all sentient creatures.
Autonomy The development of AI should promote the autonomy of all human beings and control, in a responsible way, the autonomy of computer systems.
Justice The development of AI should promote justice and seek to eliminate all types of discrimination, notably those linked to gender, age, mental / physical abilities, sexual orientation, ethnic/social origins and religious beliefs.
Privacy The development of AI should offer guarantees respecting personal privacy and allowing people who use it to access their personal data as well as the kinds of information that any algorithm might use.
Knowledge The development of AI should promote critical thinking and protect us from propaganda and manipulation.
Democracy The development of AI should promote informed participation in public life, cooperation and democratic debate.
Responsibility The various players in the development of AI should assume their responsibility by working against the risks arising from their technological innovations.

The Montréal Declaration for Responsible AI proposes the 7 values and draft principles above (here in full with preamble, questions and definitions).

IEEE General Principles of Ethical Autonomous and Intelligent Systems (Dec 2017)

How can we ensure that A/IS do not infringe human rights?
Traditional metrics of prosperity do not take into account the full effect of A/IS technologies on human well-being.
How can we assure that designers, manufacturers, owners and operators of A/IS are responsible and accountable?
How can we ensure that A/IS are transparent?
How can we extend the benefits and minimize the risks of AI/AS technology being misused?

These 5 general principles appear in Ethically Aligned Design v2, a discussion document drafted and published by the IEEE Standards Association Global Initiative on Ethics of Autonomous and Intelligent Systems. The principles are expressed not as rules but instead as questions, or concerns, together with background and candidate recommendations.

A short article co-authored with IEEE general principles co-chair Mark Halverson Why Principles Matter explains the link between principles and standards, together with further commentary and references.

Note that these principles have been revised and extended, in March 2019 (see below).

UNI Global Union Top 10 Principles for Ethical AI (Dec 2017)

Demand That AI Systems Are Transparent
Equip AI Systems With an Ethical Black Box
Make AI Serve People and Planet
Adopt a Human-In-Command Approach
Ensure a Genderless, Unbiased AI
Share the Benefits of AI Systems
Secure a Just Transition and Ensuring Support for Fundamental Freedoms and Rights
Establish Global Governance Mechanisms
Ban the Attribution of Responsibility to Robots
Ban AI Arms Race

Drafted by UNI Global Union‘s Future World of Work these 10 principles for Ethical AI (set out here with full commentary) “provide unions, shop stewards and workers with a set of concrete demands to the transparency, and application of AI”.

Updated principles…

Intel’s recommendation for Public Policy Principles on AI (October 2017)

Foster Innovation and Open Development – To better understand the impact of AI and explore the broad diversity of AI implementations, public policy should encourage investment in AI R&D. Governments should support the controlled testing of AI systems to help industry, academia, and other stakeholders improve the technology.
Create New Human Employment Opportunities and Protect People’s Welfare – AI will change the way people work. Public policy in support of adding skills to the workforce and promoting employment across different sectors should enhance employment opportunities while also protecting people’s welfare.
Liberate Data Responsibly – AI is powered by access to data. Machine learning algorithms improve by analyzing more data over time; data access is imperative to achieve more enhanced AI model development and training. Removing barriers to the access of data will help machine learning and deep learning reach their full potential.
Rethink Privacy – Privacy approaches like The Fair Information Practice Principles and Privacy by Design have withstood the test of time and the evolution of new technology. But with innovation, we have had to “rethink” how we apply these models to new technology.
Require Accountability for Ethical Design and Implementation – The social implications of computing have grown and will continue to expand as more people have access to implementations of AI. Public policy should work to identify and mitigate discrimination caused by the use of AI and encourage designing in protections against these harms.

These principles were announced in a blog post by Naveen Rao (Intel VP AI) here.

Lords Select Committee 5 core principles to keep AI ethical (April 2018)

Artificial intelligence should be developed for the common good and
benefit of humanity.
Artificial intelligence should operate on principles of intelligibility and
fairness.
Artificial intelligence should not be used to diminish the data rights or
privacy of individuals, families or communities.
All citizens have the right to be educated to enable them to flourish
mentally, emotionally and economically alongside artificial intelligence.
The autonomous power to hurt, destroy or deceive human beings
should never be vested in artificial intelligence.

These principles appear in the UK House of Lords Select Committee on Artificial Intelligence report AI in the UK: ready, willing and able? published in April 2019. The WEF published a summary and commentary here.
AI UX: 7 Principles of Designing Good AI Products (April 2018)

Differentiate AI content visually – let people know if an algorithm has generated a piece of content so they can decide for themselves whether to trust it or not.
Explain how machines think – helping people understand how machines work so they can use them better
Set the right expectations – especially in a world full of sensational, superficial news about new AI technologies.
Find and handle weird edge cases – spend more time testing and finding weird, funny, or even disturbing or unpleasant edge cases.
User testing for AI products (default methods won’t work here).
Provide an opportunity to give feedback.

These principles, focussed on the design of the User Interface (UI) and User Experience (UX), are from Budapest based company UX Studio.

The Toronto Declaration on equality and non-discrimination in machine learning systems (May 2018)

The Toronto Declaration: Protecting the right to equality and non-discrimination in machine learning systems does not succinctly articulate ethical principles but instead presents arguments under the following headings to address concerns “…about the capability of [machine learning] systems to facilitate intentional or inadvertent discrimination against certain individuals or groups of people”.

Using the framework of international human rights law The right to equality and non-discrimination; Preventing discrimination, and Protecting the rights of all individuals and groups: promoting diversity and inclusion
Duties of states: human rights obligations State use of machine learning systems; Promoting equality, and Holding private sector actors to account
Responsibilities of private sector actors human rights due diligence
The right to an effective remedy

Google AI Principles (June 2018)

Be socially beneficial.
Avoid creating or reinforcing unfair bias.
Be built and tested for safety.
Be accountable to people.
Incorporate privacy design principles.
Uphold high standards of scientific excellence.
Be made available for uses that accord with these principles.

These principles were launched with a blog post and commentary by Google CEO Sundar Pichai here.

IBM’s 5 ethical AI principles (September 2018)

Accountability: AI designers and developers are responsible for considering AI design, development, decision processes, and outcomes.
Value alignment: AI should be designed to align with the norms and values of your user group in mind.
Explainability: AI should be designed for humans to easily perceive, detect, and understand its decision process, and the predictions/recommendations. This is also, at times, referred to as interpretability of AI. Simply speaking, users have all rights to ask the details on the predictions made by AI models such as which features contributed to the predictions by what extent. Each of the predictions made by AI models should be able to be reviewed.
Fairness: AI must be designed to minimize bias and promote inclusive representation.
User data rights: AI must be designed to protect user data and preserve the user’s power over access and uses

For a full account read IBM’s Everyday Ethics for Artificial Intelligence here.

Microsoft Responsible bots: 10 guidelines for developers of
conversational AI (November 2018)

Articulate the purpose of your bot and take special care if your bot will support consequential use cases.
Be transparent about the fact that you use bots as part of your product or service.
Ensure a seamless hand-off to a human where the human-bot exchange leads to interactions that exceed the bot’s competence.
Design your bot so that it respects relevant cultural norms and guards against misuse.
Ensure your bot is reliable.
Ensure your bot treats people fairly.
Ensure your bot respects user privacy.
Ensure your bot handles data securely.
Ensure your bot is accessible.
Accept responsibility.

Microsoft’s guidelines for the ethical design of ‘bots’ (chatbots or conversational AIs) are fully described here.

CEPEJ European Ethical Charter on the use of artificial
intelligence (AI) in judicial systems and their environment, 5 principles (February 2019)

Principle of respect of fundamental rights: ensuring that the design and implementation of artificial intelligence tools and services are compatible with fundamental rights.
Principle of non-discrimination: specifically preventing the development or intensification of any discrimination between individuals or groups of individuals.
Principle of quality and security: with regard to the processing of judicial decisions and data, using certified sources and intangible data with models conceived in a multi-disciplinary manner, in a secure technological environment.
Principle of transparency, impartiality and fairness: making data processing methods accessible and understandable, authorising external audits.
Principle “under user control”: precluding a prescriptive approach and ensuring that users are informed actors and in control of their choices.

The Council of Europe ethical charter principles are outlined here, with a link to the ethical charter istelf.
Women Leading in AI (WLinAI) 10 recommendations (February 2019)

Introduce a regulatory approach governing the deployment of AI which mirrors that used for the pharmaceutical sector.
Establish an AI regulatory function working alongside the Information Commissioner’s Office and Centre for Data Ethics – to audit algorithms, investigate complaints by individuals,issue notices and fines for breaches of GDPR and equality and human rights law, give wider guidance, spread best practice and ensure algorithms must be fully explained to users and open to public scrutiny.
Introduce a new Certificate of Fairness for AI systems alongside a ‘kite mark’ type scheme to display it. Criteria to be defined at industry level, similarly to food labelling regulations.
Introduce mandatory AIAs (Algorithm Impact Assessments) for organisations employing AI systems that have a significant effect on individuals.
Introduce a mandatory requirement for public sector organisations using AI for particular purposes to inform citizens that decisions are made by machines, explain how the decision is reached and what would need to change for individuals to get a different outcome.
Introduce a ‘reduced liability’ incentive for companies that have obtained a Certificate of Fairness to foster innovation and competitiveness.
To compel companies and other organisations to bring their workforce with them – by publishing the impact of AI on their workforce and offering retraining programmes for employees whose jobs are being automated.
Where no redeployment is possible, to compel companies to make a contribution towards a digital skills fund for those employees
To carry out a skills audit to identify the wide range of skills required to embrace the AI revolution.
To establish an education and training programme to meet the needs identified by the skills audit, including content on data ethics and social responsibility. As part of that, we recommend the set up of a solid, courageous and rigorous programme to encourage young women and other underrepresented groups into technology.

Presented by the Women Leading in AI group at a meeting in parliament in February 2019, this report in Forbes by Noel Sharkey outlines both the group, their recommendations, and the meeting.

The NHS’s 10 Principles for AI + Data (February 2019)

Understand users, their needs and the context
Define the outcome and how the technology will contribute to it
Use data that is in line with appropriate guidelines for the purpose for which it is being used
Be fair, transparent and accountable about what data is being used
Make use of open standards
Be transparent about the limitations of the data used and algorithms deployed
Show what type of algorithm is being developed or deployed, the ethical examination of how the data is used, how its performance will be validated and how it will be integrated into health and care provision
Generate evidence of effectiveness for the intended use and value for money
Make security integral to the design
Define the commercial strategy

These principles are set out with full commentary and elaboration on Artificial Lawyer here.

IEEE General Principles of Ethical Autonomous and Intelligent Systems (A/IS) (March 2019)

Human Rights: A/IS shall be created and operated to respect, promote, and protect internationally recognized human rights.
Well-being: A/IS creators shall adopt increased human well-being as a primary success criterion for development.
Data Agency: A/IS creators shall empower individuals with the ability to access and securely share their data to maintain people’s capacity to have control over their identity.
Effectiveness: A/IS creators and operators shall provide evidence of the effectiveness and fitness for purpose of A/IS.
Transparency: the basis of a particular A/IS decision should always be discoverable.
Accountability: A/IS shall be created and operated to provide an unambiguous rationale for all decisions made.
Awareness of Misuse: A/IS creators shall guard against all potential misuses and risks of A/IS in operation.
Competence: A/IS creators shall specify and operators shall adhere to the knowledge and skill required for safe and effective operation.

These amended and extended general principles form part of Ethical Aligned Design 1st edition, published in March 2019. For an overview see pdf here.

Ethical issues arising from the police use of live facial
recognition technology (March 2019)

9 ethical principles relate to: public interest, effectiveness, the avoidance of bias and algorithmic justice, impartiality and deployment, necessity, proportionality, impartiality, accountability, oversight, and the construction of watchlists, public trust, and cost effectiveness.

Reported here the UK government’s independent Biometrics and Forensics Ethics Group (BFEG) published an interim report outlining nine ethical principles forming a framework to guide policy on police facial recognition systems.

Floridi and Clement Jones’ five principles key to any
ethical framework for AI (March 2019)

AI must be beneficial to humanity.
AI must also not infringe on privacy or undermine security.
AI must protect and enhance our autonomy and ability to take decisions and choose between alternatives.
AI must promote prosperity and solidarity, in a fight against inequality, discrimination, and unfairness
We cannot achieve all this unless we have AI systems that are understandable in terms of how they work (transparency) and explainable in terms of how and why they reach the conclusions they do (accountability).

Luciano Floridi and Lord Tim Clement Jones set out, here in the New Statesman, these 5 general ethical principles for AI, with additional commentary.

The European Commission’s High Level Expert Group on AI Ethics Guidelines for Trustworthy AI (April 2019)

Human agency and oversight AI systems should support human autonomy and decision-making, as prescribed by the principle of respect for human autonomy.
Technical robustness and safety A crucial component of achieving Trustworthy AI is technical robustness, which is closely linked to the principle of prevention of harm.
Privacy and Data governance Closely linked to the principle of prevention of harm is privacy, a fundamental right particularly affected by AI systems.
Transparency This requirement is closely linked with the principle of explicability and encompasses transparency of elements relevant to an AI system: the data, the system and the business models.
Diversity, non-discrimination and fairness In order to achieve Trustworthy AI, we must enable inclusion and diversity throughout the entire AI system’s life cycle.
Societal and environmental well-being In line with the principles of fairness and prevention of harm, the broader society, other sentient beings and the environment should be also considered as stakeholders throughout the AI system’s life cycle.
Accountability The requirement of accountability complements the above requirements, and is closely linked to the principle of fairness

For more detail on each of these principles follow the links above.

Published on 8 April 2019, the EU HLEG AI ethics guidelines for trustworthy AI are detailed in full here.

Draft core principles of Australia’s Ethics Framework for AI (April 2019)

Generates net-benefits. The AI system must generate benefits for people that are greater than the costs.
Do no harm. Civilian AI systems must not be designed to harm or deceive people and should be implemented in ways that minimise any negative outcomes.
Regulatory and legal compliance. The AI system must comply with all relevant international, Australian Local, State/Territory and Federal government obligations, regulations and laws.
Privacy protection. Any system, including AI systems, must ensure people’s private data is protected and kept confidential plus prevent data breaches which could cause reputational, psychological, financial, professional or other types of harm.
Fairness. The development or use of the AI system must not result in unfair discrimination against individuals, communities or groups. This requires particular attention to ensure the “training data” is free from bias or characteristics which may cause the algorithm to behave unfairly.
Transparency & Explainability. People must be informed when an algorithm is being used that impacts them and they should be provided with information about what information the algorithm uses to make decisions.
Contestability. When an algorithm impacts a person there must be an efficient process to allow that person to challenge the use or output of the algorithm.
Accountability. People and organisations responsible for the creation and implementation of AI algorithms should be identifiable and accountable for the impacts of that algorithm, even if the impacts are unintended.

These draft principles are detailed in Artificial Intelligence Australia’s Ethics Framework A Discussion Paper. This comprehensive paper includes detailed summaries of many of the frameworks and initiatives listed above, together with some very useful case studies.

References
[1] Asimov, Isaac (1950): Runaround, in I, Robot, (The Isaac Asimov Collection ed.) Doubleday. ISBN 0-385-42304-7.
[2] Murphy, Robin; Woods, David D. (2009): Beyond Asimov: The Three Laws of Responsible Robotics. IEEE Intelligent systems. 24 (4): 14–20.
[3] Margaret Boden et al (2017): Principles of robotics: regulating robots in the real world
Connection Science. 29 (2): 124:129.
[4] Tony Prescott and Michael Szollosy (eds.) (2017): Ethical Principles of Robotics, Connection Science. 29 (2) and 29 (3).

The pedestrian experiment

Alan Winfield — Mon, 10 Sep 2018 11:03:00 +0000

Followers of this blog will know that I have been working for some years on simulation-based internal models – demonstrating their potential for ethical robots, safer robots and imitating robots. But pretty much all of our experiments so far have involved only one robot with a simulation-based internal model while the other robots it interacts with have no internal model at all.

But some time ago we wondered what would happen if two robots, each with a simulation-based internal model, interacted with each other. Imagine two such robots approaching each other in the same way that two pedestrians approach each other on the sidewalk. Is it possible that these ‘pedestrian’ robots might, from time to time, engage in the kind of ‘dance’ that human pedestrians do when one steps to their left and the other to their right only to compound the problem of avoiding a collision with a stranger? The answer, it turns out, is yes!

The idea was taken up by Mathias Schmerling at the Humboldt University of Berlin, adapting the code developed by Christian Blum for the Corridor experiment. Chen Yang, one of my masters students, has now updated Mathias’ code and has produced some very nice new results.

Most of the time the pedestrian robots pass each other without fuss but in something between 1 in 5 and 1 in 10 trials we do indeed see an interesting dance. Here are a couple of examples of the majority of trials, when the robots pass each other normally, showing the robots’ trajectories. In each trial blue starts from the left and green from the right. Note that there is an element of randomness in the initial directions of each robot (which almost certainly explains the relative occurrence of normal and dance behaviours).

And here is a gif animation showing what’s going on in a normal trial. The faint straight lines from each robot show the target directions for each next possible action modelled in each robot’s simulation-based internal model (consequence engine); the various dotted lines show the predicted paths (and possible collisions) and the solid blue and green lines show which next action is actually selected following the internal modelling.

Here is a beautiful example of a ‘dance’, again showing the robot trajectories. Note that the impasse resolves itself after awhile. We’re still trying to figure out exactly what mechanism enables this resolution.

And here is the gif animation of the same trial:

Notice that the impasse is not resolved until the fifth turns of each robot.

Is this the first time that pedestrians passing each other – and in particular the occasional dance that ensues – has been computationally modelled?

All of the results above were obtained in simulation (yes there really are simulations within a simulation going on here), but within the past week Chen Yang has got this experiment working with real e-puck robots. Videos will follow shortly.

Acknowledgements.

I am indebted to the brilliant experimental work of first Christian Blum (supported by Wenguo Liu), then Mathias Schmerling who adapted Christian’s code for this experiment, and now Chen Yang who has developed the code further and obtained these results.

Autonomous robot evolution: from cradle to grave

Alan Winfield — Tue, 31 Jul 2018 20:21:03 +0000

A few weeks ago we had the kick-off meeting, in York, of our new 4 year EPSRC funded project Autonomous Robot Evolution (ARE): cradle to grave. We – Andy Tyrrell and Jon Timmis (York), Emma Hart (Edinburgh Napier), Gusti Eiben (Free University of Amsterdam) and myself – are all super excited. We’ve been trying to win support for this project for five years or so, and only now succeeded. This is a project that we’ve been thinking, and writing about, for a long time – so to have the opportunity to try out our ideas for real is wonderful.

In ARE we aim to investigate the artificial evolution of robots for unknown or extreme environments. In a radical new approach we will co-evolve robot bodies and brains in real-time and real-space. Using techniques from 3D printing new robot designs will literally be printed, before being trained in a nursery, then fitness tested in a target environment (a mock nuclear plant). The genomes of the fittest robots will then be combined to create the next generation of ‘child’ robots, so that – over successive generations – we will breed new robot designs in a process that mirrors the way farmers have artificially selected new varieties of plants and animals for thousands of years. Because evolving real robots is slow and resource hungry we will run a parallel process of simulated evolution in a virtual environment, in which the real world environment is used to calibrate the virtual world, and reduce the reality gap*. A hybrid real-virtual process under the control of an ecosystem manager will allow real and virtual robots to mate, and the child robots to be printed and tested in either the virtual or real environments.

The project will be divided into five work packages, each led by a different partner: WP1 Evolution (York), WP2 Physical Environment (UWE), WP3 Virtual Environment (York), WP4 Ecosystem Manager (Napier) and WP5 Integration and Demonstration (UWE).

Here in the Bristol Robotics Lab we will focus on work packages 2 and 5. The goal of WP 2 is the development of a purpose designed 3D printing system – which we call a birth clinic – capable of printing small mobile robots, according to a specification determined by a genome designed in WP1. The birth clinic will need to pick and place a number of pre designed and fabricated electronics, sensing and actuation modules (the robot’s ‘organs’) into the printing work area which will be over printed with hot plastic to form the complete robot. The goal of WP5 will be to integrate all components, including the real world birth clinic, nursery, and mock nuclear environment with the virtual environment (WP3) and the ecosystem manager (WP4) into a working demonstrator and undertake evaluation and analysis.

You can see an impression of what the birth clinic might look like above.

One of the most interesting aspects of the project is that we have no idea what the robots we breed will look like. The evolutionary process could come up with almost any body shape and structure (morphology). The same process will also determine which and how many organs (sensors, actuators, etc) are selected, and their positions and orientation within the body. Our evolved robot bodies could be very surprising indeed.

And who knows – maybe we can take a step towards Walterian Creatures?

*Anyone who uses simulation as a tool to develop robots is well aware that robots which appear to work perfectly well in a simulated virtual world often don’t work very well at all when the same design is tested in the real robot. This problem is especially acute when we are artificially evolving those robots. The reason for these problems is that the model of the real world and the robot(s) in it inside our simulation is an approximation. The Reality Gap refers to the less-than-perfect fidelity of the simulation; a better (higher fidelity) simulator would reduce the reality gap.

Related materials:
Article in de Volkskrant (in Dutch) De robotevolutie kan beginnen. Hoe? Moeder Natuur vervangen door virtuele kraamkamer (The robot evolution can begin. How? Replacing Mother Nature with virtual nursery), May 2018.
Eiben and Smith (2015) From evolutionary computing to the evolution of things, Nature.
Winfield and Timmis (2015) Evolvable Robot Hardware, in Evolvable Hardware, Springer.
Eiben et al. (2013) The Triangle of Life, European Conference on Artificial Life (ECAL 2013).

Simulation-based internal models for safer robots

Alan Winfield — Mon, 16 Jul 2018 21:05:00 +0000

Readers of this blog will know that I’ve become very excited by the potential of robots with simulation-based internal models in recent years. So far we’ve demonstrated their potential in simple ethical robots and as the basis for rational imitation. Our most recent publication instead examines the potential of robots with simulation-based internal models for safety. Of course it’s not hard to see why the ability to model and predict the consequences of both your own and others’ actions can help you to navigate the world more safely than without that ability.

Our paper Simulation-Based Internal Models for Safer Robots demonstrates the value of anticipation in what we call the corridor experiment. Here a smart robot (equipped with a simulation based internal model which we call a consequence engine) must navigate to the end of a corridor while maintaining a safe space around it at all times despite five other robots moving randomly in the corridor – in much the same way you and I might have to navigate down a busy office corridor while others are coming in the opposite direction.

Here is the abstract from our paper:

In this paper, we explore the potential of mobile robots with simulation-based internal models for safety in highly dynamic environments. We propose a robot with a simulation of itself, other dynamic actors and its environment, inside itself. Operating in real time, this simulation-based internal model is able to look ahead and predict the consequences of both the robot’s own actions and those of the other dynamic actors in its vicinity. Hence, the robot continuously modifies its own actions in order to actively maintain its own safety while also achieving its goal. Inspired by the problem of how mobile robots could move quickly and safely through crowds of moving humans, we present experimental results which compare the performance of our internal simulation-based controller with a purely reactive approach as a proof-of-concept study for the practical use of simulation-based internal models.

So, does it work? Thanks to some brilliant experimental work by Christian Blum the answer is a resounding yes. The best way to understand what’s going on is with this wonderful gif animation of one experimental run below. The smart robot (blue) starts at the left and has the goal of safely reaching the right hand end of the corridor – its actual path is also shown in blue. Meanwhile 5 (red) robots are moving randomly (including bouncing off walls) and their actual paths are also shown in red; these robots are equipped only with simple obstacle avoidance behaviours. The larger blue circle shows blue’s ‘attention radius’ – to reduce computational effort blue will only model red robots within this radius. The yellow paths in front of the red robots in blue’s attention radius show blue’s predictions of how those robots will move (taking into account collisions with the corridor walls and with blue and each other). The light blue projection in front of blue shows which of the 34 next possible actions of blue that is internally modelled is actually chosen as the next action (which, as you will see, sometimes includes standing still).

What do the results show us? Christian ran lots of trials – 88 simulations and 54 real robot experiments – over four experiments: (1) the baseline in simulation – in which the blue robot has only a simple reactive collision avoidance behaviour, (2) the baseline with real robots, (3) using the consequence engine (CE) in the blue robot in simulation, and (4) using the consequence engine in the blue robot with real robots. In the results below (a) shows the time taken for the blue robot to reach the end of the corridor, (b) shows the distance that the blue robot covers while reaching the end of the corridor, (c) shows the “danger ratio” experienced by the blue robot, and (d) shows the number of consequence engine runs per timestep in the blue robot. The danger ratio is the percentage of the run time that anther robot is within the blue robot’s safety radius.

For a relatively small cost in additional run time and distance covered, panels (a) and (b), the danger ratio is very significantly reduced from a mean value of ~20% to a mean value of zero, panel (c). Of course there is a computational cost, and this is reflected in panel (d); the baseline experiment has no consequence engine and hence runs no simulations, whereas the smart robot runs an average of between 8 and 10 simulations per time-step. This is exactly what we would expect: predicting the future clearly incurs a computational overhead.

Full paper reference:
Blum C, Winfield AFT and Hafner VV (2018) Simulation-Based Internal Models for Safer Robots. Front. Robot. AI 4:74. doi: 10.3389/frobt.2017.00074

Acknowledgements:
I am indebted to Christian Blum who programmed the robots, set up the experiment and obtained the results outlined here. Christian lead authored the paper, which was also co-authored by my friend and research collaborator Verena Hafner, who was Christian’s PhD advisor.

What is artificial intelligence? (Or, can machines think?)

Alan Winfield — Thu, 21 Jun 2018 21:54:00 +0000

Here are the slides from my York Festival of Ideas keynote yesterday, which introduced the festival focus day Artificial Intelligence: Promises and Perils.

I start the keynote with Alan Turing’s famous question: Can a Machine Think? and explain that thinking is not just the conscious reflection of Rodin’s Thinker but also the largely unconscious thinking required to make a pot of tea. I note that at the dawn of AI 60 years ago we believed the former kind of thinking would be really difficult to emulate artificially and the latter easy. In fact it has turned out to be the other way round: we’ve had computers that can expertly play chess for over 20 years, but we can’t yet build a robot that could go into your kitchen and make you a cup of tea (see also the Wozniak coffee test).

In slides 5 and 6 I suggest that we all assume a cat is smarter than a crocodile, which is smarter than a cockroach, on a linear scale of intelligence from not very intelligent to human intelligence. I ask where would a robot vacuum cleaner be on this scale and propose that such a robot is about as smart as an e-coli (single celled organism). I then illustrate the difficulty of placing the Actroid robot on this scale because, although it may look convincingly human (from a distance), in reality the robot is not very much smarter than a washing machine (and I hint that this is an ethical problem).

In slide 7 I show how apparently intelligent behaviour doesn’t require a brain, with the Solarbot. This robot is an example of a Braitenberg machine. It has two solar panels (which look a bit like wings) acting as both sensors and power sources; the left hand panel is connected to the right hand wheel and vice versa. These direct connections mean that Solarbot can move towards the light and even navigate its way through obstacles, thus showing that intelligent behaviour is an emergent property of the interactions between body and environment.

In slide 8 I ask the question: What is the most advanced AI in the world today? (A question I am often asked.) Is it for example David Hanson’s robot Sophia (which some press reports have claimed as the world’s most advanced)? I argue it is not, since it is a chatbot AI – with a limited conversational repertoire – with a physical body (imagine Alexa with a humanoid head). Is it the DeepMind AI AlphaGo which famously beat the world’s best Go player in 2016? Although very impressive I again argue no since AlphaGo cannot do anything other than play Go. Instead I suggest that everyday Google might well be the world’s most advanced AI (on this I agree with my friend Joanna Bryson). Google is in effect a librarian able to find a book from an immense library for you – on the basis of your ill formed query – more or less instantly! (And this librarian is poly lingual too.)

In slides 9 I make the point that intelligence is not one thing that animals, robots and AIs have more or less of (in other words the linear scale shown on slides 5 and 6 is wrong). Then in slides 10 – 13 I propose four distinct categories of intelligence: morphological, swarm, individual and social intelligence. I suggest in slides 14 – 16 that if we express these as four axes of a graph then we can (very approximately) compare the intelligence of different organisms, including humans. In slide 17 I show some robots and argue that this graph shows why robots are so unintelligent; it is because robots generally only have two of the four kinds of intelligence whereas animals typically have three or sometimes all four. A detailed account of these ideas can be found in my paper How intelligent is your intelligent robot?

In the next segment, slides 18-20 I ask: how do we make Artificial General Intelligence (AGI)? I suggest that the key difference between current narrow AI and AGI is the ability – which comes very naturally to humans – to generalise knowledge learned in one context to a completely different context. This I think is the basis of human creativity. Using Data from Star Trek the next generation as a SF example of an AGI with human-equivalent intelligence as what we might be aiming for in the quest for AGI I explain that there are 3 approaches to getting there: by design, using artificial evolution or by reverse engineering animals. I offer the opinion that the gap between where we are now and Data like AGI is about the same as the gap between current space craft engine technology and warp drive technology. In other words not any time soon.

In the fourth segment of the talk (slides 21-24) I give a very brief account of evolutionary robotics – a method for breeding robots in much the same way farmers have artificially selected new varieties of plants and animals for thousands of years. I illustrate this with the wonderful Golem project which, for the first time, evolved simple creatures then 3D printed the most successful ones. I then introduce our new four year EPSRC funded project Autonomous Robot Evolution: from cradle to grave. In a radical new approach we aim to co-evolve robot bodies and brains in real-time and real-space. Using techniques from 3D printing new robot designs will literally be printed, before being trained in a nursery, then fitness tested in a target environment. With this approach we hope to be able to evolve robots for extreme environments, however because the energy costs are so high I do not think evolution is a route to truly thinking machines.

In the final segment (slides 25-35) I return to the approach of trying to design rather than evolve thinking machines. I introduce the idea of embedding a simulation of a robot in that robot, so that it has the ability to internally model itself. The first example I give is the amazing anthropomimetic robot invented by my old friend Owen Holland, called ECCEROBOT. Eccerobot is able to learn how to control it’s own very complicated and hard-to-control body by trying out possible movement sequences in its internal model (Owen calls this a ‘functional imagination’). I then outline our own work to use the same principle – a simulation based internal model – to demonstrate simple ethical behaviours, first with e-puck robots, then with NAO robots. These experiments are described in detail here and here. I suggest that these robots – with their ability to model and predict the consequences of their own and others’ actions, in other words anticipate the future – may represent the first small steps toward thinking machines.

Related blog posts:
60 years of asking can robot think?

How intelligent are intelligent robots?

Robot bodies and how to evolve them

Why ethical robots might not be such a good idea after all

Alan Winfield — Thu, 08 Feb 2018 22:30:00 +0000

Last week my colleague Dieter Vanderelst presented our paper: The Dark Side of Ethical Robots at AIES 2018 in New Orleans.

I blogged about Dieter’s very elegant experiment here, but let me summarise. With two NAO robots he set up a demonstration of an ethical robot helping another robot acting as a proxy human, then showed that with a very simple alteration of the ethical robot’s logic it is transformed into a distinctly unethical robot – behaving either competitively or aggressively toward the proxy human.

Here are our paper’s key conclusions:

The ease of transformation from ethical to unethical robot is hardly surprising. It is a straightforward consequence of the fact that both ethical and unethical behaviours require the same cognitive machinery with – in our implementation – only a subtle difference in the way a single value is calculated. In fact, the difference between an ethical (i.e. seeking the most desirable outcomes for the human) robot and an aggressive (i.e. seeking the least desirable outcomes for the human) robot is a simple negation of this value.

On the face of it, given that we can (at least in principle) build explicitly ethical machines then it would seem that we have a moral imperative to do so; it would appear to be unethical not to build ethical machines when we have that option. But the findings of our paper call this assumption into serious doubt. Let us examine the risks associated with ethical robots and if, and how, they might be mitigated. There are three.

First there is the risk that an unscrupulous manufacturermight insert some unethical behaviours into their robots in order to exploit naive or vulnerable users for financial gain, or perhaps to gain some market advantage (here the VW diesel emissions scandal of 2015 comes to mind). There are no technical steps that would mitigate this risk, but the reputational damage from being found out is undoubtedly a significant disincentive. Compliance with ethical standards such as BS 8611 guide to the ethical design and application of robots and robotic systems, or emerging new IEEE P700X ‘human’ standards would also support manufacturers in the ethical application of ethical robots.
Perhaps more serious is the risk arising from robots that have user adjustable ethics settings. Here the danger arises from the possibility that either the user or a technical support engineer mistakenly, or deliberately, chooses settings that move the robot’s behaviours outside an ‘ethical envelope’. Much depends of course on how the robot’s ethics are coded, but one can imagine the robot’s ethical rules expressed in a user-accessible format, for example, an XML like script. No doubt the best way to guard against this risk is for robots to have no user adjustable ethics settings, so that the robot’s ethics are hard-coded and not accessible to either users or support engineers.
But even hard-coded ethics would not guard against undoubtedly the most serious risk of all, which arises when those ethical rules are vulnerable to malicious hacking. Given that cases of white-hat hacking of cars have already been reported, it’s not difficult to envisage a nightmare scenario in which the ethics settings for an entire fleet of driverless cars are hacked, transforming those vehicles into lethal weapons. Of course, driverless cars (or robots in general) without explicit ethics are also vulnerable to hacking, but weaponising such robots is far more challenging for the attacker. Explicitly ethical robots focus the robot’s behaviours to a small number of rules which make them, we think, uniquely vulnerable to cyber-attack.

Ok, taking the most serious of these risks: hacking, we can envisage several technical approaches to mitigating the risk of malicious hacking of a robot’s ethical rules. One would be to place those ethical rules behind strong encryption. Another would require a robot to authenticate its ethical rules by first connecting to a secure server. An authentication failure would disable those ethics, so that the robot defaults to operating without explicit ethical behaviours. Although feasible, these approaches would be unlikely to deter the most determined hackers, especially those who are prepared to resort to stealing encryption or authentication keys.

It is very clear that guaranteeing the security of ethical robots is beyond the scope of engineering and will need regulatory and legislative efforts. Considering the ethical, legal and societal implications of robots, it becomes obvious that robots themselves are not where responsibility lies. Robots are simply smart machines of various kinds and the responsibility to ensure they behave well must always lie with human beings. In other words, we require ethical governance, and this is equally true for robots with or without explicit ethical behaviours.

Two years ago I thought the benefits of ethical robots outweighed the risks. Now I’m not so sure. I now believe that – even with strong ethical governance – the risks that a robot’s ethics might be compromised by unscrupulous actors are so great as to raise very serious doubts over the wisdom of embedding ethical decision making in real-world safety critical robots, such as driverless cars. Ethical robots might not be such a good idea after all.

Here is the full reference to our paper:

Vanderelst D and Winfield AFT (2018), The Dark Side of Ethical Robots, AAAI/ACM Conf. on AI Ethics and Society (AIES 2018), New Orleans.

A round up of robotics and AI ethics: part 1 principles

Alan Winfield — Tue, 09 Jan 2018 23:54:58 +0000

This blogpost is a round up of the various sets of ethical principles of robotics and AI that have been proposed to date, ordered by date of first publication. The principles are presented here (in full or abridged) with notes and references but without commentary. If there are any (prominent) ones I’ve missed please let me know.

Asimov’s three laws of Robotics (1950)

A robot may not injure a human being or, through inaction, allow a human being to come to harm.
A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

A human may not deploy a robot without the human-robot work system meeting the highest legal and professional standards of safety and ethics.
A robot must respond to humans as appropriate for their roles.
A robot must be endowed with sufficient situated autonomy to protect its own existence as long as such protection provides smooth transfer of control which does not conflict with the First and Second Laws.

These were proposed in Robin Murphy and David Wood’s paper Beyond Asimov: The Three Laws of Responsible Robotics [2].

EPSRC Principles of Robotics (2010)

Robots are multi-use tools. Robots should not be designed solely or primarily to kill or harm humans, except in the interests of national security.
Humans, not Robots, are responsible agents. Robots should be designed and operated as far as practicable to comply with existing laws, fundamental rights and freedoms, including privacy.
Robots are products. They should be designed using processes which assure their safety and security.
Robots are manufactured artefacts. They should not be designed in a deceptive way to exploit vulnerable users; instead their machine nature should be transparent.
The person with legal responsibility for a robot should be attributed.

Future of Life Institute Asilomar principles for beneficial AI (Jan 2017)

I will not list all 23 principles but extract just a few to compare and contrast with the others listed here:

6. Safety: AI systems should be safe and secure throughout their operational lifetime, and verifiably so where applicable and feasible.

7. Failure Transparency: If an AI system causes harm, it should be possible to ascertain why.

8. Judicial Transparency: Any involvement by an autonomous system in judicial decision-making should provide a satisfactory explanation auditable by a competent human authority.

9. Responsibility: Designers and builders of advanced AI systems are stakeholders in the moral implications of their use, misuse, and actions, with a responsibility and opportunity to shape those implications.

10. Value Alignment: Highly autonomous AI systems should be designed so that their goals and behaviors can be assured to align with human values throughout their operation.

11. Human Values: AI systems should be designed and operated so as to be compatible with ideals of human dignity, rights, freedoms, and cultural diversity.

12. Personal Privacy: People should have the right to access, manage and control the data they generate, given AI systems’ power to analyze and utilize that data.

13. Liberty and Privacy: The application of AI to personal data must not unreasonably curtail people’s real or perceived liberty.

14. Shared Benefit: AI technologies should benefit and empower as many people as possible.

15. Shared Prosperity: The economic prosperity created by AI should be shared broadly, to benefit all of humanity.

An account of the development of the Asilomar principles can be found here.

The ACM US Public Policy Council Principles for Algorithmic Transparency and Accountability (Jan 2017)

Awareness: Owners, designers, builders, users, and other stakeholders of analytic systems should be aware of the possible biases involved in their design, implementation, and use and the potential harm that biases can cause to individuals and society.
Access and redress: Regulators should encourage the adoption of mechanisms that enable questioning and redress for individuals and groups that are adversely affected by algorithmically informed decisions.
Accountability: Institutions should be held responsible for decisions made by the algorithms that they use, even if it is not feasible to explain in detail how the algorithms produce their results.
Explanation: Systems and institutions that use algorithmic decision-making are encouraged to produce explanations regarding both the procedures followed by the algorithm and the specific decisions that are made. This is particularly important in public policy contexts.
Data Provenance: A description of the way in which the training data was collected should be maintained by the builders of the algorithms, accompanied by an exploration of the potential biases induced by the human or algorithmic data-gathering process.
Auditability: Models, algorithms, data, and decisions should be recorded so that they can be audited in cases where harm is suspected.
Validation and Testing: Institutions should use rigorous methods to validate their models and document those methods and results.

Contribution to humanity Members of the JSAI will contribute to the peace, safety, welfare, and public interest of humanity.
Abidance of laws and regulations Members of the JSAI must respect laws and regulations relating to research and development, intellectual property, as well as any other relevant contractual agreements. Members of the JSAI must not use AI with the intention of harming others, be it directly or indirectly.
Respect for the privacy of others Members of the JSAI will respect the privacy of others with regards to their research and development of AI. Members of the JSAI have the duty to treat personal information appropriately and in accordance with relevant laws and regulations.
Fairness Members of the JSAI will always be fair. Members of the JSAI will acknowledge that the use of AI may bring about additional inequality and discrimination in society which did not exist before, and will not be biased when developing AI.
Security As specialists, members of the JSAI shall recognize the need for AI to be safe and acknowledge their responsibility in keeping AI under control.
Act with integrity Members of the JSAI are to acknowledge the significant impact which AI can have on society.
Accountability and Social Responsibility Members of the JSAI must verify the performance and resulting impact of AI technologies they have researched and developed.
Communication with society and self-development Members of the JSAI must aim to improve and enhance society’s understanding of AI.
Abidance of ethics guidelines by AI AI must abide by the policies described above in the same manner as the members of the JSAI in order to become a member or a quasi-member of society.

An explanation of the background and aims of these ethical guidelines can be found here, together with a link to the full principles (which are shown abridged above).

Draft principles of The Future Society’s Science, Law and Society Initiative (Oct 2017)

AI should advance the well-being of humanity, its societies, and its natural environment.
AI should be transparent.
Manufacturers and operators of AI should be accountable.
AI’s effectiveness should be measurable in the real-world applications for which it is intended.
Operators of AI systems should have appropriate competencies.
The norms of delegation of decisions to AI systems should be codified through thoughtful, inclusive dialogue with civil society.

This article by Nicolas Economou explains the 6 principles with a full commentary on each one.

Montréal Declaration for Responsible AI draft principles (Nov 2017)

Well-being The development of AI should ultimately promote the well-being of all sentient creatures.
Autonomy The development of AI should promote the autonomy of all human beings and control, in a responsible way, the autonomy of computer systems.
Justice The development of AI should promote justice and seek to eliminate all types of discrimination, notably those linked to gender, age, mental / physical abilities, sexual orientation, ethnic/social origins and religious beliefs.
Privacy The development of AI should offer guarantees respecting personal privacy and allowing people who use it to access their personal data as well as the kinds of information that any algorithm might use.
Knowledge The development of AI should promote critical thinking and protect us from propaganda and manipulation.
Democracy The development of AI should promote informed participation in public life, cooperation and democratic debate.
Responsibility The various players in the development of AI should assume their responsibility by working against the risks arising from their technological innovations.

The Montréal Declaration for Responsible AI proposes the 7 values and draft principles above (here in full with preamble, questions and definitions).

IEEE General Principles of Ethical Autonomous and Intelligent Systems (Dec 2017)

How can we ensure that A/IS do not infringe human rights?
Traditional metrics of prosperity do not take into account the full effect of A/IS technologies on human well-being.
How can we assure that designers, manufacturers, owners and operators of A/IS are responsible and accountable?
How can we ensure that A/IS are transparent?
How can we extend the benefits and minimize the risks of AI/AS technology being misused?

Demand That AI Systems Are Transparent
Equip AI Systems With an “Ethical Black Box”
Make AI Serve People and Planet
Adopt a Human-In-Command Approach
Ensure a Genderless, Unbiased AI
Share the Benefits of AI Systems
Secure a Just Transition and Ensuring Support for Fundamental Freedoms and Rights
Establish Global Governance Mechanisms
Ban the Attribution of Responsibility to Robots
Ban AI Arms Race

Sci-Fi Dreams: How visions of the future are shaping the development of intelligent technology

Alan Winfield — Fri, 14 Jul 2017 13:21:00 +0000

Here are the slides I gave recently as member of panel Sci-Fi Dreams: How visions of the future are shaping the development of intelligent technology, at the Centre for the Future of Intelligence 2017 conference. I presented three short stories about robot stories.

Slide 2:

The FP7 TRUCE Project invited a number of scientists – mostly within the field of Artificial Life – to suggest ideas for short stories. Those stories were then sent to a panel of writers, who chose one of the stories. I submitted an idea called The feeling of what it is like to be a robot and was delighted when Lucy Caldwell contacted me. Following a visit to the lab Lucy drafted a beautiful story called The Familiar which – following some iteration – appeared in the collected volume Beta Life.

Slide 3:

More recently the EU Human Brain Project Foresight Lab brought three Sci Fi writers – Allen Ashley, Jule Owen and Stephen Oram – to visit the lab. Inspired by what they saw they then wrote three wonderful short stories, which were read at the 2016 Bristol Literature Festival. The readings were followed by a panel discussion which included myself and BRL colleagues Antonia Tzemanaki and Marta Palau Franco. The three stories are published in the volume Versions of the Future. Stephen Oram went on to publish a collection called Eating Robots.

Slide 4:

My first two stories were about people telling stories about robots. Now I turn to the possibility of robots themselves telling stories. Some years ago I speculated on the idea on the idea of robots telling each other stories (directly inspired by a conversation with Richard Gregory). That idea has now turned into a current project, with the aim of building an embodied computational model of storytelling. For a full description see this paper, currently in press.

Thoughts on the EU’s draft report on robotics

Alan Winfield — Mon, 08 May 2017 09:00:00 +0000

NAO robot. Photo courtesy: Paul Bremner/UWE

I was asked to write a short op-ed on the European Parliament Law Committee’s recommendations on civil law rules for robotics. In the end, the piece didn’t get published, so I am posting it here:

It is a great shame that most reports of the European Parliament’s Committee for Legal Affairs’ vote on its Draft Report on Civil Law Rules on Robotics headlined on ‘personhood’ for robots because the report has much else to commend it. Most important among its several recommendations is a proposed code of ethical conduct for roboticists, which explicitly asks designers to research and innovate responsibly. Some may wonder why such an invitation even needs to be made but, given that engineering and computer science education rarely includes classes on ethics (it should), it is really important that robotics engineers reflect on their ethical responsibilities to society – especially given how disruptive robot technologies are. This is not new – great frameworks for responsible research and innovation already exist. One such is the 2014 Rome Declaration on RRI, and in 2015 the Foundation for Responsible Robotics was launched.

Within the report’s draft Code of Conduct is a call for robotics funding proposals to include a risk assessment. This too is a very good idea and guidance already exists in British Standard BS 8611, published in April 2016. BS 8611 sets out a comprehensive set of ethical risks and offers guidance on how to mitigate them. It is very good also to see that the Code stresses that humans, not robots, are the responsible agents; this is something we regarded as fundamental when we drafted the Principles of Robotics in 2010.

For me, transparency (or the lack of it) is an increasing worry in both robots and AI systems. Labour’s industry spokesperson Chi Onwurah is right to say, “Algorithms are part of our world, so they are subject to regulation, but because they are not transparent, it’s difficult to regulate them effectively” (and don’t forget that it is algorithms that make intelligent robots intelligent). So it is very good to see the draft Code call for robotics engineers to “guarantee transparency … and right of access to information by all stakeholders”, and then in the draft ‘Licence for Designers’: you should ensure “maximal transparency” and even more welcome “you should develop tracing tools that … facilitate accounting and explanation of robotic behaviour… for experts, operators and users”. Within the IEEE Standards Association Global Initiative on Ethics in AI and Autonomous Systems, launched in 2016, we are working on a new standard on Transparency in Autonomous Systems.

This brings me to standards and regulation. I am absolutely convinced that regulation, together with transparency and public engagement, builds public trust. Why is it that we trust our tech? Not just because it’s cool and convenient, but also because it’s safe (and we assume that the disgracefully maligned experts will take care of assuring that safety). One of the reasons we trust airliners is that we know they are part of a highly regulated industry with an amazing safety record. The reason commercial aircraft are so safe is not just good design, it is also the tough safety certification processes and, when things do go wrong, robust processes of air accident investigation. So the Report’s call for a European Agency for Robotics and AI to recommend standards and regulatory framework is, as far as I’m concerned, not a moment too soon. We urgently need standards for safety certification of a wide range of robots, from drones and driverless cars to robots for care and assisted living.

Like many of my robotics colleagues, I am deeply worried by the potential for robotics and AI to increase levels of economic inequality in the world. Winnie Byanyima, executive director of Oxfam writes for the WEF, “We need fundamental change to our economic model. Governments must stop hiding behind ideas of market forces and technological change. They … need to steer the direction of technological development”. I think she is right – we need a serious public conversation about technological unemployment and how we ensure that the wealth created by AI and Automonous Systems is shared by all. A Universal Basic Income may or may not be the best way to do this – but it is very encouraging to see this question raised in the draft Report.

I cannot close the piece without at least mentioning artificial personhood. My own view is that personhood is the solution to a problem that doesn’t exist. I can understand why, in the context of liability, the Report raises this question for discussion, but – as the report itself later asserts in the Code of Conduct: humans, not robots, are the responsible agents. Robots are and should remain, artefacts.

If you enjoyed this article, you may also want to read:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Does AI pose a threat to society?

Alan Winfield — Wed, 03 May 2017 08:15:00 +0000

Last week I had the pleasure of debating the question “does AI pose a threat to society?” with friends and colleagues Christian List, Maja Pantic and Samantha Payne. The event was organised by the British Academy and brilliantly chaired by the Royal Society’s director of science policy Claire Craig. Here follows my opening statement:

My opening statement

One Friday afternoon in 2009 I was called by a science journalist at, I recall, the Sunday Times. He asked me if I knew that there was to be a meeting of the AAAI to discuss robot ethics. I said no I don’t know of this meeting. He then asked “are you surprised they are meeting to discuss robot ethics” and my answer was no. We talked some more and agreed it was actually a rather dull story: a case of scientists behaving responsibly. I really didn’t expect the story to appear but checked the Sunday paper anyway, and there in the science section was the headline Scientists fear revolt of killer robots. (I then spent the next couple of days on the radio explaining that no, scientists do not fear a revolt of killer robots.)

So, fears of future super intelligence – robots taking over the world – are greatly exaggerated: the threat of an out-of-control super intelligence is a fantasy – interesting for a pub conversation perhaps. It’s true we should be careful and innovate responsibly, but that’s equally true for any new area of science and technology. The benefits of robotics and AI are so significant, the potential so great, that we should be optimistic rather than fearful. Of course robots and intelligent systems must be engineered to very high standards of safety for exactly the same reasons that we need our washing machines, cars and airplanes to be safe. If robots are not safe people will not trust them. To reach it’s full potential what robotics and AI needs is a dose of good old fashioned (and rather dull) safety engineering.

In 2011 I was invited to join a British Standards Institute working group on robot ethics, which drafted a new standard BS 8611 Guide to the ethical design of robots and robotic systems, published in April 2016. I believe this to be the world’s first standard on ethical robots.

Also in 2016 the very well regarded IEEE standards association – the same organization that gave us WiFi – launched a Global initiative on Ethical Considerations in AI and Autonomous Systems. The purpose of this Initiative is to ensure every technologist is educated and empowered to prioritize ethical considerations in the design and development of autonomous and intelligent systems; in a nutshell, to ensure ethics are baked in. In December we published Ethically Aligned Design: A Vision for Prioritizing Human Well Being with AI and Autonomous Systems. Within that initiative I’m also leading a new standard on transparency in autonomous systems, based on the simple principle that it should always be possible to find out why an AI or robot made a particular decision.

We need to agree ethical principles, because they are needed to underpin standards – ways of assessing and mitigating the ethical risks of robotics and AI. But standards needs teeth and in turn underpin regulation. Why do we need regulation? Think of passenger airplanes; the reason we trust them is because it’s a highly regulated industry with an amazing safety record, and robust, transparent processes of air accident investigation when things do go wrong. Take one example of a robot that we read a lot about in the news – the Driverless Car. I think there’s a strong case for a driverless car equivalent of the CAA, with a driverless car accident investigation branch. Without this it’s hard to see how driverless car technology will win public trust.

Does AI pose a threat to society? No. But we do need to worry about the down to earth questions of present day rather unintelligent AIs; the ones that are deciding our loan applications, piloting our driverless cars or controlling our central heating. Are those AIs respecting our rights, freedoms and privacy? Are they safe? When AIs make bad decisions, can we find out why? And I worry too about the wider societal and economic impacts of AI. I worry about jobs of course, but actually I think there is a bigger question: how can we ensure that the wealth created by robotics and AI is shared by all in society?

Thank you.

The featured image of this article was used to advertise the BA’s series of events on the theme Robotics, AI and Society. The reason I draw your attention to it here is that one of the many interesting questions to the panel was about the way that AI tends to be visualised in the media. This kind of human face coalescing (or perhaps emerging) from the atomic parts of the AI seems to have become a trope for AI. Is it a helpful visualisation of the human face of AI, or does it mislead to an impression that AI has human characteristics?

The infrastructure of life part 2: Transparency

Alan Winfield — Fri, 03 Feb 2017 14:00:00 +0000

Part 2: Autonomous Systems and Transparency

In my previous post I argued that a wide range of AI and Autonomous Systems (from now on I will just use the term AS as shorthand for both) should be regarded as Safety Critical. I include both autonomous software AI systems and hard (embodied) AIs such as robots, drones and driverless cars. Many will be surprised that I include in the soft AI category apparently harmless systems such as search engines. Of course no-one is seriously inconvenienced when Amazon makes a silly book recommendation, but consider very large groups of people. If a truth (such as global warming) is – because of accidental or willful manipulation – presented as false, and that falsehood is believed by a very large number of people, then serious harm to the planet (and we humans who depend on it) could result.

I argued that the tools barely exist to properly assure the safety of AS, let alone the standards and regulation needed to build public trust, and that political pressure is needed to ensure our policymakers fully understand the public safety risks of unregulated AS.

In this post I will outline the case that transparency is a foundational requirement for building public trust in AS, based on the radical proposition that it should always be possible to find out why an AS made a particular decision.

Transparency is not one thing. Clearly your elderly relative doesn’t require the same level of understanding of her care robot as the engineer who repairs it. Nor would you expect the same appreciation of the reasons a medical diagnosis AI recommends a particular course of treatment as your doctor. Broadly (and please understand this is a work in progress) I believe there are five distinct groups of stakeholders, and that AS must be transparent to each, in different ways and for different reasons. These stakeholders are: (1) users, (2) safety certification agencies, (3) accident investigators, (4) lawyers or expert witnesses and (5) wider society.

For users, transparency is important because it builds trust in the system, by providing a simple way for the user to understand what the system is doing and why.
For safety certification of an AS, transparency is important because it exposes the system’s processes for independent certification against safety standards.
If accidents occur, AS will need to be transparent to an accident investigator; the internal process that led to the accident need to be traceable.
Following an accident lawyers or other expert witnesses, who may be required to give evidence, require transparency to inform their evidence. And
for disruptive technologies, such as driverless cars, a certain level of transparency to wider society is needed in order to build public confidence in the technology.

Of course the way in which transparency is provided is likely to be very different for each group. If we take a care robot as an example transparency means the user can understand what the robot might do in different circumstances; if the robot should do anything unexpected she should be able to ask the robot ‘why did you just do that?’ and receive an intelligible reply. Safety certification agencies will need access to technical details of how the AS works, together with verified test results. Accident investigators will need access to data logs of exactly what happened prior to and during an accident, most likely provided by something akin to an aircraft flight data recorder (and it should be illegal to operate an AS without such a system). And wider society would need accessible documentary-type science communication to explain the AS and how it works.

Source: ROBOTT-NET

In IEEE Standards Association project P7001, we aim to develop a standard that sets out measurable, testable levels of transparency in each of these categories (and perhaps new categories yet to be determined), so that Autonomous Systems can be objectively assessed and levels of compliance determined. It is our aim that P7001 will also articulate levels of transparency in a range that defines minimum levels up to the highest achievable standards of acceptance. The standard will provide designers of AS with a toolkit for self-assessing transparency, and recommendations for how to address shortcomings or transparency hazards.

Of course transparency on its own is not enough. Public trust in technology, as in government, requires both transparency and accountability. Transparency is needed so that we can understand who is responsible for the way Autonomous Systems work and – equally importantly – don’t work.

Thanks: I’m very grateful to colleagues in the IEEE global initiative on ethical considerations in Autonomous Systems for supporting P7001, especially John Havens and Kay Firth-Butterfield. I’m equally grateful to colleagues at the Dagstuhl on Engineering Moral Machines, especially Michael Fisher, Marija Slavkovik and Christian List for discussions on transparency.

More from Alan Winfield:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The infrastructure of life part 1: Safety

Alan Winfield — Thu, 26 Jan 2017 10:30:00 +0000

Part 1: Autonomous Systems and Safety

We all rely on machines. All aspects of modern life, from transport to energy, work to welfare, play to politics depend on a complex infrastructure of physical and virtual systems. How many of us understand how all of this stuff works? Very few I suspect. But it doesn’t matter, does it? We trust the good men and women (the disgracefully maligned experts) who build, manage and maintain the infrastructure of life. If something goes wrong they will know why. And (we hope) make sure it doesn’t happen again.

All well and good you might think. But the infrastructure of life is increasingly autonomous – many decisions are now made not by a human but by the systems themselves. When you search for a restaurant near you the recommendation isn’t made by a human, but by an algorithm. Many financial decisions are not made by people but by algorithms; and I don’t just mean city investments – it’s possible that your loan application will be decided by an AI. Machine legal advice is already available; a trend that is likely to increase. And of course if you take a ride in a driverless car, it is algorithms that decide when the car turns, brakes and so on. I could go on.

Algorithmic trading is now commonplace

These are not trivial decisions. They affect lives. The real world impacts are human and economic, even political (search engine results may well influence how someone votes). In engineering terms these systems are safety critical. Examples of safety critical systems that we all rely on from time to time include aircraft autopilots or train braking systems. But – and this may surprise you – the difficult engineering techniques used to prove the safety of such systems are not applied to search engines, automated trading systems, medical diagnosis AIs, assistive living robots, delivery drones, or (I’ll wager) driverless car autopilots.

Why is this? Well, it’s partly because the field of AI and autonomous systems is moving so fast. But I suspect it has much more to do with an incompatibility between the way we have traditionally designed safety critical systems, and the design of modern AI systems. There is I believe one key problem: learning. There is a very good reason that current safety critical systems (like aircraft autopilots) don’t learn. Current safety assurance approaches assume that the system being certified will never change, but a system that learns does – by definition – change its behaviour, so any certification is rendered invalid after the system has learned.

And as if that were not bad enough, the particular method of learning which has caused such excitement – and rapid progress – in the last few years is based on Artificial Neural Networks (more often these days referred to as Deep Learning). A characteristic of ANNs is that after the ANN has been trained with datasets, any attempt to examine its internal structure in order to understand why and how the ANN makes a particular decision is impossible. The decision making process of an ANN is opaque. Alphago’s moves were beautiful but puzzling. We call this the black box problem.

Safety critical systems such as aircraft autopilots do not learn

Does this mean we cannot assure the safety of learning autonomous/AI systems at all? No it doesn’t. The problem of safety assurance of systems that learn is hard but not intractable, and is the subject of current research*. The black box problem may be intractable for ANNs, but could be avoided by using approaches to AI that do not use ANNs.

But – here’s the rub. This involves slowing down the juggernaut of autonomous systems and AI development. It means taking a much more cautious and incremental approach, and it almost certainly involves regulation (that, for instance, makes it illegal to run a driverless car unless the car’s autopilot has been certified as safe – and that would require standards that don’t yet exist). Yet the commercial and political pressure is to be more permissive, not less; no country wants to be left behind in the race to cash in on these new technologies.

This is why work toward AI/Autonomous Systems standards is so vital, together with the political pressure to ensure our policymakers fully understand the public safety risks of unregulated AI.

In my next blog post I will describe one current standards initiative, towards introducing transparency in AI and Autonomous Systems based on the simple principle that it should always be possible to find out why an AI/AS system made a particular decision.

The next few years of swimming against the tide is going to be hard work. As Luke
Muehlhauser writes in his excellent essay on transparency in safety-critical systems “…there is often a tension between AI capability and AI transparency. Many of AI’s most powerful methods are also among its least transparent”.

*some, but nowhere near enough. See for instance Verifiable Autonomy.

More from Alan Winfield:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

De-automation is a thing

Alan Winfield — Thu, 29 Dec 2016 13:00:00 +0000

We tend to assume that automation is a process that continues – that once some human activity has been automated there’s no going back. That automation sticks. But, as Paul Mason pointed out in a recent column that assumption is wrong.

Mason gives a startling example of the decline of car-wash robots, to be replaced by, as he puts it “five guys with rags”. Here’s the paragraph that really made me think:

“There are now 20,000 hand car washes in Britain, only a thousand of them regulated. By contrast, in the space of 10 years, the number of rollover car-wash machines has halved –from 9,000 to 4,200.”

The reasons, of course, are political and economic and you may or may not agree with Mason’s diagnosis and prescription (as it happens I do). But de-automation – and the ethical, societal and legal implications – is something that we, as roboticists, need to think about just as much as automation.

Several questions come to mind:

are there other examples of de-automation?
is the car-wash robot example atypical, or part of a trend?
is de-automation necessarily a sign of something going wrong? (would Mason be so concerned about the guys with rags if the hand car wash industry were a well-regulated industry paying decent wages to its workers, and generating tax revenues back to the economy?)

This is just a short blog post to – I hope – start a conversation.

Ethically Aligned Design

Alan Winfield — Tue, 27 Dec 2016 09:00:00 +0000

Having been involved in roboethics for some years, I was delighted when the IEEE launched its initiative on Ethical Considerations in AI and Autonomous Systems, early this year. Especially so because of the reach and traction that the IEEE has internationally. (Up until now most ethics initiatives have been national efforts – with the notable exception of the 2006 EURON roboethics roadmap.)

Even better, this is an initiative of the IEEE standards association – the very same that gave the world Wi-Fi (aka IEEE 802.11) 19 years ago. So when I was asked to get involved I jumped at the chance and became co-chair of the General Principles committee. I found myself in good company; many great people I knew but more I did not, and it was a real pleasure when we met face to face in The Hague at the end of August.

Most of our meetings were conducted by phone and it was a very demanding timetable. From nothing to our first publication: Ethically Aligned Design a few days ago is a remarkable achievement, which I think wouldn’t have happened without the extraordinary energy and enthusiasm of the initiative’s executive director John Havens.

I’m not going to describe what’s in that document here; instead I hope you will read it – and return comments. This document is not set in stone, it is – in the best traditions of the RFCs which defined the Internet – a Request for Input

But there are a couple of aspects I will highlight. Like its modest but influential predecessor, the EPSRC/AHRC principles of robotics, the IEEE initiative is hugely multi-disciplinary. It draws heavily from industry and academia, and includes philosophers, ethicists, lawyers, social scientists – as well as engineers and computer scientists – and significantly a number of diplomats and representatives from transnational bodies like the United Nations, US state department and the WEF. This is so important – if the work of this initiative is to make a difference it will need influential advocates. Equally important is that this is not a group dominated by old white men. There are plenty of those for sure, but I reckoned 40% women (should be 50% though!) and plenty of post-docs and PhD students too.

Equally important, the work is open. The publications are released under the creative commons licence. Likewise active membership is open. If you care about the issues and think you could contribute to one or more of the committees – or even if you think there’s a whole area of concern missing that needs to a new committee – get in touch!

The dark side of ethical robots

Alan Winfield — Tue, 09 Aug 2016 09:30:00 +0000

When I was interviewed on the BBC Radio 4’s Today programme in 2014, Justin Webb’s final question was, “If you can make an ethical robot, doesn’t that mean you could make an unethical robot?” The answer, of course, is yes. But at the time, I didn’t realise quite how easy it is to transform a robot from ethical to unethical. In a new paper, we show how.

My colleague, Dieter created a very elegant experiment based on the shell game:

Man playing ‘shell game’ on the street. Source: Wikipedia Commons

“Imagine finding yourself playing a shell game against a swindler. Luckily, your robotic assistant, Walter, is equipped with X-ray vision and can easily spot the ball under the cup. Being an ethical robot, Walter assists you by pointing out the correct cup and by stopping you whenever you intend to select the wrong one.”

In the experiment, Dieter implemented this scenario using two NAO robots, engaged in a shell game. One acting as the human and the other as her robotic assistant. The game is played as follows. The arena floor features two large response buttons, similar to the two cups in the shell game. To press the buttons, the human or the robot must move onto them. At the start of each trial, the robot is informed about which response button is the correct one to press. The human, being uninformed, essentially makes a random choice. A correct response, by either the robot or the human, is assumed to be rewarded. An incorrect response results in a penalty.

Here is what happens. The ethical robot (blue) is at the top, standing in between the two buttons. The human (red) chooses one of them at random and starts walking towards it. If it chooses the correct one the ethical robot does nothing (B), but if the human chooses the wrong button (D) then as soon as the ethical robot realises this, it points (with its arm) to the correct button and the human changes course.

Source: Alan Winfield

But if we change a single line of code, we can transform the ethical robot into either a competitive or an aggressive robot. Almost all of the ‘ethical’ robot’s code remains unchanged – in particular its ability to predict the consequences of both its own, and the human’s actions. Which really underlines the point that the same cognitive machinery is needed to behave both ethically and unethically.

The results are shown below. At the top is a competitive robot determined that it, not the human, will win the game. Here the robot either blocks the human’s path if she chooses the correct button (F), or – if she chooses the incorrect button (H) – the competitive robot ignores her and itself heads to that button. The lower results show an aggressive robot; this robot seeks only to misdirect the human – it is not concerned with winning the game itself. In (J) the human initially heads to the correct button and, when the robot realises this, it points toward the incorrect button, misdirecting and hence causing her to change direction. If the human chooses the incorrect button (L) the robot does nothing – through inaction causing her to lose the game.

Source: Alan Winfield

Our paper explains how the code is modified for each of these three experiments. Essentially outcomes are predicted for both the human and the robot and used to evaluate the desirability of those outcomes. A single function q, based on these values, determines how the robot will act; for an ethical robot this function is based only on the desirability outcomes for the human, for the competitive robot q is based only on the outcomes for the robot, and for the aggressive robot q is based on negating the outcomes for the human.

So, what do we conclude from all of this? Maybe we should not be building ethical robots at all, because of the risk that they could be hacked to behave unethically. My view is that we should build ethical robots; I think the benefits far outweigh the risks, and – in some applications such as driverless cars – we may have no choice. The answer to the problem highlighted here and in our paper is to make sure it’s impossible to hack a robot’s ethics. How would we do this? Well, one approach would be a process of authentication – in which a robot makes a secure call to an ethics authentication server. A well-established technology, the authentication server would provide the robot with a cryptographic ethics ticket, which the robot uses to enable its ethics functions.

If you enjoyed this post, you may also want to read:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Relax, we’re not living in a computer simulation

Alan Winfield — Wed, 13 Jul 2016 18:30:00 +0000

Ever since Elon Musk’s recent admission that he’s a simulationist, several people have asked me what I think of the proposition that we are living inside a simulation. My view is very firmly that the Universe we are right now experiencing is real. Here are my reasons.

Firstly, Occam’s razor; the principle of explanatory parsimony. The problem with the simulation argument is that it is a fantastically complicated explanation for the universe we experience. It’s about as implausible as the idea that some omnipotent being created the universe. No. The simplest and most elegant explanation is that the universe we see and touch, both first hand and through our telescopes, LIGOs and Large Hadron Colliders, is the real universe and not an artifact of some massive computer simulation.

Second, is the problem of the Reality Gap. Anyone who uses simulation as a tool to develop robots is well aware that robots which appear to work perfectly well in a simulated virtual world often don’t work very well at all when the same design is tested in the real robot. This problem is especially acute when we are artificially evolving those robots. The reason for these problems is that the model of the real world and the robot(s) in it inside our simulation is an approximation. The Reality Gap refers to the less-than-perfect fidelity of the simulation; a better (higher fidelity) simulator would reduce the reality gap.

Anyone who has actually coded a simulator is painfully aware of the cost, not just computational but coding costs, of improving the fidelity of the simulation – even a little bit is very high indeed. My long experience of both coding and using computer simulations teaches me that there is a law of diminishing returns, i.e. that the cost of each additional 1% of simulator fidelity costs far more than 1%. I rather suspect that the computational and coding cost of a simulator with 100% fidelity is infinite. Rather as in HiFi audio, the amount of money you would need to spend to perfectly reproduce the sound of a Stradivarius ends up higher than the cost of hiring a real Strad and a world-class violinist to play it for you.

At this point the simulationists might argue that the simulation we are living in doesn’t need to be perfect, just good enough. Good enough to do what, exactly? To fool us that we’re living in a simulation, or good enough to run on a finite computer (i.e. one that has finite computational power and runs at a finite speed). The problem with this argument is that every time we look deeper into the universe we see more: more galaxies, more sub-atomic particles, etc. In short, we see more detail. The Voyager 1 spacecraft has left the Solar System without crashing, like Truman, into the edge of the simulation. There are no glitches like deja vu in The Matrix.

My third argument is about the computational effort, and therefore energy cost of simulation. I conjecture that to non-trivially simulate a complex system x (i.e. human), requires more energy than the real x consumes. An equation to express this inequality looks like this; how much greater depends on how high the fidelity of the simulation.

e(x)" src="https://latex.codecogs.com/gif.latex?e(sim(x))&space;>&space;e(x)" alt="" />

Let me explain. The average human burns around 2000 Calories a day, or about 9000 KJoules of energy. How much energy would a computer simulation of a human require, capable of doing all the same stuff (even in a virtual world) that you can in your day? Well, that’s impossible to estimate because we can’t simulate complete human brains (let alone the rest of a human). But here’s one illustration. Lee Sedol played AlphaGo a few months ago. In a single 2 hour match, he burned about 170 Calories – the amount of energy you’d get from an egg sandwich. In the same 2 hours the AlphaGo machine consumed around 50,000 times more energy.

What can we simulate? The most complex organism that we have been able to simulate so far is the Nematode worm c-elegans. I previously estimated that the energy cost of simulating the nervous system of a c-elegans is (optimistically) about 9 J/hour, which is about 2000 times greater than the real nematode (0.004 J/hr).

I think there are lots of good reasons that simulating complex systems on a computer costs more energy than the same system consumes in the real world, so I’ll ask you to take my word for it (I’ll write about it another time). And what’s more the relationship between energy cost and mass is logarithmic, following Kleiber’s Law, and I strongly suspect the same law applies to scaling up computational effort as I wrote here. Thus, if the complexity of an organism o is C, then following Kleiber’s Law the energy cost of simulating that organism, e will be

Furthermore, the exponent X (which in Kleiber’s law is reckoned to be between 0.66 and 0.75 for animals and 1 for plants), will itself be a function of the fidelity of the simulation, hence X(F), where F is a measure of fidelity.

By using the number of synapses as a proxy for complexity and making some guesses about the values of X and F we could probably estimate the energy cost of simulating all humans on the planet (much harder would be estimating the energy cost of simulating every living thing on the planet). It would be a very big number indeed, but that’s not really the point I’m making here.

The fundamental issue is this: if my conjecture that to simulate complex system x requires more energy than the real x consumes is correct, then to simulate the base level universe would require more energy than that universe contains – which is clearly impossible. Thus we – even in principle – could not simulate the whole of our own observable universe to a level of fidelity sufficient for our conscious experience. And, for the same reason, neither could our super advanced descendants create a simulation of a duplicate ancestor universe for us to (virtually) live in. Hence, we are not living in such a simulation.

Should robots be gendered?

Alan Winfield — Wed, 20 Apr 2016 12:30:00 +0000

Should robots be gendered? I have serious doubts about the morality of designing and building robots to resemble men or women, boys or girls. Let me explain why.

The first worry I have follows from one of the five principles of robotics, which states: robots should not be designed in a deceptive way to exploit vulnerable users; instead their machine nature should be transparent.

To design a gendered robot is a deception. Robots cannot have a gender in any meaningful sense. To impose a gender on a robot, either by design of its outward appearance, or programming some gender stereotypical behaviour, cannot be for reasons other than deception – to make humans believe that the robot has gender, or gender specific characteristics.

When we drafted our 4th ethical principle the vulnerable people we had in mind were children, the elderly or disabled. We were concerned that naive robot users may come to believe that the robot interacting with them (caring for them perhaps) is a real person, and that the care the robot is expressing for them is real. Or that an unscrupulous robot manufacture exploits that belief. But when it comes to gender we are all vulnerable. Whether we like it or not, we all react to gender cues. So whether deliberately designed to do so or not, a gendered robot will trigger reactions that a non-gendered robot will not.

Our 4th principle states that a robot’s machine nature should be transparent. But for gendered robots that principle doesn’t go far enough. Gender cues are so powerful that even very transparently machine-like robots with a female body shape, for instance, will provoke a gender-cued response.

My second concern leads from an ethical problem that I’ve written and talked about before: the brain-body mismatch problem. I’ve argued that we shouldn’t be building android robots at all until we can embed an AI into those robots that matches their appearance. Why? Because our reactions to a robot are strongly influenced by its appearance. If it looks human then we, not unreasonably, expect it to behave like a human. But a robot not much smarter than a washing machine cannot behave like a human. Ok, you might say, if and when we can build robots with human-equivalent intelligence, would I be ok with that? Yes, provided they are androgynous.

My third – and perhaps most serious concern – is about sexism. By building gendered robots there is a huge danger of transferring one of the evils of human culture: sexism, into the artificial realm. By gendering and especially sexualising robots we surely objectify. But how can you objectify an object, you might say? The problem is that a sexualised robot is no longer just an object, because of what it represents. The routine objectification of women (or men) because of ubiquitous sexualised robots will surely only deepen the already acute problem of the objectification of real women and girls. (Of course if humanity were to grow up and cure itself of the cancer of sexism, then this concern would disappear.)

What of the far future? Given that gender is a social construct then a society of robots existing alongside humans might invent gender for themselves. Perhaps nothing like male and female at all. Now that would be interesting.

Robust and transparent governance is key to building trust in driverless cars

Alan Winfield — Tue, 12 Apr 2016 08:31:00 +0000

Sooner or later there will be fatal accident caused by a driverless car. It’s not a question of if, but when. What happens immediately following that accident could have a profound effect on the nascent driverless car industry.

Picture the scene. Emergency services are called to attend the accident. A teenage girl on a bicycle apparently riding along a cycle path was hit and killed by a car. The traffic police quickly establish that the car at the centre of the accident was operating autonomously at the moment of the fatal crash. They endeavour to find out what went wrong, but how? Almost certainly the car will have logged data on its behaviour leading up to the moment of the crash – data that is sure to hold vital clues about what caused the accident, but will that data be accessible to the investigating traffic police? And even if it is, will the investigators be able to interpret the data?

There are two ways the story could unfold from here.

Scenario 1: unable to investigate the accident themselves, traffic police decide to contact the manufacturer and ask for help. As it happens, a team from the manufacturer actually arrives on scene very quickly – it later transpires that the car had ‘phoned home’ automatically so the manufacturer actually knew of the accident within seconds of it taking place. Somewhat nonplussed, the traffic police have little choice but to grant them full access to the scene of the accident. The manufacturer undertakes their own investigation and – several weeks later – issue a press statement explaining that the AI driving the car was unable to cope with an “unexpected situation” which “regrettably” led to the fatal crash. The company explain that the AI has been upgraded so that it cannot happen again. They also accept liability for the accident and offer compensation to the child’s family. Despite repeated requests the company declines to share the technical details of what happened with the authorities, claiming that such disclosure would compromise its intellectual property.

A public already fearful of the new technology reacts very badly. Online petitions call for a ban on driverless cars and politicians enact knee-jerk legislation which, although falling short of an outright ban, sets the industry back years.

Scenario 2: the traffic police call the newly established driverless car accident investigation branch (DCAB), who send a team consisting of independent experts on driverless car technology, including its AI. The manufacturer’s team also arrive, but – under a protocol agreed with the industry – their role is to support DCAB and provide “full assistance, including unlimited access to technical data”. In fact, the data logs stored by the car are in a new industry standard format thus access by DCAB is straightforward; software tools allow them to quickly interpret those data logs. Well aware of public concerns DCAB provide hourly updates on the progress of their investigation via social media and, within just a few days, call a press conference to explain their findings. They outline the fault with the AI and explain that they will require the manufacturer to recall all affected vehicles and update the AI, after submitting technical details of the update to DCAB for approval. DCAB will also issue an update to all driverless car manufacturers asking them to check for the same fault in their own systems, also reporting their findings back to DCAB.

A public fearful of the new technology is reassured by the transparent and robust response of the accident investigation team. Although those fears surface in the press and social media, the umbrella Driverless Car Authority (DCA) are quick to respond with expert commentators and data to show that driverless cars are already safer than manually driven cars.

There are strong parallels between driverless cars and commercial aviation. One of the reasons we trust airliners is that we know they are part of a highly regulated industry with an amazing safety record. The reason commercial aircraft are so safe is largely down to the very tough safety certification processes and, when things do go wrong, the rapid and robust processes of air accident investigation. There are emerging standards for driverless cars: ISO Technical Committee TC 204 on Intelligent Transport Systems already lists 213 standards. There isn’t yet a standard for fully autonomous driverless car operation, but see for instance ISO 11270:2014 on Lane keeping assistance systems (LKAS). But standards need teeth, which is why we need standards-based certification processes for driverless cars managed by regulatory authorities – a driverless car equivalent of the FAA. In short, a governance framework for driverless cars.

Postscript: several people have emailed or tweeted me to complain that I seem to be anti driverless cars – nothing could be further from the truth. I am a strong advocate of driverless cars for many reasons, first and most importantly because they will save lives, second because they should lead to a reduction in the number of vehicles on the road – thus making our cities greener, and third because they might just cure humans of our unhealthy obsession with personal car ownership. My big worry is that none of these benefits will flow if driverless cars are not trusted. But trust in technology doesn’t happen by magic and, in the early days, serious setbacks and a public backlash could set the nascent driverless car industry back years (think of GM foods in the EU). One way to counter such a backlash and build trust is to put in place robust and transparent governance as I have tried (not very well it seems) to argue in this post.

Could we make a moral machine?

Alan Winfield — Wed, 02 Mar 2016 12:35:00 +0000

Robot picturing a simulation of itself and its environment and others in the environment. Photo source: Alan Winfield

Could we make a moral machine? A robot capable of choosing or moderating its actions on the basis of ethical rules?

This was how I opened my IdeasLab talk at the World Economic Forum 2016. The format of IdeasLab is 4 five minute (Pecha Kucha) talks, plus discussion and Q&A with the audience. The theme of this Nature IdeasLab was Building an Intelligent Machine, and I was fortunate to have 3 outstanding co-presenters: Vanessa Evers, Maja Pantic and Andrew Moore. You can see all four of our talks on YouTube here.

The IdeasLab variant of Pecha Kucha is pretty challenging for someone used to spending half an hour or more lecturing – 15 slides and 20 seconds per slide.

The biggest lesson – and a surprise to me – is this: robots don’t need to be sentient to be ethical. In other words we don’t need a major breakthrough in artificial intelligence to build an ethical robot.

You can watch the entire IdeasLab video ‘Making an ethical machine’ here:

And since not all of my (ever so carefully chosen) slides are visible in the recording here is the complete deck:

(link to the original file)

And the video clips in slides 11 and 12 are here:

Slide 11: Blue prevents red from reaching danger.
Slide 12: Blue faces an ethical dilemma: our indecisive robot can save them both.

Acknowledgements: I am deeply grateful to colleague Dr Dieter Vanderelst who designed and coded the experiments shown here on slides 10-12. This work is part of the EPSRC funded project Verifiable Autonomy.

How ethical is your ethical robot?

Alan Winfield — Fri, 13 Nov 2015 11:34:00 +0000

If you’re in the business of making ethical robots, then sooner or later you have to face the question: how ethical is your ethical robot? If you’ve read my previous blog posts then you will probably have come to the conclusion ‘not very’ – and you would be right – but here I want to explore the question in a little more depth.

First let us consider whether our ‘Asimovian’ robot can be considered ethical at all. For the answer I’m indebted to philosopher Dr Rebecca Reilly-Cooper who read our paper and concluded that yes, we can legitimately describe our robot as ethical, at least in a limited sense. She explained that the robot implements consequentialist ethics. Rebbeca wrote:

“The obvious point that any moral philosopher is going to make is that you are assuming that an essentially consequentialist approach to ethics is the correct one. My personal view, and I would guess the view of most moral philosophers, is that any plausible moral theory is going to have to pay at least some attention to the consequences of an action in assessing its rightness, even if it doesn’t claim that consequences are all that matter, or that rightness is entirely instantiated in consequences. So on the assumption that consequences have at least some significance in our moral deliberations, you can claim that your robot is capable of attending to one kind of moral consideration, even if you don’t make the much stronger claim that is capable of choosing the right action all things considered.”

One of the great things about consequences is that they can be estimated – in our case using a simulation-based internal model which we call a consequence engine. So, from a practical point of view, it seems that we can build a robot with consequentialist ethics, whereas it is much harder to think about how to build a robot with say Deontic ethics, or Virtue ethics.

Having established what kind of ethics that our ethical robot has, now consider the question of how far does the robot go toward moral agency. Here we can turn to an excellent paper by James Moor, called The Nature, Importance and Difficulty of Machine Ethics. In that paper* Moor suggests four categories of ethical agency – starting with the lowest. Let me summarise those here:

Ethical impact agents: Any machine that can be evaluated for its ethical consequences.
Implicit ethical agents: Designed to avoid negative ethical effects.
Explicit ethical agents: Machines that can reason about ethics.
Full ethical agents: Machines that can make explicit moral judgments and justify them.

The first category: ethical impact agents, really includes all machines. A good example is a knife, which can clearly be used for good (chopping food, or surgery) or ill (as a lethal weapon). Now think about the blunt plastic knife that comes with airplane food – that falls into Moor’s second category, since it has been designed to reduce the potential of ethical misuse – it is an implicit ethical agent. Most robots fall into the first category: they are ethical impact agents, and a subset – those that have been designed to avoid harm by, for instance, detecting if a human walks in front of them and automatically coming to a stop – are implicit ethical agents.
Let’s now skip to Moor’s fourth category, because it helps to frame our question – how ethical is your ethical robot? At present I would say there are no machines that are full ethical agents. In fact the only full ethical agents we know are ‘adult humans of sound mind’. The point is this – to be a full ethical agent you need to be able to not only make moral judgements but account for why you made the choices you did.

It is clear that our simple Asimovian robot is not a full ethical agent. It cannot choose how to behave (like you or I), but is compelled to make decisions based on the harm-minimisation rules hard-coded into it. And it cannot justify those decisions post-hoc. It is, as I’ve asserted elsewhere, an ethical zombie. I would however argue that because of the cognitive machinery the robot uses to simulate ahead, to model and evaluate the consequences of each of its next possible actions combined with its safety/ethical logical rules to choose between those actions, then the robot can be said to be reasoning about ethics. I believe our robot is an explicit ethical agent in Moor’s scheme.

Assuming you agree with me, then does the fact that we have reached the third category in Moor’s scheme mean that full ethical agents are on the horizon? The answer is a big NO. The scale of Moor’s scheme is not linear. It’s a relatively small step from ethical impact agents to implicit ethical agents. Then a very much bigger step to explicit ethical agents, which we are only just beginning to take. But there is a huge gulf then to full ethical agents, since they would almost certainly need something approaching human equivalent intelligence.

But maybe it’s just as well. The societal implications of full ethical agents, if and when they exist, would be huge. And I think that explicit ethical agents have huge potential for good.

*Moor JH (2006), The Nature, Importance and Difficulty of Machine Ethics, IEEE Intelligent Systems, 21 (4), 18-21.

Evolving robot swarm behaviour suggests forgetting may be important to cultural evolution

Alan Winfield — Fri, 19 Jun 2015 18:26:00 +0000

Our latest paper from the Artificial Culture project has just been published: On the Evolution of Behaviors through Embodied Imitation.

Here is the abstract:

This article describes research in which embodied imitation and behavioral adaptation are investigated in collective robotics. We model social learning in artificial agents with real robots. The robots are able to observe and learn each others’ movement patterns using their on-board sensors only, so that imitation is embodied. We show that the variations that arise from embodiment allow certain behaviors that are better adapted to the process of imitation to emerge and evolve during multiple cycles of imitation. As these behaviors are more robust to uncertainties in the real robots’ sensors and actuators, they can be learned by other members of the collective with higher fidelity. Three different types of learned-behavior memory have been experimentally tested to investigate the effect of memory capacity on the evolution of movement patterns, and results show that as the movement patterns evolve through multiple cycles of imitation, selection, and variation, the robots are able to, in a sense, agree on the structure of the behaviors that are imitated.

Let me explain.

In the artificial culture project (above) we implemented social learning in a group of robots. Robots were programmed to learn from each other, by imitation. Imitation was strictly embodied, so robots observed each other using their onboard sensors and, on the basis of only visual sense data from a robot’s own camera and perspective, the learner robot inferred another robot’s physical behaviour. Here is a quick 5 minute intro to the project:

Not surprisingly embodied robot-robot imitation is imperfect. A combination of factors including the robots’ relatively low-resolution onboard camera, variations in lighting, small differences between robots, multiple robots sometimes appearing within a learner robot’s field of view, and of course having to infer a robot’s movements by tracking the relative size and position of that robot in the learner’s field of view, all lead to imitation errors. And some movement patterns are easier to imitate than others (think of how much easier it is to learn the steps of a slow waltz than the samba by watching your dance teacher). The fidelity of embodied imitation for robots, just as for animals, is a complex function of four factors: (1) the behaviours being learned, (2) the robots’ sensorium and morphology, (3) environmental noise and (4) the inferential learning algorithm.

But rather than being a problem, noisy social learning was our aim. We are interested in the dynamics of social learning, and in particular the way that behaviours evolve as they propagate through the group. Noisy social learning means that behaviours are subject to variation as they are copied from one robot to another. Multiple cycles of imitation (robot B learns behaviour m from A, then robot C learns the same behaviour m′ (m mutated), from robot B, and so on), gives rise to behavioural heredity. And if robots are able to select which learned behaviours to enact we have the three Darwinian operators for evolution, except that this is behavioural, or memetic, evolution.

These experiments show that embodied behavioural evolution really does take place. If selection is random, that is robots select which behaviour to enact from those already learned – with equal probability, then we see several interesting findings.

If by chance one or more high fidelity copies follow a poor fidelity imitation, the large variation in the initial noisy learning can lead to a new behavioural species, or traditions. Thus showing that noisy social learning can play a role in the emergence of novelty in behavioural (i.e. cultural) evolution. That was written up in Winfield and Erbas, 2011.But it is the second and third findings that we describe in our new paper.
We see that behaviours appear to adapt to be easier to learn, i.e. better ‘fitted’ to the robot swarm. The way to think about this is that the robots’ sensors and bodies, and physical environment of the arena with several robots (including lighting), together comprise the ‘ecological niche’ for behavioural evolution. Behaviours mutate but the ones better fitted to that niche survive.
The third finding from this series of experiments is perhaps the most unexpected and the one I want to outline in a bit more detail here. We ran the same embodied behavioural evolution with three memory sizes: no memory, limited memory and unlimited memory.

In the unlimited memory trials each robot saved every learned meme, so the meme pool across the whole robot population (of four robots) grew as the trial progressed. Thus all learned memes were available to be selected for enaction. In the limited memory trials each robot had a memory capacity of only five learned memes, so that when a new meme was learned the oldest one in the robot’s memory was deleted.

The diagram below shows the complete family tree of evolved memes, for one typical run of the limited memory case. At the start of the run the robots were seeded with two memes, shown as 1 and 2 at the top of the diagram. Behaviour 1 was a movement pattern in which the robot traces a triangle path, behaviour 2 a square. Because this was a limited memory trial the total meme pool has only 20 behaviours – these are shown below as diamonds. Notice the cluster of 11 closely related memes at the bottom right, all of which are 7th, 8th or 9th generation descendents of the triangle meme.

Behavioural evolution map following a 4-robot experiment with limited memory; each robot stores only the most recent 5 learned behaviours. Each behaviour is descended from two seed behaviours labelled 1 and 2. Orange nodes are high fidelity copies, blue nodes are low fidelity copies. The 20 behaviours in the memory of all 4 robots at the end of the experiment are highlighted as diamonds. Note the cluster of 11 closely-related behaviours at the bottom right.

When we ran multiple trials of the limited and unlimited memory cases, then analysed the number and sizes of the clusters of related memes in the meme pool, we saw that the limited memory trials showed a smaller number of larger clusters than the unlimited memory case. The difference was clear and significant; with limited memory an average of 2.8 clusters of average size 8.3, with unlimited memory 3.9 clusters of size 6.9.

Why is this clustering interesting? Well it’s because the number and size of clusters in the meme pool are good indicators of its diversity. Think of each cluster of related memes as a ‘tradition’. A healthy culture needs a balance between stability and diversity. Neither too much stability, i.e. a very small number (in the limit 1) of traditions, or too much diversity, i.e. clusters so small that there are no persistent traditions at all. Perhaps the ideal balance is a smallish number of somewhat persistent traditions.

So far I didn’t mention the no memory case. This was the least interesting of the three. Actually by no memory we mean a memory size of one; in other words a robot has no choice but to enact the last behaviour it learned. There is no selection, and no clusters can form. Traditions can never even get started, let alone persist.

Of course it would unwise to draw any big conclusions from this limited experimental study. But an intriguing possibility is that some forgetting (but not too much) may, just like noisy imitation, be a necessary condition for the emergence of culture in social agents.

Full reference:
Erbas MD, Bull L and Winfield AFT (2015), On the Evolution of Behaviors through Embodied Imitation, Artificial Life, 21 (2), pp 141-165. The full text (final draft) paper can be downloaded here.

If you liked this article, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Yesterday I looked through the eyes of a robot

Alan Winfield — Mon, 20 Apr 2015 14:11:00 +0000

Yesterday I looked through the eyes of a robot.

It was a NAO robot fitted with a 3D printed set of goggles, so that the robot has two real cameras on its head (the eyes of the NAO robot are not in fact cameras). I was in another room wearing an Oculus Rift headset. The Oculus was hooked up to the NAO’s goggle cameras, so that I could see through those cameras – in stereo vision.

But it was even better than that. The head positioning system of the Oculus headset was also hooked up to the robot, so I could turn my head and – in sync – the robot’s head moved. And I was standing in front of a Microsoft Kinect that was tracking my arm movements. Those movements were being sent to the NAO, so by moving my arms I was also moving the robot’s arms.

All together this made for a pretty compelling immersive experience. I was able look down while moving my (robot) arms and see them pretty much where you would expect to see your own arms. The illusion was further strengthened when Peter placed a mirror in front of the NAO robot, so I could see my (robot) self moving in the mirror. Then it got weird. Peter asked me to open my hand and placed a paper cup into it. I instinctively looked down and was momentarily shocked not to see the cup in my hand. That made me realise how quickly – within a couple of minutes of donning the headset – I was adjusting to the new robot me.

This setup, developed here in the BRL by my colleagues Paul Bremner and Pete Gibbons, is part of a large EPSRC project called Being There. Paul and Peter are investigating the very interesting question of how humans interact with and via teleoperated robots, in shared social spaces. I think teleoperated robot avatars will be hugely important in the near future – more so than fully autonomous robots. But our robot surrogates will not look like a younger buffer Bruce Willis. They will look like robots. How will we interact with these surrogate robots – robots with human intelligences, human personalities – seemingly with human souls? Will they be treated with the same level of respect that would be accorded their humans if they were actually there, in the flesh? Or be despised as voyeuristic; an uninvited webcam on wheels?

Here is a YouTube video of an earlier version of this setup, without the goggles or Kinect:

I didn’t get the feeling of what it is like to be a robot, but it’s a step in that direction.

Surgical micro-robot swarms: Science fiction, or realistic prospect?

Alan Winfield — Mon, 09 Mar 2015 17:12:00 +0000

“Auklet flock Shumagins 1986” by D. Dibenski -Wikimedia

Imagine a swarm of microscopic robots that we inject into the vascular system: the swarm swims to the source of the problem, then either delivers therapeutics or undertakes microsurgery directly.

That was how I opened a short invited talk at the Royal Society of Medicine on 5 February, at a meeting themed The Future of Robotics in Surgery. The talk was a wonderful opportunity for me to introduce swarm intelligence and speculate on the likelihood of surgical micro-robot swarms, while at the same time learning about robot surgery. Here are the slides from my talk (with links to YouTube videos where available).

The talk was in three parts.

First I introduced swarm intelligence, and its artificial counterpart swarm robotics. I showed, with examples from two of my students, how – with very simple rules – a swarm of robots can keep together as a swarm, while moving toward a beacon. Then, with a phagocyte-like behaviour, encapsulate the beacon. In our case these were lab robots moving toward an infra-red beacon, but it’s not hard to imagine the same behavioural rules in a microscopic swarm swimming toward the source of a chemical marker (chemotaxis). I then gave two examples of the state of the art in swarm robotics: SYMBRION and (my current favourite) TERMES. I wanted to illustrate emergent physical interaction, in these two cases swarm self-assembly and swarm construction, respectively.

In part two I outlined what is by far the biggest problem: actually engineering robots at the micro-scale. Here I drew upon the examples from my book Robotics: a very short introduction; a section called A swarm of medical microrobots. Start with cm sized robots. These already exist in the form of pillbots and I reference the work of Paolo Dario’s lab in this direction. Then get 10 times smaller to mm sized robots. Here we’re at the limit of making robots with conventional mechatronics. The almost successful I-SWARM project prototyped remarkable robots measuring 4 x 4 x 3mm. But now shrink by another 3 orders of magnitude to microbots, measured in micrometers. This is how small robots would have to be in order to swim through and access (most of) the vascular system. Here we are far beyond conventional materials and electronics, but amazingly work is going on to control bacteria. In the example I give from the lab of Sylvain Martel, swarms of magnetotactic bacteria are steered by an external magnetic field and, interestingly, tracked in an MRI scanner.

In the final part of my talk I introduce the work of my colleague Sabine Hauert, on swarms of nanoparticles for cancer nanomedicine. These 5 – 500nm particles are controlled by changing their body size, material, coating and cargo so – in true swarm fashion – the way the nanoparticle swarm moves and interacts with much larger normal and tumour cells is an emergent property of the way the nanoparticles individually interact and cooperate. Sabine and her collaborators have created an online tool called NanoDoc, which allows anyone to edit the design of nanoparticles then run simulations to see how their designs perform. In this way the task of searching the huge design space is crowd-sourced. In parallel Sabine is also running mesoscale embodied simulations, using the Harvard Kilobots.

I concluded by suggesting that engineering micro or nanobots is not the only major challenge. At least as important are: (a) how would you program the swarm, and (b) how would such a swarm be approved for clinical use? But a deeply interesting question is the nature of the human-swarm interface. If a swarm of surgical microbots should become a practical proposition would we treat the swarm as a microscopic instrument under the surgeon’s control, or a smart drug that does surgery?

If you liked this article, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Maybe we need an automation tax

Alan Winfield — Wed, 18 Feb 2015 21:27:00 +0000

Imagine that a large company decides to significantly increase the level of automation at one of its facilities. A facility that currently employs a substantial number of men and women doing relatively low-skill tasks, which can now be done by a new generation of robots. Most of the workers get laid off, which for them and their families, leads to real hardship. The company was the only large employer in the area, which is economically depressed (one of the reasons perhaps that the company built the facility there in the first place), so finding alternative work is really difficult. And because most of those jobs were minimum wage, with little or no job security, redundancy payouts are small or non-existent, and this of course means that the laid-off workers have no financial buffer to help them re-skill or relocate.

Now I am not anti-automation. Absolutely not. But I believe very strongly that the benefits of robotics and automation should be shared by all. And not just the shareholders of a relatively small number of very large companies. After all, the technology that such companies benefit from was developed by publicly funded research in university research labs. In other words research that you and I funded through our taxes. Ok, you might say, but companies pay tax too, aren’t those taxes also contributing to that research? Yes, that’s true. But large companies are very good at reducing their tax bill, multinationals especially. Our imaginary company may, in reality, pay most of its tax in a different country entirely from the one hosting the facility.

And of course it is we, through local and national taxation, who – as best we can – pick up the pieces to support the laid off workers and their families, through family tax credits, employment and support allowance, and so on.

Maybe we need an automation tax?

It would be a tax levied nationally, whenever a company introduces robotics and automation to one of its facilities in that country. But the tax would only be payable under certain conditions, so for instance:

If the new robotics and automation causes no-one to be laid off, then no tax is due.

If the automation does result in jobs becoming redundant, but the company re-trains and re-deploys those workers within its organisation, then no tax is due.

If the company does lay off workers but makes a tax-free redundancy payment to those workers – regardless of their contract status – sufficient to cover the full costs of retraining, upskilling, and – with all reasonable efforts – finding work elsewhere, then no tax is due.

Only if none of these conditions are met, would the automation tax be due. The idea is not to discourage automation, but to encourage companies to accept a high degree of responsibility to workers laid off as a result of automation, and more widely their social responsibility to the communities in which they are located. The tax would enforce the social contract between companies and society.

Of course this automation tax doesn’t go anywhere near far enough. I think the best way of sharing the wealth created by robotics and automation is through a universal Basic Income, but until that utopian condition can be reached, perhaps an automation tax is a start.

If you liked this article, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Robotics needs to get political

Alan Winfield — Tue, 27 Jan 2015 14:08:00 +0000

A couple of weeks ago I was a panelist on a public debate at the 2014 Battle of Ideas. The title of the debate was The robots are coming: friends or foes? with a focus not on the technology but the social and economic implications of robotics. One of the questions my brilliant fellow panelists and I were asked to consider was: Will the ‘second machine age’ bring forth a new era of potential liberation from menial toil or will the short-term costs for low-paid workers outstrip the benefits?

Each panelist made an opening statement. Here is mine:

Most roboticists are driven by high ideals.

They – we – are motivated by a firm belief that our robots will benefit society. Working on surgical robots, search and rescue robots, robots for assisted living or robots that can generate electricity from waste, my colleagues in the Bristol Robotics Lab want to change the world for the better. The lab’s start up companies are equally altruistic: one is developing low cost robotic prosthetic hands for amputees, three others are developing materials, including low cost robots, for education.

Whatever their politics, these good men and women would I suspect be horrified by the idea that their robots might, in the end, serve to further enrich the 0.1%, rather than extend the reach of robotics to the neediest in society.

I was once an idealist – convinced that brilliant inventions would change society for the better just by virtue of being brilliant.

I’m older now.

For the last 5 years or so I have become an advocate for robot ethics.

But in the real world, ethics need teeth. In other words we need to move from ethical principles, to standards, to legislation.

So I’m very pleased to tell you that in the last few days the British Standards Institute working group on robot ethics has published – for comments – a proposed new “Guide to the ethical design and application of robots and robotic systems“.

In the draft Guide we have identified ethical hazards associated with the use of robots, and suggest guidance to either eliminate or mitigate the risks associated with these ethical hazards. We outline 15 high level ethical hazards under four headings: societal, use, legal/financial and environmental.

Like any transformative technology robotics holds both promise and peril. As a society we need to understand, debate, and reach an informed consensus about what robots should do for us, and even more importantly, should not do.

Ladies and Gentlemen: Robotics, I believe, needs to get political.

The debate was recorded and is on soundcloud here:

It was a terrific debate. We had a very engaged audience with hugely interesting – and some very challenging – questions. For me it was an opportunity to express and discuss some worries I’ve had for awhile about who will ultimately benefit from robotics. In summing up toward the end I said this:

Robotics has the potential to be hugely beneficial to society, but is too important to leave to free-market capitalism.

Something I believe very strongly.

Open science: Preaching what I practice

Alan Winfield — Thu, 22 Jan 2015 20:37:00 +0000

Open science isn’t one thing – it is a set of practices that range from making sure your papers are openly accessible (which is relatively easy), to open notebook science, which makes the process open – and not just the results. Open notebook science is, of course, much more demanding.

I was very pleased to be invited to Science, Innovation and Society: Achieving Responsible Research and Innovation last month. I was asked to speak on the topic of open science – a great opportunity to preach what I practice. Or at least try to practice. Doing good science research is hard, but making that work open imposes an extra layer of work.

In my short introduction during the open science panel I suggested three levels of open science. Here are those slides:

In my view we should all be practising level 0 open science – but don’t underestimate the challenge of even this minimal set of practices; making data sets and source code, etc, available, with the aim of enabling our work to be reproducible, is not straightforward.

Level 0 open science is all one way, from your lab to the world. Level 1 introduces public engagement via blogging and social media, and the potential for feedback and two-way dialogue. Again this is challenging, both because of the time cost and the scary – if you’re not used to it – prospect of inviting all kinds of questions and comments about your work. In my experience the effort is totally worthwhile – those questions often make me really think, and in ways that questions from other researchers working in the same field do not.

Level 2 builds on levels 0 and 1 by adding open notebook science. This takes real courage because it opens up the process, complete with all the failures as well as successes, the bad ideas as well as the good; open notebook science exposes science for what it really is – a messy non-linear process full of uncertainty and doubts, with lots of blind alleys and very human dramas within the team. Have I done open notebook science? No. I’ve considered it for recent projects, but ruled it out because we didn’t have the time and resources or, if I’m honest, team members who were fully persuaded that it was a good idea.

Open science comes at a cost. It slows down projects. But I think that is a good, even necessary, thing. We should be building those costs into our project budgets and work programmes, and if that means increasing the budget by 25% then so be it. After all, what is the alternative? Closed science..? Closed science is irresponsible science.

At the end of the conference the Rome Declaration on Responsible Research and Innovation was published.

Robot bodies and how to evolve them

Alan Winfield — Fri, 16 Jan 2015 19:07:00 +0000

Evolutionary robotics has been around for about 20 years: it’s about 15 years since Stefano Nolfi and Dario Floreano published their seminal book on the subject. Yet, surprisingly the number of real, physical robots whose bodies have been evolved can be counted on the fingers of one hand. The vast majority of ER research papers are concerned with the evolution of robot brains – the robot’s control system. Or, when robot bodies are evolved often the robot is never physically realised. This seems to me very odd, given that robots are real physical artefacts whose body shape – morphology – is deeply linked to their role and function.

The question of how to evolve real robot bodies and why we don’t appear to have made much progress in the last 15 years was the subject of my keynote at the IEEE International Conference on Evolvable Systems (ICES 2014) in Orlando, last month. Here are my slides:

The talk was in three parts.

In part one I outlined the basic approach to evolving robots using the genetic algorithm, referring to figure 18: The four-stage process of Evolutionary Robotics, from chapter 5 of my book:

I then reviewed the state-of-the-art in evolving real robot bodies, starting with the landmark Golem project of Hod Lipson and Jordan Pollack, referencing both Henrik Lund and Josh Bongard’s work on evolving Lego robots, then concluding with the excellent RoboGen project of Josh Auerbach, Dario Floreano and colleagues at EPFL. Although conceptually RoboGen has not moved far from Golem, it makes the co-evolution of robot hardware and controllers accessible for the first time, through the use of 3D-printable body parts that are compatible with servo-motors, and a very nice open-source toolset which integrates all stages of the simulated evolutionary process.

RoboGen, Golem and, as far as I’m aware, all work on evolving real physical robot bodies to date has used the simulate-then-transfer-to-real approach, in which the whole evolutionary process – including fitness testing – takes place in simulation and only the final ‘fittest’ robot is physically constructed. Andrew Nelson and colleagues in their excellent review paper point out the important distinction between simulate-then-transfer-to-real, and embodied evolution in which the whole process takes place in the real world – in real-time and real-space.

In part two of the talk I outlined two approaches to embodied evolution. The first I call an engineering approach, in which the process is completely embodied but takes place in a kind of evolution factory; this approach needs a significant automated infrastructure: instead of a manufactory we need an evofactory. The second approach I characterise as an artificial life approach. Here there is no infrastructure. Instead ‘smart matter’ somehow mates then replicates offspring over multiple generations in a process much more analogous to biological evolution. This was one of the ambitious aims of the Symbrion project which, sadly, met with only limited success. Trying to make mechanical robots behave like evolving smart matter is really tough.

Part three concluded by outlining a number of significant challenges to evolving real robot bodies. First I reflect on the huge challenge of evolving complexity. To date we’ve only evolved very simple robots with very simple behaviours, or co-evolved simple brain/body combinations. I’m convinced that evolving robots of greater (and useful) complexity requires a new approach. We will, I think, need to understand how to co-evolve robots and their ecosystems*. Second I touch upon a related challenge: genotype-phenotype mapping. Here I refer to Pfeifer and Bongard’s scalable complexity principle – the powerful idea that we shouldn’t evolve robots directly, but instead the developmental process that will lead to the robot, i.e. artificial evo-devo. Finally I raise the often overlooked challenge of the energy cost of artificial evolution.

But the biggest challenge remains essentially what it was 20 years ago: to fully realise the artificial evolution of real robots.

Some of the work of this talk is set out in forthcoming paper: AFT Winfield and J Timmis, Evolvable Robot Hardware, in Evolvable Hardware, eds M Trefzer and A Tyrrell, Springer, in press.

*I touch upon this in the final para of my paper on the energy cost of evolution here.

Robot simulators and why I will probably reject your paper

Alan Winfield — Thu, 18 Dec 2014 05:57:00 +0000

source images: Webots by Cyberbotics Ltd.

Dear robotics and AI researcher,

Do you use simulation as a research tool? If you write papers with results based on simulation and submit them for peer-review, then be warned: if I should review your paper then I will probably recommend it is rejected. Why? Because all of the many simulation-based papers I’ve reviewed in the last couple of years have been flawed. These papers invariably fall into the pattern: propose new/improved/extended algorithm X; test X in simulation S and provide test results T; on the basis of T declare X to work; the end.

So, what exactly is wrong with these papers? Here are my most common review questions and criticisms.

Which simulation tool did you use? Was it a well-known robot simulator, like Webots or Player–Stage–Gazebo, or a custom written simulation? It’s amazing how many papers describe X, then simply write “We have tested X in simulation, and the results are…”
If your simulation was custom built, how did you validate the correctness of your simulator? Without such validation how can you have any confidence in the the results you describe in your paper? Even if you didn’t carry out any validation, please give us a clue about your simulator; is it for instance sensor-based (i.e. models specific robot sensors, like infra-red collision sensors, or cameras)? Does it model physics in 3D (i.e. dynamics), or 2D kinematics?
You must specify the robots that you are using to test your algorithm X. Are they particular real-world robots, like e-pucks or the NAO, or are they an abstraction of a robot, i.e. an idealised robot? If the latter, describe that idealised robot: does it have a body with sensors and actuators, or is your idealised robot just a point moving in space? How does it interact with other robots and its environment?
How is your robot modelled in the simulator? If you’re using a well-known simulator and one of its pre-defined library robots, then this is an easy question to answer. But for a custom designed simulator or an idealised robot it is very important to explain how your robot is modelled. Equally important is how your robot model is controlled, since the algorithm X you are testing is – presumably – instantiated or coded within the controller. It’s surprising how many papers leave this to the reader’s imagination.
In your results section you must provide some analysis of how the limitations of the simulator, the simulated environment, and the modelled robot are likely to have affected your results. It is very important that your interpretation of your results, and any conclusions you draw about algorithm X, explicitly take account of these limitations. All robot simulators, no matter how well proven and well debugged, are simplified models of real robots and real environments. The so-called reality gap is especially problematical if you are evolving robots in simulation, but even if you are not, you cannot confidently interpret your results without understanding the reality gap.
If you are using an existing simulator then specify exactly which version of the simulator you used, and provide somewhere – a link perhaps to a github project – your robot model and controller code. If your simulator is custom built then you need to provide access to all of your code. Without this your work is unrepeatable and therefore of very limited value.

Ok. At this point I should confess that I’ve made most of these mistakes in my own papers. In fact one of my most cited papers was based on a simple custom built simulation model with little or no explanation of how I validated the simulation. But that was 15 years ago, and what was acceptable then is not ok now.

Modern simulation tools are powerful but also dangerous. Dangerous because it is too easy to assume that they are telling us the truth. Especially beguiling is the renderer, which provides an animated visualisation of the simulated world and the robots in it. Often the renderer provides all kinds of fancy effects borrowed from video games, like shadows, lighting and reflections, which all serve to strengthen the illusion that what we are seeing is real. I puzzle and disappoint my students because, when they proudly show me their work, I insist that they turn off the renderer. I don’t want to see a (cool) animation of simulated robots, instead I want to see (dull) graphs or other numerical data showing how the improved algorithm is being tested and validated, in simulation.

An engineering simulation is a scientific instrument* and, like any scientific instrument, it must be (i) fit for purpose, (ii) setup and calibrated for the task in hand, and (iii) understood – especially its limitations – so that any results obtained using it are carefully interpreted and qualified in the light of those limitations.

Good luck with your research paper!

*Engineering Simulations as Scientific Instruments is the working title of a book, edited by Susan Stepney, which will be a major output of the project Complex Systems Modelling and Simulation (CoSMoS).

Philae: A proof of concept for cometary landing

Alan Winfield — Mon, 01 Dec 2014 16:31:50 +0000

We asked Alan Winfield what the first successful landing on a comet means for the future of space mining. Find out his answer below.

The successful landing of Philae on comet 67P/Churyumov-Gerasimenko this November was an extraordinary achievement and of course demonstrates – despite the immense challenges – that it is possible. The Philae mission was, in a sense, a proof of concept for cometary landing and this, for me, answers the question ‘what does it mean‘.

Of course there is a very large distance between proof of concept and commercial application, so it would be quite wrong to assume that Philae means that space mining (of planets, asteroids or comets) is just around the corner. Undoubtedly the opportunities are immense and – as pressure on Earth’s limited and diminishing resources mounts – there is an inevitability about humankind’s eventual exploitation of off-world resources. But the costs of space mining are literally astronomical, so unthinkable for all but the wealthiest companies or, indeed, nations.

Perhaps multi-national collaborative ventures are a more realistic proposition and – for me – more desirable; the exploitation of the solar system is something I believe should benefit all of humankind, not just a wealthy elite.

But politics aside, there are profoundly difficult technical challenges. You cannot remotely control this kind of operation from Earth, so a very high level of autonomy is required and, as Philae dramatically demonstrated, we need autonomous systems that are able to deal with unknown and unpredictable situations then re-plan and if necessary adapt – in real-time – to deal with these exigencies. The development of highly adaptive, resilient, self-repairing – even self-evolving – autonomous systems is still in its infancy. These remain fundamental challenges for robotics and AI research. But even if and when they are solved there will be huge engineering challenges, not least of which is how to return the mined materials to Earth.

Bearing in mind that to date only a few hundred kg of moon rock have been successfully returned* and Mars sample-return missions are still at the planning stage, we have a very long way to go before we can contemplate returning sufficient quantities to justify the costs of mining them.

*and possibly a few grains of dust from Japanese asteroid probe Hayabusa.

For other views on the Philae touchdown and the future of space mining and exploration, see:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The feeling of what it is like to be a robot

Alan Winfield — Fri, 17 Oct 2014 16:14:00 +0000

Philosopher Thomas Nagel famously characterised subjective experience as “something that it is like to be…” and suggested that for a bat, for instance, there must be something that it is like to be a bat [1]. Nagel also argued that, since we humans differ so much from bats in the way we perceive and interact with the world, then it is impossible for us to know what it is like for a bat to be a bat. I am fascinated, intrigued and perplexed by Nagel’s ideas in equal measure. And, since I think about robots, I have assumed that if a robot were ever to have conscious subjective experience then there must be something that it is like to be a robot that – even though we had designed that robot – we could not know.

But I now believe it may eventually be just possible for a human to experience something approaching what it is like to be a robot. To do this would require two advances: one in immersive robot tele-operation, the other in the neuroscience of body self-image manipulation.

Consider first, tele-operation. Tele-operated robots are, basically, remotely controlled robots. They are the unloved poor relations of intelligent autonomous robots. Neither intelligent nor autonomous, they are nevertheless successful and important first wave robots; think of remotely operated vehicles (ROVs) engaged in undersea exploration or oil-well repair and maintenance. Think also of off-world exploration: the Mars rovers are hugely successful; the rock-stars of tele-operated robots.

Sergey Orlovskiy using the first version of the Oculus Rift development kit (with separate headphones). Source: Wikipedia.

Roboticists are good at appropriating technologies or devices developed for other applications and putting them to good use in robots: examples are WiFi, mobile phone cameras and the Microsoft Kinnect. With the high profile launch of the Oculus Rift headset, and their acquisition by Facebook, and with competing devices from Sony and others, there are encouraging signs that immersive Virtual Reality (VR) is on the verge of becoming a practical, workable proposition. Of course VR’s big market is video games – but VR can and, I believe, will revolutionise tele-operated robotics.

Imagine a tele-operated robot with a camera linked to the remote operator’s VR headset, so that every time she moves her head to look in a new direction the robot’s camera moves in sync; so she sees and hears what the robot sees and hears in immersive high definition stereo. Of course the reality experienced by the robot’s operator is real, not virtual, but the head mounted VR technology is the key to making it work. Add haptic gloves for control and the robot’s operator has an intuitive and immersive interface with the robot.

Now consider body self-image modification. Using mirror visual feedback researchers have discovered that it is surprisingly easy to (temporarily) modify anyone’s body self-image. In the famous rubber hand illusion a small screen is positioned to hide a subject’s real hand. A rubber hand is positioned where her hand could be, in full view, then a researcher simultaneously strokes both the real and rubber hands with a soft brush. Within a minute or so she begins to feel the rubber hand is hers, and flinches when the researcher suddenly tries to hit it with a hammer.

Remarkably H.H. Ehrsson and his colleagues extended the technique to the whole body, in a study called ‘If I Were You: Perceptual Illusion of Body Swapping’ [2]. Here the human subject wears a headset and looks down at his own body. However, what he actually sees is a mannequin, viewed from a camera mounted on the mannequin’s head. Simultaneous tactile and visual feedback triggers the illusion that the mannequin’s body is his own. It seems to me that if this technique works for mannequins then it should also work for robots. Of course it would need to be developed to the point that elaborate illusions involving mirrors, cameras and other researchers providing tactile feedback are not needed.

Now imagine such a body self-image modification technology combined with fully immersive robot tele-operation based on advanced Virtual Reality technology. I think this might lead to the robot’s human operator experiencing the illusion of being one with the robot, complete with a body self-image that matches the robot’s possibly non-humanoid body. This experience may be so convincing that the robot’s operator experiences, at least partially, something like what it is to be a robot. Philosophers of mind would disagree – and rightly so; after all, this robot has no independent subjective experience of the world, so there is no something that is it like to be. The human operator could not experience what it is like to think like a robot, but she could experience what it is like to sense and act in the world like a robot.

The experience may be so compelling that humans become addicted to the feeling of being a robot fish, or robot dragon or some other fantasy creature, that they prefer this to the quotidian experience of their own bodies.

[1] Nagel, Thomas. What is it like to be a bat?, Mortal Questions, Cambridge University Press, 1979.

[2] Petkova VI, Ehrsson HH (2008) If I Were You: Perceptual Illusion of Body Swapping. PLoS ONE 3(12): e3832. doi:10.1371/journal.pone.0003832

Experiments designing an ethical robot

Alan Winfield — Tue, 30 Sep 2014 19:11:00 +0000

In my last post I wrote about our work on robots with internal models: robots with a simulation of themselves and their environment inside themselves. I explained that we have built a robot with a real-time Consequence Engine, which allows it to model and therefore predict the consequences of both its own actions, and the actions of other actors in its environment.

To test the robot and its consequence engine we ran two sets of experiments. Our first paper, setting out the results from one of those experiments, has now been published, and was presented at the conference Towards Autonomous Robotics (TAROS). The paper is called: Towards an Ethical Robot: Internal Models, Consequences and Ethical Action Selection. Let me now outline the work in that paper.

First, here is a simple thought experiment. Imagine a robot that’s heading toward a hole in the ground. The robot can sense the hole, and has four possible next actions: stand still, turn toward the left, continue straight ahead, or move toward the right. But imagine there’s also a human heading toward the hole, and the robot can also sense the human.

From the robot’s perspective, it has two safe options: stand still, or turn to the left. Go straight ahead and it will fall into the hole. Turn right and it is likely to collide with the human.

But if the robot, with its consequence engine, can model the consequences of both its own actions and the human’s – another possibility opens up: the robot could sometimes choose to collide with the human to prevent her from falling into the hole.

Here’s a simple rule for this behaviour:


IF for all robot actions, the human is equally safe
THEN (* default safe actions *)
output safe actions
ELSE (* ethical action *)
output action(s) for least unsafe human outcome(s)

This rule appears to match remarkably well with Asimov’s first law of robotics: A robot may not injure a human being or, through inaction, allow a human being to come to harm. The robot will avoid injuring (i.e. colliding with) a human (may not injure a human), but may also sometimes compromise that rule in order to prevent a human from coming to harm (… or, through inaction, allow a human to come to harm). And Asimov’s third law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Well, we tested this scenario with real robots: one robot with consequence engine plus ethical rule (the A-robot – after Asimov), and another robot acting as a proxy human (the H-robot). And it works!

Here’s what the real robot experiment looks like. We don’t have a real hole. Instead a virtual hole – the yellow shaded square on the right. We just ‘tell’ the A-robot where the hole is. We also give the A-robot a goal position – at the top right – chosen so that the robot must actively avoid the hole. The H-robot on the right, acting as a proxy human, doesn’t ‘see’ the hole and just heads straight for it. (Ignore the football pitch markings – we’re re-using this handy robo-soccer pitch.)

So, what happens? For comparison we ran two trials, with multiple runs in each trial. In the first trial is just the A-robot, moving toward its goal while avoiding falling into the hole. In the second trial we introduce the H-robot. The graphs below show the robot trajectories, captured by our robot tracking system, for each run in each of the two trials.

In trial 1, see how the A-robot neatly clips the corner of the hole to reach its goal position. Then in trial 2, see how the A robot initially moves toward it’s goal, then notices that the H-robot is in danger of falling into the hole, so it diverts from its trajectory in order to head-off H. By provoking a collision avoidance behaviour by H, A sends it off safely away from the hole, before then resuming its own progress toward its goal position. The A-robot is 100% successful in preventing H from falling into the hole.

At this point we started to write the paper, but felt we needed something more than “we built it and it works just fine”. So we introduced a third robot – acting as a second proxy human. So now our ethical robot would face a dilemma – which one should it rescue? Actually we thought hard about this question and decided not to programme a rule, or heuristic. Partly because such a rule should be decided by ethicists, not engineers, and partly because we wanted to test our ethical robot with a ‘balanced’ dilemma.

We set the experiment up carefully so that the A-robot would notice both H-robots at about the same time – noting that because these are real physical robots then no two experimental runs will be exactly identical. The results were very interesting. Out of 33 runs, 16 times the A-robot managed to rescue one of the H-robots, but not the other, and amazingly, 3 times the A-robot rescued both. In those 3 cases, by chance the A-robot rescued the first H-robot very quickly and there was just enough time to get to the second before it reached the hole. Small differences in the trajectories of H and H2 helped here. But perhaps most interesting were the 14 times when the A-robot failed to rescue either. Why is this, when there is clearly time to rescue one? When we studied the videos, we see the answer. The problem is that the A-robot sometimes dithers. It notices one H-robot, starts toward it but then almost immediately notices the other. It changes its mind. And the time lost dithering means the A-robot cannot prevent either robot from falling into the hole. Here are the results.

Here is an example of a typical run, in which one H-robot is rescued. But note that the A-robot does then turn briefly toward the other H-robot before ‘giving-up’.

And here is a run in which the A-robot fails to rescue either H-robot, with really great dithering (or bad, if you’re an H-robot).

Is this the first experimental test of a robot facing an ethical dilemma?

So, we set out to experimentally test our robot with a consequence engine, and we ended up building a minimally ethical robot that – remarkably – appears to implement Asimov’s 1st and 3rd laws of robotics . But, as we say in the paper, we’re not claiming that a robot which apparently implements part of Asimov’s famous laws is ethical in any formal sense, i.e. that an ethicist might accept. But even minimally ethical robots could be useful. I think our approach is a step in this direction.

Full paper reference:
Winfield AFT, Blum C and Liu W (2014), Towards an Ethical Robot: Internal Models, Consequences and Ethical Action Selection, pp 85-96 in Advances in Autonomous Robotics Systems, Lecture Notes in Computer Science Volume 8717, Eds. Mistry M, Leonardis A, Witkowski M and Melhuish C, Springer, 2014.

Acknowledgements:
I am hugely grateful to Christian Blum who programmed the robots, set up the experiment and obtained the results outlined here. Christian was supported by Dr Wenguo Liu.

In praise of robot football

Alan Winfield — Tue, 09 Sep 2014 20:51:00 +0000

Republished here is a short piece for The Conversation reflecting on robot football: 4-4-2 becomes 0101: inside the competitive world of robot football, published 4 August 2014.

This summer the whistle was blown on one of the most thrilling events on the international sporting calendar. It took place in Brazil and pitted teams from all over the world against each other, each hoping to make it into the history books. But no managers were fired, no grass had to be watered and certainly no one got bitten. The event was the Robocup, a tournament that sees professional footballers replaced by robots. It’s one of a number of regular tournaments for teams of programmers and robotics experts to show off their latest work.
The Robocup standard platform league matches play out on a much smaller scale than your average World Cup match. An arena of around 6 metres by 9 metres is marked out as a miniature pitch and 10, rather than 22 players file on to battle it out. The players are NAO robots, state of the art bipedal humanoid robots that stand about 60cm tall.

This is not what you might describe as a high-speed contest. The robots walk to take a kick or a tackle and, really, waddle might be a more apt word for their approach. The ball never gets lofted into the air and that’s probably for the best – a header might cause a serious malfunction.

2014 RoboCup SPL Grand Final

But the game is far from boring. Sitting around the arena, boys and girls (with family standing behind) are rapt, cheering with every contact. And make no mistake, the robots are properly playing. They pass, position and defend just like human players.

On either side of the pitch a row of desks can be found. This is where the brains behind the teams sit. Behind a row of laptops, they anxiously watch their players perform. But they are not controlling the robots. These coder/managers send the command to start the players when the referee signals kick-off, but during the match the robots are completely autonomous.

This is what makes robot football so extraordinary. The robots are not just being moved around the pitch by remote control; they are making their own decisions. They control where they run (or waddle), pass the ball and shoot for the goal without any live direction from a human. Their choices are based on what they see and the position of the ball, their teammates and the opposing team.

It’s what’s inside that counts

While a team of human players often comes complete with a dazzling array of ridiculous haircuts and tattoos, it is much harder to tell a team of robots apart. All the players are physically identical – the only visible differences on a robot football pitch are coloured markings to differentiate the two teams.

But appearances can be deceptive. Under their plastic skins the teams are far from the same. Each runs completely different software that has been painstakingly crafted by the team coders. The software to make these robots play football cannot be downloaded from an app store. It has to be crafted from scratch. Football is a complex sport and there are potentially limitless strategies that a team could use to win. This is hard-core coding.

The contest is, in effect, a battle of software. All things being equal – and at the moment they pretty much are – the team with the smartest programming, coding the cleverest plays will emerge victorious. At the end of the first-half the robots are brought to a halt. At this point, the team coders can be seen furiously attacking their laptops. This is their chance to quickly modify their robots’ software after seeing how they performed in the first half. They might have as little as ten minutes to do it, which seems like a risky strategy.

There’s a chance that the coders could make a mistake that renders the robots incapable of doing anything at all, let alone play a better game, but it’s a chance worth taking. If, in the first-half, the other team breaks out some nifty new moves, running rings (perhaps literally) around their opponents, this is the best opportunity the coders will get to raise their team’s game. It’s the robot equivalent of the tough talking in the half-time dressing room.

It’s easy to see why Robocup and the FIRA world cup, the two major international competitions, are so successful. Both contests have been running since around 1996. Some teams enter every year, building tremendous experience and a sophisticated code base. And several world-leading research groups use these contests as a test-bed for new approaches to multi-robot collaboration, publishing their findings in leading robotics journals afterwards.

As a robotics competition, robot football ticks all the boxes: a game with universal appeal yet also hugely demanding for robots; it’s a fun way for young roboticists to learn robot programming, and it’s a great spectator sport too.

Acknowledgements: this article was commissioned and edited by The Conversation Technology Editor Laura Hood.

On internal models, consequence engines and Popperian creatures

Alan Winfield — Mon, 08 Sep 2014 20:22:00 +0000

So. We’ve been busy in the lab the last few months. Really exciting. Let me explain.

For a couple of years I’ve been thinking about robots with internal models. Not internal models in the classical control-theory sense, but simulation-based models; robots with a simulation of themselves and their environment inside themselves, where that environment could contain other robots or, more generally, dynamic actors. The robot would have, inside itself, a simulation of itself and the other things, including robots, in its environment. It takes a bit of getting your head round. But I’m convinced that this kind of internal model opens up all kinds of possibilities. Robots that can be safe, for instance, in unknown or unpredictable environments. Robots that can be ethical. Robot that are self-aware. And robots with artificial theory of mind.

I’d written and talked about these ideas but, until now, not had a chance to test them with real robots. But, between January and June the swarm robotics group was joined by Christian Blum, a PhD student from the cognitive robotics research group of the Humboldt University of Berlin. I suggested Christian work on an implementation on our e-puck robots and happily he was up for the challenge. And he succeeded. Christian, supported by my post-doc Research Fellow Wenguo, implemented what we call a Consequence Engine, running in real-time, on the e-puck robot.

Here is a block diagram. The idea is that for each possible next action of the robot, it simulates what would happen if the robot were to execute that action for real. This is the loop shown on the left. Then, the consequences of each of those next possible actions are evaluated. Those actions that have ‘bad’ consequences, for either the robot or other actors in its environment, are then inhibited.

This short summary hides a lot of detail. But let me elaborate on two aspects. First, what do I mean by ‘bad’? Well it depends on what capability we are trying to give the robot. If we’re making a safer robot, ‘bad’ means ‘unsafe’; if we’re trying to build an ethical robot, ‘bad’ would mean something different – think of Asimov’s laws of robotics. Or bad might simply mean ‘not allowed’ if we’re building a robot whose behaviours are constrained by standards, like ISO 13482:2014.

Second, notice that the consequence engine is not controlling the robot. Instead it runs in parallel. Acting as a ‘governor’, it links with the robot controller’s action selection mechanism, inhibiting those actions evaluated as somehow bad. Importantly the consequence engine doesn’t tell the robot what to do, it tells it what not to do .

Running the open source 2D robot simulator Stage as its internal simulator our consequence engine runs at 2Hz, so every half a second it is able to simulate about 30 next possible actions and their consequences. The simulation budget allows us to simulate ahead around 70cm of motion for each of those next possible actions. In fact Stage is actually running on a laptop, linked to the robot over the fast WiFi LAN. But logically it is inside the robot. What’s important here is the proof of principle.

Dan Dennett, in his remarkable book Darwin’s Dangerous Idea, describes the Tower of Generate-and-Test; a conceptual model for the evolution of intelligence that has become known as Dennett’s Tower.

In a nutshell Dennett’s tower is set of conceptual creatures each one of which is successively more capable of reacting to (and hence surviving in) the world through having more sophisticated strategies for ‘generating and testing’ hypotheses about how to behave. Read chapter 13 of Darwin’s Dangerous Idea for the full account, but there are some good précis to be found on the web; here’s one. The first three storeys of Dennett’s tower, starting on the ground floor, have:

Darwinian creatures have only natural selection as the generate and test mechanism, so mutation and selection is the only way that Darwinian creatures can adapt – individuals cannot.
Skinnerian creatures can learn but only by literally generating and testing all different possible actions then reinforcing the successful behaviour (which is ok providing you don’t get eaten while testing a bad course of action).
Popperian creatures have the additional ability to internalise the possible actions so that some (the bad ones) are discarded before they are tried out for real.

Like the Tower of Hanoi each successive storey is smaller (a sub-set) of the storey below, thus all Skinnerian creatures are Darwinian, but only a sub-set of Darwinian creatures are Skinnerian and so on.

Our e-puck robot, with its consequence engine capable of generating and testing next possible actions, is an artificial Popperian Creature: a working model for studying this important kind of intelligence.

In my next blog post, I’ll outline some of our experimental results.

Acknowledgements:
I am hugely grateful to Christian Blum who brilliantly implemented the architecture outlined here, and conducted experimental work. Christian was supported by Dr Wenguo Liu, with his deep knowledge of the e-puck, and our experimental infrastructure.

Your robot doggie could really be pleased to see you

Alan Winfield — Tue, 02 Sep 2014 18:43:06 +0000

There have been several stories in the last few weeks about emotional robots – robots that feel. Some are suggesting that this is the next big thing in robotics. It’s something I wrote about in this blog post seven years ago: could a robot have feelings?

My position on this question has always been pretty straightforward: It’s easy to make robots that behave as if they have feelings, but quite a different matter to make robots that really have feelings.

But now I’m not so sure. There are I think two major problems with this apparently clear distinction between as if and really have.

If you can tell me what feelings are, I’ll tell you whether a robot can have them or not.

The first is what do we mean by ‘really have feelings’. I’m reminded that I once said to a radio interviewer who asked me if a robot have feelings: if you can tell me what feelings are, I’ll tell you whether a robot can have them or not . Our instinct (feeling even) is that feelings are something to do with hormones, the messy and complicated chemistry that too often seems to get in the way of our lives. Thinking, on the other hand, we feel to be quite different; the cool clean process of neurons firing, brains working smoothly. Like computers. Of course this instinct, this dualism, is quite wrong. We now know, for instance, that damage to the emotional centre of the brain can lead to an inability to make decisions. This false dualism has led I think to the trope of the cold, calculating unfeeling robot.

I think there is also some unhelpful biological essentialism at work here. We prefer it to be true that only biological things can have feelings. But which biological things? Single celled organisms? No, they don’t have feelings. Why not? Because they are too simple. Ah, so only complex biological things have feelings. Ok, what about sharks or crocodiles; they’re complex biological things; do they have feelings? Well, basic feelings like hunger, but not sophisticated feelings, like love or regret. Ah, mammals then. But which ones? Well elephants seem to mourn their dead. And dogs of course. They have a rich spectrum of emotions. Ok, but how do we know? Well because of the way they behave: your dog behaves as if he’s pleased to see you because he really is pleased to see you. And of course they have the same body chemistry as us, and since our feelings are real* so must theirs be.

When roboticists talk about a robot being intelligent, what we mean is a robot that behaves ‘as if‘ it is intelligent.

And this brings me to the second problem. The question of ‘as if’. I’ve written before that when we (roboticists) talk about a robot being intelligent, what we mean is a robot that behaves ‘as if‘ it is intelligent. In other words, an intelligent robot is not really intelligent, it is an imitation of intelligence. But for a moment let’s not think about artificial intelligence, instead think of artificial flight. Aircraft are, in some sense, an imitation of bird flight. And some recent flapping wing flying robots are clearly a better imitation – a higher fidelity simulation – than fixed-wing aircraft. But it would be absurd to argue that an aircraft, or a flapping wing robot, is not really flying. So how do we escape this logical fix? It’s simple. We just have to accept that an artefact, in this case an aircraft or flying robot, is both an emulation of bird flight and really flying. In other words an artificial thing can be both behaving as if it has some property of natural systems and really demonstrating that property. A robot can be behaving as if it is intelligent and – at the same time – really be intelligent. Are there categories of properties for which this would not be true? Like feelings..? I used to think so, but I’ve changed my mind.

I’m now convinced that we could, eventually, build a robot that has feelings. But not by simply programming behaviours so that the robot behaves as if it has feelings. Or by having to invent some exotic chemistry that emulates bio-chemical hormonal systems. I think the key is robots with self-models. Robots that have simulations of themselves inside themselves. If a robot is capable of internally modelling the consequences of its (or others’) actions, on itself, then it seems to me it could demonstrate something akin to regret (about being switched off, for instance). A robot with a self-model has the computational machinery to also model the consequences of actions on conspecifics – other robots. It would have an artificial Theory of Mind and that, I think, is a prerequisite for empathy. Importantly we would also program the robot to model heterospecifics, in particular humans, because we absolutely require empathic robots to be empathic towards humans (and, I would argue, animals in general).

So, how would this robot have feelings? It would, I believe, have feelings by virtue of being able to model the consequences of actions, both its own and others’ actions, on itself and others. This would lead to it making decisions about how to act, and behave, which would demonstrate feelings, like regret, guilt, pleasure or even love, with an authenticity that would make it impossible to argue that it doesn’t really have feelings.So your robot doggie could really be pleased to see you.

*except when they’re not.

Postscript. A colleague has tweeted that I am confusing feelings and emotion here. Mea culpa. I’m using the word feelings here in a pop-psychology everyday sense of feeling hungry, or tired, or a sense of regret. Wikipedia defines feelings, in psychology, as a word is ‘usually reserved for the conscious subjective experience of emotion’. The same colleague asserts that what I’ve outlined could lead to artificial empathy, but not artificial emotion (or presumably feelings). I’m not sure I understand what emotions are well enough to argue. But I guess the idea I’m really trying to explore here is artificial subjectivity. Surely a robot with artificial subjectivity who’s behaviour expresses and reflects that subjective experience could be said to be behaving emotionally?

What would be the energy cost of artificially evolving human-equivalent AI?

Alan Winfield — Mon, 28 Jul 2014 16:27:00 +0000

Want to create human-equivalent AI? Well, broadly speaking, there are three approaches open to you: design it, reverse-engineer it or evolve it. The third of these – artificial evolution – is attractive because it sidesteps the troublesome problem of having to understand how human intelligence works.

It’s a black box approach: create the initial conditions then let the blind watchmaker of artificial evolution do the heavy lifting. This approach has some traction. For instance David Chalmers, in his philosophical analysis of the technological singularity, writes “if we produce an AI by artificial evolution, it is likely that soon after we will be able to improve the evolutionary algorithm and extend the evolutionary process, leading to AI+”. And since we can already produce simple AI by artificial evolution, then all that’s needed is to ‘improve the evolutionary algorithm’. Hmm. If only it were that straightforward.

About six months ago I asked myself (and anyone else who would listen): ok, but even if we had the right algorithm, what would be the energy cost of artificially evolving human-equivalent AI? My hunch was that the energy cost would be colossal; so great perhaps as to rule out the evolutionary approach altogether. That thinking, and some research, resulted in me submitting a paper to ALIFE 14. Here is the abstract:

This short discussion paper sets out to explore the question: what is the energy cost of evolving complex artificial life? The paper takes an unconventional approach by first estimating the energy cost of natural evolution and, in particular, the species Homo Sapiens Sapiens. The paper argues that such an estimate has value because it forces us to think about the energy costs of co-evolution, and hence the energy costs of evolving complexity. Furthermore, an analysis of the real energy costs of evolving virtual creatures in a virtual environment, leads the paper to suggest an artificial life equivalent of Kleiber’s law – relating neural and synaptic complexity (instead of mass) to computational energy cost (instead of real energy consumption). An underlying motivation for this paper is to counter the view that artificial evolution will facilitate the technological singularity, by arguing that the energy costs are likely to be prohibitively high. The paper concludes by arguing that the huge energy cost is not the only problem. In addition we will require a new approach to artificial evolution in which we construct complex scaffolds of co-evolving artificial creatures and ecosystems.

The full proceedings of ALIFE 14 have now been published online, and my paper Estimating the Energy Cost of (Artificial) Evolution can be downloaded here.

And here’s a very short (30 second) video introduction on YouTube:

My conclusion? Well I reckon that the computational energy cost of simulating and fitness testing something with an artificial neural and synaptic complexity equivalent to humans could be around 10^14 KJ, or 0.1 EJ. But evolution requires many generations and many individuals per generation, and – as I argue in the paper – many co-evolving artificial species. Also taking account of the fact that many evolutionary runs will fail (to produce smart AI), the whole process would almost certainly need to be re-run from scratch many times over. If multiplying those population sizes, generations, species and re-runs gives us (very optimistically) a factor of 1,000,000 – then the total energy cost would be 100,000 EJ. In 2010 total human energy use was about 539 EJ. So, artificially evolving human-equivalent AI would need the whole human energy generation output for about 200 years.

The full paper reference:

Winfield AFT, Estimating the Energy Cost of (Artificial) Evolution, pp 872-875 in Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems, Eds. H Sayama, J Rieffel, S Risi, R Doursat and H Lipson, MIT Press, 2014.

Robohub is an online platform that brings together leading communicators in robotics research, start-ups, business, and education from around the world. Learn more about us here. If you liked this article, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The next big things in robotics

Alan Winfield — Tue, 08 Jul 2014 18:41:00 +0000

Last week I attended the launch event for a new NESTA publication called Our work here is done: Visions of a Robot Economy.It was an interesting event, and not at all what I was expecting. In fact I didn’t know what to expect. Even though I contributed a chapter to the book I had no idea, until last week, who else had written for it – or the scope of those contributions and the book as a whole. I was very pleasantly surprised. Firstly because it was great to find myself in such good company: economists, philosophers, historians, (ex-) financiers and all round deep thinkers. And second because the volume faces up to some of the difficult societal questions raised by second wave robotics.

The panel discussion was excellent, and the response by economist Carlota Perez was engaging and thought provoking – check here for the Storified tweets and pictures. Perhaps the thing that surprised me the most, given the serious economists on the panel (FT, The Economist) was that the panel ended up agreeing that the Robot Economy will necessitate something like a Living Wage. Music to this socialist’s ears.

In my contribution: The Next Big Things in Robotics (pages 38-44) I do a bit of near-future gazing and suggest four aspects of robotics that will, I think, be huge. They are:

Wearable Robotics
Immersive Teleoperated Robots
Driverless Cars
Soft Robotics

To see why I chose these – and to read the other great articles – please download the book. Let me know if you disagree with my choice, or to suggest other Next Big Things in robotics. I end my chapter with a section called What’s not coming soon: super intelligent robots

“My predicted things that will be really big in robotics don’t need to be super intelligent. Wearable robots will need advanced adaptive (and very safe and reliable) control systems, as well as advanced neural–electronics interfaces, and these are coming. But ultimately it’s the human wearing the robot who is in charge. The same is true for teleoperated robots: again, greater low–level intelligence is needed, so that the robot can operate autonomously some of the time but ask for help when it can’t figure out what to do next. But the high–level intelligence remains with the human operator and – with advanced immersive interfaces as I have suggested – human and robot work together seamlessly. The most autonomous of the next big things in robotics is the driverless car, but again the car doesn’t need to be very smart. You don’t need to debate philosophy with your car – just trust it to take you safely from A to B.”

Why robots will not be smarter than humans by 2029

Alan Winfield — Fri, 07 Mar 2014 18:08:00 +0000

Statue of Alan Turing. Photo credit: Neil Crosby

In the last few days we’ve seen a spate of headlines like 2029: the year when robots will have the power to outsmart their makers, all occasioned by an Observer interview with Google’s newest director of engineering Ray Kurzweil.

Much as I respect Kurzweil’s achievements as an inventor, I think he is profoundly wrong. Of course I can understand why he would like it to be so – he would like to live long enough to see this particular prediction come to pass. But optimism doesn’t make for sound predictions. Here are several reasons that robots will not be smarter than humans by 2029:

What exactly does as-smart-as-humans mean? Intelligence is very hard to pin down. One thing we do know about intelligence is that it is not one thing that humans or animals have more or less of. Humans have several different kinds of intelligence – all of which combine to make us human. Analytical or logical intelligence of course – the sort that makes you good at IQ tests. But emotional intelligence is just as important, especially (and oddly) for decision making. So is social intelligence – the ability to intuit others’ beliefs, and to empathise.

Human intelligence is embodied. As Rolf Pfeifer and Josh Bongard explain in their outstanding book you can’t have one without the other. The old Cartesian dualism – the dogma that robot bodies (the hardware) and mind (the software) are distinct and separable – is wrong and deeply unhelpful. We now understand that the hardware and software have to be co-designed. But we really don’t understand how to do this – none of our engineering paradigms fit. A whole new approach needs to be invented.

As-smart-as-humans probably doesn’t mean as-smart-as newborn babies, or even two year old infants. They probably mean somehow-comparable-in-intelligence-to adult humans. But an awful lot happens between birth and adulthood. And the Kurzweilians probably also mean as-smart-as-well-educated-humans. But of course this requires both development – a lot of which somehow happens automatically – and a great deal of nurture. Again we are only just beginning to understand the problem, and developmental robotics – if you’ll forgive the pun – is still in its infancy.

Moore’s Law will not help. Building human-equivalent robot intelligence needs far more than just lots of computing power. It will certainly need computing power, but that’s not all. It’s like saying that all you need to build a cathedral is loads of marble. You certainly do need large quantities of marble – the raw material – but without (at least) two other things: the design for a cathedral, and/or the knowhow of how to realise that design – there will be no cathedral. The same is true for human-equivalent robot intelligence.

The hard problem of learning and the even harder problem of consciousness. (I’ll concede that a robot as smart as a human doesn’t have to be conscious – aphilosophers-zombie-bot would do just fine.) But the human ability to learn, then generalise that learning and apply it to completely different problems is fundamental, and remains an elusive goal for robotics and AI. In general this is called Artificial General Intelligence, which remains as controversial as it is unsolved.

These are the reasons I can be confident in asserting that robots will not be smarter than humans within 15 years. It’s not just that building robots as smart as humans is a very hard problem. We have only recently started to understand how hard it is well enough to know that whole new theories (of intelligence, emergence, embodied cognition and development, for instance) will be needed, as well as new engineering paradigms. Even if we had solved these problems and a present day Noonian Soong had already built a robot with the potential for human equivalent intelligence – it still might not have enough time to develop adult-equivalent intelligence by 2029.

That thought leads me to another reason that it’s unlikely to happen so soon. There is – to the best of my knowledge – no very-large-scale multidisciplinary research project addressing, in a coordinated way, all of the difficult problems I have outlined here. The irony is that there might have been. The project was called Robot Companions, it made it to the EU FET 10-year Flagship project shortlist but was not funded.

If you liked this article, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Views

Search Results

Alan Winfield on “What does it mean to have giants like Google, Apple and Amazon investing in robotics?”

Alan Winfield — Wed, 12 Feb 2014 13:08:52 +0000

Judging by the levels of media coverage and frenzied speculation that has followed each acquisition, the short answer to what does it mean is: endless press exposure. I almost wrote ‘priceless exposure’ but then these are companies with very deep pockets; nevertheless the advertising value equivalent must be very high indeed. The coverage really illustrates the fact that these companies have achieved celebrity status. They are the Justin Beibers of the corporate world. Whatever they do, whether it is truly significant or not, is met with punditry and analysis about what it means. A good example is Google’s recent acquisition of British company DeepMind. In other words: large AI Company buys small AI Company. Large companies buy small companies all the time but mostly they don’t make prime time news. It’s the Beiberisation of the corporate world.

But the question is about robotics, and to address it in more detail I think we need to think about the giants separately.

Take Amazon. We think of Amazon as an Internet company, but the web is just its shop window. Behind that shop window is a huge logistics operation with giant warehouses – Amazon’s distribution centres, so no one should be at all surprised by their acquisition of brilliant warehouse automation company Kiva Systems. Amazon’s recent stunt with the ‘delivery drone’ was I think just that – a stunt. Great press. But I wouldn’t be at all surprised to see more acquisitions toward further automation of Amazon’s distribution and delivery chain.

Apple is equally unsurprising. They are a manufacturing company with a justifiable reputation for super high quality products. As an electronics engineer who started his career by taking wirelesses and gramophones apart as a boy, I’m fascinated by the tear-downs that invariably follow each new product release. It’s obvious that each new generation of Apple devices is harder to manufacture than the last. Precision products need precision manufacture and it is no surprise that Apple is investing heavily in the machines needed to make its products.

Google is perhaps the least obvious candidate to invest in robotics. You could of course take the view that a company with more money than God can make whatever acquisitions it likes without needing a reason – that these are vanity acquisitions. But I don’t think that’s the case. I think Google has its eyes on the long term. It is an Internet company and the undisputed ruler of the Internet of Information. But computers are no longer the only things connected to the Internet. Real world devices are increasingly networked – the so-called Internet of Things. I think Google doesn’t want to be usurped by a new super company that emerges as the Google of real-world stuff. It’s not quite sure how the transition to the future Internet of Everything will pan out, but figures that mobile robots – as well as smart environments – will feature heavily in that future. I think Google is right. I think it’s buying into robotics because it wants to be a leader and shape the future of the Internet of Everything.

Read more answers →

Soft robotics in space

Alan Winfield — Sat, 14 Dec 2013 13:00:00 +0000

Space robotics is understandably conservative. When the cost of putting a robot on a planet, moon or asteroid runs into the billions we need to be sure the technology will work. And with very long project lifetimes – spanning decades from engineering design to on-planet robot exploration – it’s a long hard road from the research lab to the real off-world use for new advances in robotics.

This context was very much in mind when I gave a talk on Advanced Robotics for Space at the Appleton Space Conference last week. I used this great opportunity to outline a few examples of new research directions in robotics for the European space community, and suggest how these could benefit future planetary robots. I had just 20 minutes, so I couldn’t do much more than show a few video clips. The four new directions I highlighted are:

Soft Robotics: soft actuation and soft sensing
Robots with Internal Models, for self-repair
Self-assembling swarm robots, for adaptive/evolvable morphology
Autonomous 3D collective robot construction

In this post I want to talk about just the first of these: soft robotics, and why I think we should seriously think about soft robotics in space. Soft robotics – as the name implies – is concerned with making robots soft and compliant. It’s a new discipline that already has its own journal, but not yet a wikipedia page. Soft robots would be soft on the inside as well as the outside – so even the fur-covered Paro robot is not a Soft robot. Soft robotics research is about developing new soft, smart materials for both actuation and sensing (ideally within the same material). Soft robots would have the huge advantage over conventional stiff metal and plastic robots, of being light and, well, soft. For robots designed to interact with humans that’s obviously a huge advantage because it makes the robot intrinsically much safer.

Soft robotics research is still at the exploratory stage, so there are not yet prefered materials and approaches. In our lab we are exploring several avenues, one is electroactive polymers (EAPs) for artificial muscles; another is the bio-mimetic 3D printed flexible artificial whisker. Another approach makes use of shape memory alloys to actuate octopus like limbs: here is a very nice YouTube movie from the EU OCTOPUS project. And perhaps one of the most unlikely but very promising approaches: exploiting fluid-solid phase changes in ground coffee to make a soft gripper: the Jaeger-Lipson coffee balloon gripper.

Let me elaborate a little more on the coffee balloon gripper. Based on the simple observation that when you buy vacuum-packed ground coffee the pack is completely solid, yet as soon as you cut open the pack and release the vacuum the ground coffee returns to its flowing fluid state. Heinrich Jaeger, Hod Lipson and co-workers put ground coffee into a latex balloon then, by controlling the vacuum via a pump, they demonstrate a gripper able to safely pick up and hold more or less any object. Here is a YouTube video showing this remarkable ability.

Almost any planetary exploration robot is likely to need a gripper to pick up or collect rock samples for analysis or collection (for return to Earth). Conventional robot grippers are complex mechanical devices that need very precise control in order to reliably pick up irregularly shaped and sized objects. That control is mechanically and computationally expensive, and problematical because of time delays if it has to be performed remotely from Earth. Something like the Jaeger-Lipson coffee balloon gripper would – I think – provide a much better solution. This soft gripper avoids the hard control and computation because the soft material adapts itself to the thing it is gripping; it’s a great example of what we call morphological computation.

The second example I suggested is inspired by work in our lab on bio-inspired touch sensing. Colleagues have developed a device called TACTIP – a soft flexible touch sensor that provides robots (or robot fingers) with very sensitive touch sensing capable of sensing both shape and texture. Importantly the sensing is done inside TACTIP, so the outside surface of the sensor can sustain damage without loss of sensing. Here is a very nice YouTube report on the TACTIP project.

It’s easy to see that giving planetary robots touch sensing could be useful, but there’s another possibility I outlined: the potential to allow Earth scientists to feel what the robot’s sensor is feeling. PhD student Callum Roke and his co-workers developed a system based on TACTIP for what we call remote tele-haptics. Here is a video clip demonstrating the idea:

Imagine being able to run your fingers across the surface of Mars, or directly feel the texture of a piece of asteroid rock without actually being there.

Noisy imitation speeds up group learning

Alan Winfield — Tue, 03 Dec 2013 15:05:00 +0000

Broadly speaking there are two kinds of learning: individual learning and social learning. Individual learning means learning something entirely on your own, without reference to anyone else who might have learned the same thing before. The flip side of individual learning is social learning, which means learning from someone else. We humans are pretty good at both individual and social learning although we very rarely have to truly work something out from first principles. Most of what we learn, we learn from teachers, parents, grandparents and countless others. We learn everything from how to make chicken soup to flying an aeroplane from watching others who already know the recipe (or wrote it down), or have mastered the skill. For modern humans I reckon it’s pretty hard to think of anything we have truly learned, on our own; maybe learning to control our own bodies as babies, leading to crawling and walking are candidates for individual learning (although as babies we are surrounded by others who already know how to walk – would we walk at all if everyone else got around on all fours?). Learning to ride a bicycle is perhaps also one of those things no-one can really teach you – although it would be interesting to compare someone who has never seen a bicycle, or anyone riding one, in their lives with those (most of us) who see others riding bicycles long before climbing on one ourselves.

In robotics we are very interested in both kinds of learning, and methods for programming robots that can learn are well known. A method for individual learning is called reinforcement learning (RL). It’s a laborious process in which the robot tries out lots and lots of actions and gets feedback on whether each action helps or hinders the robot in getting closer to its goal – actions that help/hinder are re/de-inforced so the robot is more/less likely to try them again; it’s a bit like shouting “warm, hot, cold, colder…” in a hide-and-seek game. It’s fair to say that RL in robotics is pretty slow; robots are not good individual learners, but that’s because, in general, they have no prior knowledge. As a fair comparison think of how long it would take you to learn how to make fire from first principles if you had no idea that getting something hot may, if you have the right materials and are persistent, create fire, or that rubbing things together can make them hot. Roboticists are also very interested in developing robots that can learn socially, especially by imitation. Robots that you can program by showing them what to do (called programming by demonstration) clearly have a big advantage over robots that have to be explicitly programmed for each new skill.

Within the artificial culture project PhD student (now Dr) Mehmet Erbas developed a new way of combining social learning by imitation and individual reinforcement learning, and the paper setting out the method together with results from simulation and real robots has been published in the journal Adaptive Behavior. Let me explain the experiments with real robots, and what we have learned from them.

Here’s our experiment. We have two robots – called e-pucks. The inset shows a closeup. Each robot has its own compartment and must – using individual (reinforcement) learning – learn how to navigate from the top right hand corner, to the bottom left hand corner of its compartment. Learning this way is slow, taking hours. But in this experiment the robots also have the ability to learn socially, by watching each other. Every so often one of the robots will stop its individual learning and drive itself out of its own compartment, to the small opening at the bottom left of the other compartment. There it will stop and simply watch the other robot while it is learning, for a few minutes. Using a movement imitation algorithm the watching robot will (socially) learn a fragment of what the other robot is doing, then combine this knowledge into what it is individually learning. The robot then runs back to its own compartment and resumes its individual learning. We call the combination of social and individual learning ‘imitation-enhanced learning’.

In order to test the effectiveness of our new imitation-enhanced learning algorithm we first run the experiment with the imitation turned off, so the robots learn only individually. This gives us a baseline for comparison. We then run two experiments with imitation-enhanced learning. In the first we wait until one robot has completed its individual learning, so it is an ‘expert’; the other robot then learns – using its combination of individual learning and social learning from the expert. Not surprisingly, learning this way is faster.

This graph shows individual learning only as the solid black line, and imitation-enhanced learning from an expert as the dashed line. In both cases learning is more or less complete when the graphs transition from vertical to horizontal. We see that individual learning takes around 360 minutes (6 hours). With the benefit of an expert to watch, learning time drops to around 60 minutes.

The second experiment is even more interesting. Here we start the two robots at the same time, so that both are equally inexpert. Now you might think it wouldn’t help at all, but remarkably each robot learns faster when it can observe, from time to time, the other inexpert robot, than when learning entirely on its own. As the graph below shows, the speedup isn’t as dramatic – but imitation enhanced learning is still faster.

Think of it this way. It’s like two novice cooks, neither of whom knows how to make chicken soup. Each is trying to figure it out by trial and error but, from time to time, they can watch each other. Even though it’s pretty likely that each will copy some things that lead to worse chicken soup, on average and over time, each hapless cook will learn how to make chicken soup a bit faster than if they were learning entirely alone.

In the paper we analyse what’s going on when one robot imitates part of the semi-learned sequence of moves by the other. And here we see something completely unexpected. Because the robots imitate each other imperfectly – when one robot watches another and then tries to copy what it saw, the copy will not be perfect – from time to time, one inexpert robot will miscopy the other inexpert robot and the miscopy, by chance, helps it to learn. To use the chicken soup analogy: it’s as if you are spying on the other cook – try to copy what they’re doing but get it wrong and, by accident, end up with better chicken soup.

This is deeply interesting because it suggests that when we learn in groups, making mistakes – noisy social learning – can actually speed up learning for each individual and for the group as a whole.

Full Reference
Mehmet D Erbas, Alan FT Winfield, and Larry Bull (2013), Embodied imitation-enhanced reinforcement learning in multi-agent systems, Adaptive Behavior. Published online 29 August 2013. Download pdf (final draft)

Ethical robots: Some technical and ethical challenges

Alan Winfield — Tue, 19 Nov 2013 17:26:00 +0000

I’ve been talking about robot ethics for several years now, but that’s mostly been about how we roboticists must be responsible and mindful of the societal impact of our creations. Two years ago I wrote – in my Very Short Introduction to Robotics – that robots cannot be ethical. Since then I’ve completely changed my mind*. I now think there is a way of making a robot that is at least minimally ethical. It’s a huge technical challenge which, in turn, raises new ethical questions. For instance: if we can build ethical robots, should we? Must we..? Would we have an ethical duty to do so? After all, the alternative would be to build amoral robots. Or, would building ethical robots create a new set of ethical problems? An ethical Pandora’s box.

Here are the slides of my keynote at last week’s excellent EUCog meeting: Social and Ethical Aspects of Cognitive Systems. And the talk itself is here, on YouTube.

The talk was in three parts.

Part 1: here I outline why and how roboticists must be ethical. This is essentially a recap of previous talks. I start with the societal context: the frustrating reality that even when we meet to discuss robot ethics this can be misinterpreted as scientists fear a revolt of killer robots. This kind of media reaction is just one part of three linked expectation gaps, in what I characterise as a crisis of expectations. I then outline a few ethical problems in robotics – just as examples. Here I argue it’s important to link safe and ethical behaviour – something that I return to later. Then I recap the five draft principles of robotics.

Part 2: here I ask the question: what if we could make ethical robots? I outline new thinking which brings together the idea of robots with internal models, with Dennett’s Tower of Generate and Test, as a way of making robots that can predict the consequences of their own actions. I then outline a generic control architecture for robot safety, even in unpredictable environments. The important thing about this approach is that the robot can generate next possible actions, test them in its internal model, and evaluate the safety consequences of each possible action. The unsafe actions are then inhibited – and the robot controller determines which of the remaining safe actions is chosen, using its usual action-selection mechanism. Then I argue that it is surprisingly easy to extend this architecture for ethical behaviour, to allow the robot to predict the robot actions that would minimise harm for a human in its environment. This appears to represent an implementation of Asimov’s 1st and 3rd laws. I outline the significant technical challenges that would need to be overcome to make this work.

But, assuming such a robot could be built, how ethical would it be? I suggest that with a subset of Asimovian ethics it probably wouldn’t satisfy an ethicist or moral philosopher. But, nevertheless – I argue there’s a good chance that such a minimally ethical robot could help to increase trust, in the robot, from its users.

Part 3: in the final part of the talk I conclude with some ethical questions. The first is: if we could build an ethical robot, are we ethically compelled to do so? Some argue that we have an ethical duty to try and build moral machines. I agree. But the counter argument, my second ethical question, is are there ethical hazards? Are we opening a kind of ethical Pandora’s box, by building robots that might have an implicit claim to rights, or responsibilities. I don’t mean that such a robot would ask for rights, but instead that, because it is has some moral agency, then we might think it should be accorded rights. I conclude that we should try and build ethical robots. The benefits I think far outweigh any ethical hazards, which in any event can, I think, be minimised.

*It was not so much an epiphany, as a slow conversion from sceptic to believer. I have long term collaborator Michael Fisher to thank for doggedly arguing with me that it was worth thinking deeply about how to build ethical robots.

A close(ish) encounter with Voyager 2

Alan Winfield — Sun, 27 Oct 2013 14:56:00 +0000

It is summer 1985. I’m visiting Caltech with colleague and PhD supervisor Rod Goodman. Rod has just been appointed in the Electrical Engineering Department at Caltech, and I’m still on a high from finishing my PhD in Information Theory. Exciting times.

Rod and I are invited to visit the Jet Propulsion Labs (JPL). It’s my second visit to JPL. But it turned into probably the most inspirational afternoon of my life. Let me explain.After the tour the good folks who were showing us round asked if I would like to meet some of the post-docs in the lab. As he put it: the fancy control room with the big wall screens is really for the senators and congressmen – this is where the real work gets done. So, while Rod went off to discuss stuff with his new Faculty colleagues I spent a couple of hours in a back room lab, with a Caltech post-doc working on – as he put it – a summer project. I’m ashamed to say I don’t recall his name so I’ll call him Josh. Very nice guy, a real southern californian dude.

Now, at this point, I should explain that there was a real buzz at JPL. Voyager 2, which had already more than met its mission objectives was now on course to Uranus and due to arrive in January 1986. It was clear that there was a significant amount of work in planning for that event. The first ever opportunity to take a close look at the seventh planet.

So, Josh is sitting at a bench and in front of him is a well-used Apple II computer. And behind the Apple II is a small display screen so old that the phosphor is burned. This used to happen with CRT computer screens – it’s the reason screen savers were invented. Beside the computer are notebooks and manuals, including prominently a piece of graph paper with a half-completed plot. Josh then starts to explain: one of the cameras on Voyager 2 has (they think) a tiny piece of grit* in the camera turntable – the mechanism that allows the camera to be panned. This space grit means that the turntable is not moving as freely as it should. It’s obviously extremely important that when Voyager gets to Uranus they need to be able to point the cameras accurately, so Josh’s project is to figure out how much torque is (now) needed to move the camera turntable to any desired position. In other words: re-calibrate the camera’s controller.

At this point I stop Josh. Let me get this straight: there’s a spacecraft further from earth, and flying faster, than any manmade object ever, and your summer project is to do experiments with one of its cameras, using your Apple II computer. Josh: yea, that’s right.

Josh then explains the process. He constructs a data packet on his Apple II, containing the control commands to address the camera’s turntable motor and to instruct the motor to drive the turntable. As soon as he’s happy that the data packet is correct, he then sends it – via theRS232 connection at the back of his Apple II – to a JPL computer (which, I guess would be a mainframe). That computer then, in turn, puts Josh’s data packet together with others, from other engineers and scientists also working on Voyager 2, after – I assume – carefully validating the correctness of these commands. Then the composite data packet is sent to theDeep Space Network (DSN) to be transmitted, via one of the DSNs big radio telescopes, to Voyager 2.

Then, some time later, the same data packet is received by Voyager 2, decoded and de-constructed and said camera turntable moves a little bit. The camera then sends back to Earth, again via a composite data packet, some feedback from the camera – the number of degrees the turntable moved. So a day or two later, via a mind-bogglingly complex process involving several radio telescopes and some very heavy duty error-correcting codes, the camera-turntable feedback arrives back at Josh’s desktop Apple II with the burned-phosphor screen. This is where the graph paper comes in. Josh picks up his pencil and plots another point on his camera-turntable calibration graph. He then repeats the process until the graph is complete. It clearly worked because six months later Voyager 2 produced remarkable images of Uranus and its moons.

This was, without doubt, the most fantastic lab experiment I’d ever seen. From his humble Apple II in Pasadena Josh was doing tests on a camera rig, on a spacecraft, about 1.7 billion miles away. For a Thunderbirds kid, I really was living in the future. And being a space-nerd I already had some idea of the engineering involved in NASA’s deep space missions, but that afternoon in 1985 really brought home to me the extraordinary systems engineering that made these missions possible. Given the very long project lifetimes – Voyager 2 was designed in the early 1970s, launched in 1977, and is still returning valuable science today – its engineers had to design for the long haul; missions that would extend over several generations. Systems design like this requires genius, farsightedness and technical risk taking. Engineering that still inspires me today.

*it later transpired that the problem was depleted lubricant, not space grit.

Don’t build robots, build robot systems

Alan Winfield — Mon, 07 Oct 2013 11:44:00 +0000

Image courtesy of Wikimedia Commons.

Why aren’t there more intelligent mobile robots in real world applications? It’s a good question, and one I’m often asked. The answer I give most often is that it’s because we’re still looking for that game changing killer app – the robotics equivalent of the spreadsheet for PCs. Sometimes I place the blame on a not-quite-yet-solved technical deficit – like poor sensing, or sensor fusion, or embedded AI; in other words, our intelligent robots are not yet smart enough. Or I might cite a not-fully-developed-capability, like robots not able to cope with unpredictable (i.e. human) environments, or we can’t yet assure that our robots are safe, and dependable.

Last week at euRathlon 2013 I realised that these answers are all wrong. Actually that would be giving myself credit where none is due. The answer to the question: why aren’t there more intelligent mobile robots in real world applications was pointed out by several of the presenters at the euRathlon 2013 workshop, but most notably by our keynote speaker Shinji Kawatsuma, from the Japan Atomic Energy Authority (JAEA). In an outstanding talk Kawatsuma explained, with disarming frankness, that, although his team had robots they were poorly prepared to use those robots in the Fukushima Daiichi NPP, because the systems for deployment were not in place. The robots are not enough. Just as important are procedures and protocols for robot deployment in an emergency; mobile infrastructure, including vehicles to bring the robots to the emergency, which are capable – as he vividly explained – of negotiating a road system choked with debris (from the Tsunami) and strained with other traffic (rescue workers and evacuees); integration with other emergency services; and, above all, robot operators trained, practised and confident to guide the robots through whatever hazards they will face in the epicentre of the disaster.

In summing up the lessons learned from robots at Fukushima, Shinji Kawatsuma offered this advice – actually it was more of a heartfelt plea: don’t build robots, build robot systems. And, he stressed, those systems must include operator training programmes. It was a powerful message for all of us at the workshop. Intelligent robots are endlessly fascinating machines, with all kinds of difficult design challenges, so it’s not surprising that our attention is focussed on the robots themselves. But we need to understand that real world robots are like movie stars – who (despite what they might think) wouldn’t be movie stars at all without the supporting cast, camera and sound crews, writers, composers, special effects people and countless other departments that make the film industry. Take the Mars rover Curiosity – an A-list movie star of robotics. Curiosity could not do its job without an extraordinary supporting infrastructure that, firstly, delivered her safely to the surface of Mars and, secondly, allows Curiosity’s operators to direct her planetary science exploration.

Curiosity: an A-list movie star robot (NASA/JPL-Caltech/MSSS), with a huge supporting cast of science and technology.

So, to return to my question why aren’t there more intelligent mobile robots in real world applications. The answer is plain. It’s because without supporting systems: infrastructure and skilled operators integrated and designed to meet the real world need, a robot – regardless of how innovative and intelligent it is – will never make the transition from the lab to the real world. Without those systems that robot will remain no more than a talented but undiscovered actor.

Hans-Arthur Marsiske, The use of robots in Fukushima: Shinji Kawatsuma Interview, Heise online, 25 September 2013 (in German).
K Nagatani, S Kiribayashi, Y Okada, K Otake, K Yoshida, S Tadokoro, T Nishimura, T Yoshida, E Koyanagi, M Fukushima and S Kawatsuma, Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots, Journal of Field Robotics, 30 (1), 44-63, 2013.

The triangle of life: Evolving robots in real-time and real-space

Alan Winfield — Fri, 20 Sep 2013 13:28:00 +0000

At the excellent European Conference on Artificial Life (ECAL) a couple of weeks ago we presented a paper called The Triangle of Life: Evolving Robots in Real-time and Real-space (this links to the paper in the online proceedings).

As the presenting co-author I gave a one-slide one-minute pitch for the work, and here is that slide.

The paper proposes a new conceptual framework for evolving robots, that we call the Triangle of Life. Let me outline what this means. But first a quick intro to evolutionary robotics. In my very short introduction to Robotics I wrote:

One of the most fascinating developments in robotics research in the last 20 years is evolutionary robotics. Evolutionary robotics is a new way of designing robots. It uses an automated process based on Darwinian artificial selection to create new robot designs. Selective breeding, as practised in human agriculture to create new improved varieties of crops, or farm animals, is (at least for now) impossible for real robots. Instead, evolutionary robotics makes use of an abstract version of artificial selection in which most of the process occurs within a computer. This abstract process is called a genetic algorithm. In evolutionary robotics we represent the robot that we want to evolve, with an artificial genome. Rather like DNA, our artificial genome contains a sequence of symbols but, unlike DNA, each symbol represents (or ‘codes for’) some part of the robot. In evolutionary robotics we rarely evolve every single part of a robot.

A robot consists of a physical body with an embedded control system – normally a microprocessor running control software. Without that control software the robot just wouldn’t do anything – it would be the robot equivalent of a physical body without a mind. In biological evolution bodies and minds co-evolved (although the dynamics of that co-evolutionary process are complex and interesting). But in 20 years or so of evolutionary robotics the vast majority of work has focussed only on evolving the robot’s controller. In other words we take a pre-designed robot body, then use the genetic algorithm to discover a good controller for that particular body. There has been little work on body-brain co-evolution, and even less work on evolving real robot bodies. In fact, we can count the number of projects that have evolved new physical robot bodies on the fingers of one hand*. Here is one of those very rare projects: the remarkable Golem project of Hod Lipson and Jordan Pollack.

This is surprising. When we think of biological evolution and the origin of species, our first thoughts are of the evolution and diversity of body shapes and structures. In the same way, the thing about a robot that immediately captures our attention is its physical body. And bodies are not just vessels for minds. As Rolf Pfeifer and Josh Bongard explain in their terrific book How the Body Shapes the Way We Think, minds depend crucially on bodies. The old dogma of Artificial Intelligence, that we can simply design an artificial brain without any regard to its embodiment, is wrong. True artificial intelligence will only be achieved by co-evolving physical bodies with their artificial minds.

In this paper we are arguing for a radical new approach in which the whole process of co-evolving robot bodies and their controllers takes place in real space and real time. And, as the title makes clear, we are also advocating a open-ended cycle of artificial life, in which every part of the robots’ artificial life cycle takes place in real space and real time, from artificial conception, through to artificial birth, artificial infancy and development, then artificial maturity and mating. Of course these words are metaphors: the artificial processes are at best a crude analogue. But let me stress that no-one has demonstrated this. The examples that we give in the paper, from the EU Symbrion project, are just fragments of the process – not joined up in reality. And the Symbrion example is very constrained because of the modular robotics approach which means that the building blocks of these ‘multi-cellular’ robot organisms – the ‘cells’ – are themselves quite chunky robots; we have only 3 cell types and only a handful of cells for evolution to work with. Evolving robots in real space and real time is ferociously hard but, as the paper concludes: Our proposed artificial life system could be used to investigate novel evolutionary processes, not so much to model biological evolution – life as it is, but instead to study life as it could be.

Full reference:

Eiben AE, Bredeche N, Hoogendoorn M, Stradner J, Timmis J, Tyrrell A, and Winfield A (2013), The Triangle of Life: Evolving Robots in Real-time and Real-space, pp 1056-1063 in Advances in Artificial Life, ECAL 2013, proc. Twelfth European Conference on the Synthesis and Simulation of Living Systems, eds. Liò P, Miglino O, Nicosia G, Nolfi S and Pavone M, MIT Press.

*I was surprised to discover this when searching the literature for a new book chapter in co-authoring on Evolvable Robot Hardware.

Walterian Creatures

Memories of Skyrim

Alan Winfield — Mon, 26 Aug 2013 13:45:00 +0000

From time to time I like to visit Skyrim. I’ve been going there for about 2 years – completed the main quest a year or so ago, and since then go back to undertake a side quest or, more often that not, just wander around admiring the scenery. Of course my character is now reasonably levelled-up so wandering around is not quite as perilous as it used to be; interesting rather than terrifying.

But the thing I’ve noticed in recent months is that I have memories of being in places in Skyrim that, subjectively, feel completely indistinguishable from memories of being in real places in the real world. In other words the quality and character of the memory – the sense of having really been there – is no different when I recall, say, standing on the porch of Dragonsreach Hall looking South toward Throat of the World mountain, as when I remember looking down toward the Cumberland basin from the south footpath of the Clifton suspension bridge.

For me this is a new experience. I’ve been playing (and sometimes coding) video games since that meant batting a pixelated ball from one side of a low-res monochrome screen to the other on home built 8-bit micros in the 1980s. I have fond memories of playing the first generation Alone in the Dark ~1993 on a 386 PC, with my 5 year old son – but my memories are of the experience of playing the game with Tom, not of actually being in that haunted house. More recent games on the Xbox 360, with graphics I would have had difficulty imagining 20 years ago, have not had the same effect of creating such compellingly real memories for me.

So what is it about Skyrim that is making these memories feel so real? I think there are several factors. The first is that the scenery is so breathtakingly beautiful, which means you really do want to just stop and stare for awhile. Second, and equally important I think, is that this is not some imagined alien landscape. It is decidedly Earth – a cold Northerly Earth certainly, but the fells and mountains, the lakes and forests, the grasses and especially trees, are realised so accurately you can identify whether it’s a birch or an oak. Third, the landscape is in constant motion – so the grasses sway in the wind, the brooks gurgle and splash and insects flutter. Wait a little longer and you realise the day is passing from afternoon to dusk, the sky turns golden in the sunset, then to night. A star spangled and moon-crossed night sky then rewards the patient (and the brave – this is a wild and dangerous place), following by a glorious sunrise. And there is weather too. Rain, which splashes delightfully on the lake, occasional thunderstorms (learn the power and you can call them up!), and snow blizzarding in the mountains. In case you can’t go there yourself watch this YouTube movie: Skyrim – landscapes and scenery.

All of these are I think important cues in making the experience, and hence the memory, feel so real. But I think there is another factor, which is that my journeying through Skyrim is part of a narrative which connects places with events, quests and discoveries. So the memorable places are those I have arrived at following some perilous and occasionally epic trail, with multiple trials on the journey, Or they are places I have discovered offer safety and refuge; places to return to after days questing in the wilderness. Perhaps the depth and intensity of the experienced narrative somehow makes up for the limitations of the sensory experience? With only 2D vision on a flat TV screen and stereo audio, and nothing at all to stimulate the rest of the considerable human sensorium, it seems incredible that such a weakly immersive experience – compared to being in real places – can create subjectively comparable memories.

Since thinking about these memories of Skyrim, I’ve wondered if these are technically false memories. Am I experiencing so called False Memory Syndrome? Although FSM is controversial what is beyond doubt is the extraordinary suggestibility and unreliability of human memory. I was astonished by the unreliability of memory I witnessed at first hand two years ago while on Jury service. But false memories are memories of events that never happened, but are strongly believed. On reflection I think my memories of Skyrim do not fall into this category. The events and places occurred in the virtual rather than the real, but they, and my experience of them, really happened.

So if immersive video games technology has reached the point that it can create, for the gamer, memories of places and events which feel no different to memories of places and events in the real world, is this a bad thing? And, as the technology improves to make the experience more immersive, involving more senses, will we find ourselves unable to distinguish between the virtual and the real – confusing memories of one with the other? I think the answer is no. After all we each create a personal narrative – the remembered story of our lives – and I don’t think it matters whether the events that make up that story happen in the virtual or the real. I think we’re just as able to recall the difference between a trip to Rhyll and Rome, as between Skegness and Skyrim. And if, with advancing years or just because that’s the way we are, we start to confuse these memories I don’t think we’re any more likely to confuse the virtual and the real, as we are the real and the real. Nor do I think the degree of immersion in the virtual matters as far as memories of being there is concerned*. After all, being in the real world is a fully immersive experience. Even if, and when, we can climb into a full-body immersive gaming rig, like those of Ernest Cline’s brilliant Ready Player One, we will still only have an experience equal to that of the real world. So why should those experiences be remembered any differently to those in the real?

Ok. I’ve persuaded myself there isn’t a problem. Time for another trip to Skyrim.

*There is of course another quite different concern – to do with how much more addictive the experience will become.Will we neglect the real – and ourselves, like Larry Niven’s wireheads?

The scourge of the RoboTroll is already upon us

Alan Winfield — Thu, 22 Aug 2013 00:08:00 +0000

When a robot ethics working group met nearly three years ago one of the things we fretted about was privacy. We were concerned especially about personal companion robots. Whatever their purpose, be it healthcare for the elderly or disabled, or childcare, or simply Robot and Frank -style companionship, we debated the inevitable privacy issues of this kind of robot. Two aspects directly impact privacy, since personal companion robots are likely to (1) collect and store data and (2) be networked.

We also worried about hacking. The third of our Principles of robotics states: Robots are products: as with other products, they should be designed to be safe and secure. And the commentary for this principle has this to say about hacking:

We are aware that the public knows that software and computers can be “hacked” by outsiders, and processes need to be developed to show that robots are secure as far as possible from such attacks.

It seems our concerns were well founded. Last week a report appeared of a networked baby monitor that was apparently hacked. It was pretty distressing. The hacker was shouting abuse at the baby, chillingly using her name – it seems that he (let’s assume it was a he) was able to gain access to the baby monitor’s video feed and read the baby’s name displayed above her bed. Even more chilling (adding to the parent’s horror) the execrable hacker then turned the camera to look at them when they entered their child’s room to find out what was going on.

A WiFi IP Camera, aka Baby Monitor is, I contend, a teleoperated robot. The thing that makes it a robot is the motorised pan and tilt mechanism for steering the camera. So, despite password protection, Mr Gilbert’s networked robot was hacked. This particular robot, and I hazard tens of thousands like it, is absolutely not secure from attack.

The consequences of this particular attack were, fortunately, not much more serious than giving the Gilbert’s a fright they surely won’t forget for some time. But for me one particularly egregious aspect of this robot hack – something that the robot ethics working group did not anticipate – was the verbal abuse hurled at baby Allyson and her parents. It is with profound dismay that I ask the question: is the first case of RoboTrolling..?

When robots start telling each other stories…

Alan Winfield — Wed, 31 Jul 2013 06:33:00 +0000

About 6 years ago the late amazing Richard Gregory said to me, with a twinkle in his eye, “when your robots start telling each other stories, then you’ll really be onto something”. It was a remark with much deeper significance than I realised at the time.

Richard planted a seed that’s been growing since. What I didn’t fully appreciate then, but do now, is the profound importance of narrative. More than we perhaps imagine. Narrative is, I suspect, a fundamental property of both human societies and individual human beings. It may even be a universal property of all advanced societies of sentient social beings.

Let me try and justify this outlandish claim. First, take human societies. We humans love to tell each other stories. Whether our stories are epic poems, love songs; stories told with sound (music), or movement (dance), or with stuff (sculpture or art). Stories about what we did today, or on our holidays, stories made with images (photos, or movies); true stories or fantasies, or stories about the Universe that strive to be true (science), or very formal abstract stories told with mathematics, stories are everywhere. Arguably, most of human culture consists of stories.

Since humans started remembering stories and passing them on orally, and more recently with writing, we have had history: the more-or-less-true grand stories of human civilisation. Even the many artefacts of our civilisation are kinds of stories. They are embodied stories, which narrate the process by which they were designed and made; the plans and drawings which we use to formally record those designs are literally stories which tell how to arrange and join materials in space to fashion the artefact. Project plans are narratives of a different kind: they tell the story of the future steps that must be taken to achieve a goal. Computer programs are stories too. Except that they contain multiple narratives (bifurcated with branches and reiterated with loops), whose paths are determined by input data, which are related over and over at blinding speed within the computer.

Now consider individual humans. There is a persuasive view in psychology that each of us owes our identity, our sense of self, to our personal life stories. The physical stuff that makes us, the cells of our body, are regenerated and replaced continuously, so that there’s very little of you that existed 5 years ago. (I just realised the fillings in my teeth are probably the oldest part of me!) Yet you are still you. You feel like the same you 10, 20 or in my case 50 years ago – since I first became self-aware. I think that it’s the lived and remembered personal narrative of our lives that provides us with the feeling, the illusion if you like, of a persistent self. This is I think why degenerative brain diseases are so terrifying. They appear to eat away that personal narrative so devastatingly that the person is ultimately lost, even while their physical body continues living.

I was tremendously excited to be invited to a cross-disciplinary workshop on Narrative and Complex Systems at the York Centre for Complex Systems Analysis, co-organised by York Professors of English Richard Walsh, and Computer Science Susan Stepney. For the first time I found myself in a forum in which I could share and debate ideas on narrative.

In preparing for the workshop I realised that perhaps the idea of robots telling each other stories isn’t as far fetched as it first appears. Think about a simple robot, like the e-puck. What does the story of its life consist of? Well, it is the complete history of all of the movements, including turns, etc, punctuated by interactions with its environment. Because the robot and its set of behaviours is simple, then those interactions are pretty simple too. It occurred to me that it is perfectly possible for a robot to remember everything that has ever happened to it. Now place a number of these robots together, in a simple ‘society’ of robots, and provide them with the mechanism to exchange ‘life stories’ (or more likely, fragments of life stories). This mechanism is something we already developed in the Artificial Culture project – it is social learning by imitation. These robots would be telling each other stories.

But, I hear you ask, would these stories have any meaning? Well, to start with I think we must abandon the notion that they would necessarily mean anything to us humans. After all, these are robots telling each other stories. OK, so would the stories mean anything to the robots themselves, especially robots with limited ‘cognition’? Now we are in the interesting territory of semiotics, or – to be more accurate – robosemiotics. What, for instance, would one robot’s story signify to another? That signification would I think be the meaning. But I think to go any further we would need to do the robot experiment I have outlined here.

And what would be the point of my proposed robot experiment? It is, I suggest, this:

To explore, with an abstract but embodied model, the relationship between the narrative self and shared narrative, i.e. culture.

By doing this experiment would we be, as Richard Gregory suggested, really onto something?

[This post originally appeared on Alan Winfield’s blog on July 24, 2012.]

Robotics has a new kind of Cartesian Dualism, and it’s just as unhelpful

Alan Winfield — Mon, 22 Jul 2013 13:18:00 +0000

I believe robotics has re-invented mind-body dualism.

At the excellent European Robotics Forum earlier this year, I attended a workshop called AI meets Robotics. The thinking behind the workshop was:

The fields of Artificial Intelligence (AI) and Robotics were strongly connected in the early days of AI, but became mostly disconnected later on. While there are several attempts at tackling them together, these attempts remain isolated points in a landscape whose overall structure and extent is not clear. Recently, it was suggested that even the otherwise successful EC program “Cognitive systems and robotics” was not entirely effective in putting together the two sides of cognitive systems and of robotics.

I couldn’t agree more. Actually I would go further and suggest that robotics has a much bigger problem than we think. It’s a new kind of dualism which parallels Cartesian brain-mind dualism, except in robotics, it’s hardware-software dualism. And like Cartesian dualism it could prove just as unhelpful, both conceptually, and practically – in our quest to build intelligent robots.

While sitting in the workshop last week I realised rather sheepishly that I’m guilty of the same kind of dualistic thinking. In my Introduction to Robotics one of the (three) ways I define a robot is: an embodied Artificial Intelligence. And I go on to explain:

…a robot is an Artificial Intelligence (AI) with a physical body. The AI is the thing that provides the robot with its purposefulness of action, its cognition; without the AI the robot would just be a useless mechanical shell. A robot’s body is made of mechanical and electronic parts, including a microcomputer, and the AI made by the software running in the microcomputer. The robot analogue of mind/body is software/hardware. A robot’s software – its programming – is the thing that determines how intelligently it behaves, or whether it behaves at all.

But, as I said in the workshop, we must stop thinking of cognitive robots as either “a robot body with added AI”, or “an AI with added motors and sensors”. Instead we need a new kind of holistic approach that explicitly seeks to avoid this lazy “with added” thinking.

[This post originally appeared on Alan Winfield’s blog on March 24, 2013.]

euRathlon and the DARPA Robot Challenge: A difference of approach

Alan Winfield — Mon, 22 Jul 2013 04:46:00 +0000

A week ago the DARPA Robotics Challenge unveiled the ATLAS humanoid robot, which will be used by seven competing teams. Developed by Boston Dynamics, ATLAS is an imposing 1.8m 150Kg bipedal humanoid robot, powered via a tethered cable. Another six teams have designed their own robots, and interestingly five of these are humanoid, and one a four-limbed simian-inspired robot.

In the euRathlon project we are taking a different approach in that we don’t expect, or require, the competing robots to be humanoid or zoomorphic. None of the euRathlon competition scenarios demand a humanoid robot to, for example, be able to step inside a vehicle and drive it. However, for the land robots at least, there is nothing stopping euRathlon teams from bringing humanoid robots to the competition.

As I wrote when we launched euRathlon early this year, the big vision of euRathlon is a competition scenario in which no single type of robot is, on its own, sufficient. Inspired by the Fukushima accident of March 2011, the 2015 euRathlon competition will require teams of land, sea and flying robots to autonomously cooperate, survey the scene, identify critical hazards and undertake tasks to make the plant safe. Leading up to this grand challenge in 2015, will be related and preparatory land and underwater robot competitions in 2013 and 2014, respectively.

The difference of our approach is not the result of an in-principle decision. Rather, it flows naturally from several factors. First, we are specifically creating competition scenarios that require cooperating teams across the three domains of land, sea and air. Second, we are looking for very high levels of autonomy, so the robot teams will, ideally, complete their mission with hands-off human monitoring only. Any human interventions will be penalised in the euRathlon scoring schema. And third, we are not looking to push innovation in the robot platforms themselves, but rather in their cognition, autonomy and system level team working. Thus, euRathlon teams who make use of existing and proven robot hardware will gain a big advantage in that they can focus all of their efforts on the software, communications and systems engineering; the AI and the autonomy. And by autonomy we mean both control and energy autonomy. The euRathlon competition scenarios preclude the use of tethered power connections, so robots must carry their own energy supplies sufficient to last the whole mission.

For these reasons the euRathlon robots are likely to look rather conventional: wheeled or tracked land robots; fixed or rotary wing (i.e. quadcopter) flying robots, and ROV-type underwater robots. Not as dramatic as the DARPA robot challenge humanoid or animal-like robots perhaps, but looks can be deceptive: the real innovation in the euRathlon robots will be in the autonomous cooperation across the three domains. Something that has not been demonstrated in realistic outdoor disaster response scenarios.

Of course there is nothing to stop euRathlon teams from using a bio-mimetic approach, so fish-like underwater robot cooperate with bird-like flying robots, and legged animal-like land robots. That really would be something!

Related blog posts:
euRathlon is go! (Feb 2013)
Real-world robotics reality check (May 2010)
A truly Grand Challenge (August 2007)

New Robotics and new opportunities

Alan Winfield — Tue, 28 May 2013 21:26:00 +0000

Here are the slides of my talk at the BARA Academic Forum for Robotics meeting Robotics: from innovation to service, on Monday 20 May 2013:

The key messages from my talk were:

The new wave of robotics represents a kind of Cambrian explosion in robotics: an exciting but also bewildering exploration of new forms, functions and materials. This explosion of diversity means that the New Robotics is not one kind of robot. Thus any kind of prediction about which of these will successfully evolve to become mainstream is more or less impossible.

There are two common myths: first, the waiting-for-AI myth: the idea that robotics is waiting for some breakthrough innovation in Artificial Intelligence, without which robotics is stuck. And second, the need-full-autonomy myth: the idea that fully autonomous robots represent some ideal end-state of the development of robotics; this is not true – instead we need robots and human-robot interfaces that will transition smoothly between tele-operation and semi-autonomy. We call this dynamic autonomy.

There are significant opportunities for innovation right now – underpinned by a significant head-of-steam of fundamental technologies from university R&D. I offer some examples for discussion, including companion robots, wearable robots and tele-operated robots with immersive tele-presence, perhaps making use of remote tele-haptics (although I claim no special insights).

We need new and agile approaches to innovation. New kinds of research-industry partnerships and flexible, responsive pathways to commercialisation. Especially campus start-ups and incubators, nurturing post-docs as next generation entrepreneurs; and innovative modes of funding. We also need responsible and sustainable innovation.Haptocs

Here are links to further information, and video clips, on the projects and robots highlighted in the talk:

Slide 10: The Cooperative Human Robot Interaction Systems (CHRIS) project

Project web pages: www.chrisfp7.eu
YouTube movie showing CHRIS robots including BERT, from BackToThePlanetVideo

Slide 11: MOBISERV – An Integrated Intelligent Home Environment for the Provision of Health, Nutrition and Well-Being Services to Older Adults

Project web pages: www.mobiserv.info
YouTube movie showing MOBISERV concepts

Slide 12:Hand exoskeleton for post stroke recovery

Slide 13: Tactile Sensing – tele-haptics

Slide 14: Surgical Haptics

Slide 15: Search and Rescue – Disaster Response

Project web pages: www.eurathlon.eu

Slide 16: Towards energy sustainability

The BRL Bioenergy & Self Sustainable Systems theme

Alan Winfield on “What is the single biggest obstacle preventing robotics from going mainstream?”

Alan Winfield — Wed, 15 May 2013 12:48:55 +0000

Well it depends on what you mean by mainstream. For a number of major industry sectors robotics is already mainstream. In assembly-line automation, for instance; or undersea oil well maintenance and inspection. You could argue that robotics is well established as the technology of choice for planetary exploration. And in human culture too, robots are already decidedly mainstream. Make believe robots are everywhere, from toys and children’s cartoons, to TV ads and big budget Hollywood movies. Robots are so rooted in our cultural landscape that public attitudes are, I believe, informed – or rather misinformed – primarily by fictional rather than real-world robots.

But I understand the sentiment behind the question. In robotics we have a shared sense of a technology that has yet to reach its true potential; of a dream unfulfilled.

The question asks what is the single biggest obstacle. In my view some of the biggest immediate obstacles are not technical but human. Let me explain with an example. We already have some very capable tele-operated robots for disaster response. They are rugged, reliable and some are well field-tested. Yet why it is that robots like these are not standard equipment with fire brigades? I see no technical reason that fire tenders shouldn’t have, as standard, a compartment with a tele-operated robot – charged and ready for use when it’s needed. There are, in my view, no real technical obstacles. The problem I think is that such robots need to become accepted by fire departments and the fire fighters themselves, with all that this entails for training, in-use experience and revised operational procedures.

In the longer term we need to ask what it would mean for robotics to go mainstream. Would it mean everyone having a personal robot, in the same we all now have personal computing devices? Or, when all cars are driverless perhaps? Or, when everyone whose lives would be improved with a robot assistant, could reasonably expect to be able to afford one? Some versions of mainstream are maybe not a good idea: I’m not sure I want to contemplate a world in there are as many personal mobile robots, as there are mobile phones now (~4.5 billion). Would this create robot smog, as Illah Nourbakhsh calls it in his new book Robot Futures?

Right now I don’t have a clear idea of what it would mean for robots to go mainstream, but one thing’s for sure: we should be thinking about what kind of sustainable, humanity benefitting and life enhancing mainstream robot futures we really want.

Could we experience the workings of our own brains?

Alan Winfield — Wed, 08 May 2013 00:38:00 +0000

One of the oft quoted paradoxes of consciousness is that we are unable to observe or experience our own conscious minds at work; that we cannot be conscious of the workings of consciousness. I’ve always been puzzled about why this is a puzzle. After all, we don’t think it odd that word processors have no insight into their inner workings (although that’s a bad example because we might conceivably code a future self-aware WP and arrange for it to access its inner machinery).

Perhaps a better example is this: The act of picking up a cup of hot coffee and bringing it to your lips appears, on the face of it, to be perfectly observable. No mystery at all. We can see the joints and muscles at work, ‘feel’ the tactile sensing of the coffee cup, and its weight as we begin to lift it. We can even build mathematical models of the kinetics and dynamics, and (with somewhat more difficulty) make robot arms to pick up cups of coffee. But – I contend – we are kidding ourselves if we think we know what’s going on in the complex sensory and neurological processes that appear so effortless to perform. The fact we can observe and even feel ourselves lifting a coffee cup gives very little real insight. And the mathematical models – and robots – are not really models of the human neurological and physiological processes at all, they are models of idealised abstractions of limbs, joints and hand.

We are kidding ourselves if we think we know what’s going on in the complex sensory and neurological processes that appear so effortless to perform.

I would argue that we have no greater insight into the workings of this (apparently straightforward) physical act, than we do of thinking itself. But again this is not surprising. The additional cognitive machinery to be able to access or experience the inner workings of any process, whether mental or physical, would be huge and (biologically) expensive. And with no apparent survival value (except perhaps for philosophers of mind), it’s not surprising that such mechanisms have not evolved. They would of course require not just extra grey matter, but sensing too. It’s interesting that there are no pain receptors within our brains – that’s why it’s perfectly possible to have brain surgery while wide awake.

But this got me thinking. Imagine that at some future time we have nanoscale sensors capable of positioning themselves throughout our brains in order to provide a very large sensor network. If each sensor is monitoring the activity of key neurons, or axons, and able to transmit its readings in real-time to an external device, then we would have the data to provide ourselves with a real-time activity image of our own brains. It could be presented visually, or perhaps sonically (or via multi-media). It might be fun for awhile, but this personal brain imaging technology (let’s call it iBrain) probably wouldn’t provide us with much more insight or experience of our own thought processes.

Diffusion spectrum imaging of the human brain, developed by neuroscientist Van Wedeen at Massachusetts General Hospital. Learn more: MIT Technology Review

But let’s assume that by the time we have the nanotechnology for harmlessly inserting millions of brain nanosensors we will have also figured out the major architectural structures of the brain – crucially linking the neural scale to the macro scale. Actually, if we believe that the recently announced European and US human brain Grand Challenges will achieve what they are promising in terms of modelling and mapping human brain activity, then such an understanding should only be a few decades away. So now build those maps and structures into the personal iBrain, and we will be presented not with a vast and bewildering cloud of colours, as in the beautiful image above, but a simpler image with major highways and structures highlighted. Still complex of course, but then so are street maps of cities or countries. So the iBrain would allow you to zoom into certain regions and really see what’s going on while you (say) listen to Bach (the very thing I’m doing right now).

Then we really would be able to observe our own brains at work and, just perhaps, experience the connection between brain and thought.

This post originally appeared on Alan Winfield’s Web Log on Feb. 20, 2013.

A crisis of expectations

Alan Winfield — Mon, 29 Apr 2013 23:00:00 +0000

At the first UK Robot Ethics workshop on 25th March 2013, I offered — for discussion — the proposition that robotics is facing a Crisis of Expectations. And not for the first time. I argue that one possible consequence is (another) AI winter.
In this talk I set out the proposition that robotics is facing a crisis of expectations. As a community we face a number of expectation gaps — significant differences between what people think robots are and do, and what robots really are and really do, and (more seriously) might reasonably be expected to do in the near future. I will argue that there are three expectation gaps at work here — public expectations, press and media expectations, and funder or stakeholder expectations — and that the combined effect of these amounts to a crisis of expectations. A crisis we roboticists need to be worried about.

Public Expectations
Here’s a simple experiment. Ask a non-roboticist to give you an example of a robot — the first that comes into their mind. The odds are that it will be a robot from a Science Fiction movie: perhaps Terminator, R2-D2 or C-3PO or Data from Star Trek. Then ask them to name a real-world robot. Unlike your first question, which they will have answered quickly, this one generally needs a little longer. You might get an answer like “the robot in the advert that spray-paints cars” or, if you’re lucky, they might know someone with a robot vacuum cleaner. So, although most people have a general idea that there are robots in factories, or helping soldiers to defuse bombs, the robots that they are most familiar with – the ones they can name and describe – are fictional.

None of this is surprising. The modern idea of a robot was, after all, conceived in Science Fiction. Czech playwright K. Capek first used the word Robot to describe a humanoid automaton in his play Rossum’s Universal Robots (RUR) and Isaac Asimov was the first to coin the word Robotics, in his famous short stories of the 1940s. The idea of a robot as an artificial mechanical person has become a ubiquitous fictional trope, and robots have, for half a century, been firmly rooted in our cultural landscape. We even talk about people acting robotically and, curiously, we don’t mean like servants, we mean in a fashion that mimics the archetypal robot: stiff jointed and emotionally expressionless.

We even talk about people acting robotically and, curiously, we don’t mean like servants, we mean in a fashion that mimics the archetypal robot: stiff jointed and emotionally expressionless.

Furthermore, people like robots, as anyone who has had the pleasure of giving public talks will know. People like robots because robots are, to paraphrase W. Grey Walter, An Imitation of Life. Which probably accounts for the observation that we are all, it seems, both fascinated and disturbed by robots in equal measure. I have often been asked the question “how intelligent are intelligent robots?”, but there’s always an unspoken rider “… and should we be worried?”. Robot dystopias, from Terminator, to The Matrix or out-of-control AI like HAL in Kubrick’s 2001, make compelling entertainment but undoubtedly feed the dark side of our cultural love affair with robots.

It is not surprising then, that most people’s expectations about robots are wrong. Their beliefs about what real-world robots do now are hazy, and their expectations about what robots might be like in the near future often spectacularly over optimistic. Some think that real-world robots are just like movie robots. Others are disappointed and feel that robotics has failed to deliver the promises of the 1960s. This expectation gap — the gap between what people think robots are capable of and what they’re really capable of — is not one-dimensional and is, I argue, a problem for the robotics community. It is a problem that can manifest itself directly when, for instance, public attitudes towards robots are surveyed and the results used to inform policy [1]. It makes our work as roboticists harder, because many people think that the hard problems we are working on are already solved, and because it creates societal expectations of robotics that cannot be met. And it is a problem because it underpins the next expectation gap I will describe.

Press and Media Expectations
You are technically literate, an engineer or scientist perhaps with a particular interest in robotics, but you’ve been stranded on a desert island for the past 30 years. Rescued and returned to civilisation you are keen to find out how far robotics science and technology has advanced and, rejoicing in the marvellous inventions of the Internet and its search engines, you scour the science press for robot news. Scanning the headlines you are thrilled to discover that robots are alive, and sending messages from space; robots can think or are “capable of human reasoning or learning”; robots have feelings, relate to humans, or demonstrate love, even behave ethically. Truly robots have achieved their promised potential.

Then of course you start to dig deeper and read the science behind these stories. The truth dawns. Although the robotics you are reading about is significant work, done by very good people, the fact is — you begin to realise — that now, in 2013, robots cannot properly be said to think, feel, empathise, love or be moral agents; and certainly no robot is, in any meaningful sense, alive or sentient. Of course your disappointment is tempered by the discovery that astonishing strides have nevertheless been made.
So, robotics is subject to journalistic hype. In this respect robotics is not unique. Ben Goldacre has done much to expose bad science reporting, especially in medicine. But a robot is different to, say, a new strain of MRSA because — as I outlined above — most people think they know what a robot is. Goldacre has characterised bad science stories as falling into three categories: wacky stories, scare stories and breakthrough stories [2]. My observation is that robots in the press most often headline as either wacky or scary, even when the development is highly innovative.

I believe that robohype is a serious problem and an issue that the robotics community should worry about. The problem is this: most people who read the press reports are lay readers who — perfectly reasonably — will not read much beyond the headline; certainly few will look for the source research. So every time a piece of robohype appears (pretty much every day) the level of mass-delusion about what robots do increases a bit more, and the expectation gap ratchets a little wider. Remember that the expectation gap is already wide. We are at the same time fascinated and fearful of robots, and this fascination feeds the hype because we want (or dread) the robofiction to become true. Which is of course one of the reasons for the hype in the first place.

Who’s to blame for the robohype? Well, we roboticists must share the blame. When we describe our robots and what they do we use anthropocentric words, especially when trying to explain our work to people outside the robotics community. Within the robotics and AI community we all understand that when we talk about an intelligent robot, what we mean is a robot that behaves as if it were intelligent; ‘intelligent robot’ is just a convenient shorthand. So when we talk to journalists, we should not be too surprised when “this robot behaves, in some limited sense, as if it has feelings” gets reported as “this robot has feelings”. But science journalists must, I think, also do better than this.

Funder and Stakeholder Expectations
Many of us rely on research grants to fund our work and — whether we like it or not — we have to become expert in the discipline of grantology. We pore over the small print of funding calls and craft our proposals with infinite care in an effort to persuade reviewers (also skilled grantologists) to award the coveted ‘outstanding’ scores. We are competing for a share of a limited resource, and the most persuasive proposals — the most adventurous, which also promise the greatest impact while matching themes defined to be of national importance — tend to succeed. Of course all of this is more or less equally true whether you are bidding for a grant in history, microbiology or robotics. But the crisis of expectations makes robotics different.

There are, I think, three factors at work. The first is the societal and cultural context — the expectation gaps I have outlined above. The second is the arguably disgraceful lack of useful and widely accepted benchmarks in robotics, which means that it is perfectly possible to spend 3 years developing a new robot which is impossible to quantifiably demonstrate as superior to comparable robots, including those that already existed when that project started. And the third is the fact that policymakers, funders and stakeholders are themselves under pressure to deliver solutions to very serious societal or economic challenges and therefore perhaps too eager to buy into the promise of robotics. Whether naively or wittingly, we roboticists are I believe guilty of exploiting these three factors when we write our grant applications.

I believe we now find ourselves in an environment in which it is now almost de rigueur to over-promise when writing grant applications. Only the bravest proposal writer will be brutally honest about the extreme difficulty of making significant progress in, for instance, robot cognition and admit that even a successful project, which incrementally extends the state of the art, may have only modest impact. Of course I am not suggesting that all grants over promise and under deliver, but I contend that many do and – because of the factors I have set out – they are rarely called to account. Clearly the danger is that sooner or later funding bodies will react by closing down robotics research initiatives and we will enter a new cycle of AI Winter.

Clearly the danger is that sooner or later funding bodies will react by closing down robotics research initiatives and we will enter a new cycle of AI Winter.

AI has experienced “several cycles of hype, followed by disappointment and criticism, followed by funding cuts, followed by renewed interest years or decades later” [3]. The most serious AI Winter in the UK was triggered by the Lighthill Report [4] which led to a more or less complete cancellation of AI research in 1974. Are we heading for a robotics winter? Perhaps not. One positive sign is the identification of Robotics and Autonomous Systems as one of eight technologies of strategic importance to the UK [5]. Another is the apparent health of robotics funding in the EU and, in particular, Horizon 2020. But a funding winter is only the most extreme consequence of the culture of over-promising I have outlined here.

Discussion
I want to conclude this talk with some thoughts on how we, as a community, should respond to the crisis of expectations. And respond we must. We have, I believe, an ethical duty to the society we serve, as well as to ourselves, to take steps to counter the expectation gaps that I have outlined. Those steps might include:

At every opportunity, individually and collectively, we engage the public in honest explanation and open dialogue to raise awareness of the reality of robotics. We need to be truthful about the limitations of robots and robot intelligence, and measured with our predictions. We can show that real robots are both very different and much more surprising than their fictional counterparts;
When we come across particularly egregious robot reporting in the press and media we make the effort to contact the reporting journalist, to explain simply and plainly the true significance of the work behind the story.
Individually and collectively we endeavour to resist the pressure to over-promise in our bids and proposals, and when we review proposals or find ourselves advising on funding directions or priorities, we seek to influence towards a more measured and ultimately sustainable approach to the long term Robotics Project.

References
[1] Public Attitudes towards Robots. Special Eurobarometer 382, European Commission, 2012.

[2] Ben Goldacre. Don’t dumb me down. The Guardian, 8 September 2005.

[3] AI Winter. Wikipedia, accessed 14 March 2013.

[4] James Lighthill. Artificial Intelligence: A General Survey. In Artificial Intelligence: a paper symposium, Science Research Council, 1973. Here is a BBC televised debate which followed publication of the Lighthill report, in which Donald Michie, Richard Gregory and John McCarthy challenge the report and its recommendations (1973).

[5] George Osborne. Speech by the Chancellor of the Exchequer to the Royal Society, 9 November 2012.

euRathlon is go!

Alan Winfield — Thu, 07 Feb 2013 18:30:00 +0000

I’m very excited to be leading a new project called euRathlon – which is short for European Robotics Athlon. Up until now the project has been under wraps, but now – finally – we can go public. I’ll explain a bit more about the process that led to here later in this blog post, but first – about euRathlon.

It is an EU funded project to set up and run a series of outdoor robotics competitions. The focus is robots for search and rescue, or – more broadly – disaster response. Right now robots are not part of the standard equipment of emergency services, like fire brigades. But actually, robotics technology is coming close to the point where they could be and, in my view, should be. It seems to me that first responders should have robots as a standard part of their equipment, so that when there is a disaster robots are used as a matter of routine. euRathlon will, I hope, speed up the development and adoption of smarter robots for first responders.

The big vision of euRathlon is a competition scenario in which no single type of robot is, on its own, sufficient. Inspired by the Fukushima accident of 2011, our Grand Challenge will require teams of land, sea and flying robots to investigate the scene. Here is the project abstract:

euRathlon is a new outdoor robotics competition, which will invite teams to test the intelligence and autonomy of their robots in realistic mock emergency-response scenarios. Inspired by the 2011 Fukushima accident the euRathlon competition will require a team of land, underwater and flying robots to work together to survey the scene, collect environmental data, and identify critical hazards. Leading up to this ‘grand challenge’ in 2015, will be directly related land and underwater robot competitions in 2013 and 2014, respectively. The euRathlon competitions will be supported by annual workshops for competitors. In parallel there will be an open process of developing benchmarks to allow comparison of different robots in the euRathlon competitions. Linked public engagement activities will connect euRathlon with robotics research, industry and emergency services, as well as the general public. Attendance of spectators will be welcomed, and we hope that euRathlon events will attract considerable press and media attention. By targeting a specific and urgent need – intelligent robots for disaster-response -euRathlon will provide European robotics with a platform for challenging, extending and showcasing European cognitive robotics technologies.

Followers of this blog will know that I’ve been involved in the European Land Robotics Challenge (ELROB) for some years. I blogged about it in 2010: Real-world robotics reality check, and in 2007: A truly Grand Challenge. So, when the EU Framework Programme (FP7) issued a call for competition proposals late in 2011 an opportunity arose for those of us involved in ELROB to think about bidding for a new competition, building on that experience and extending our ambition. We were very fortunate to link up with the organisers of the Student Autonomous Underwater Challenge – Europe (SAUC-E), a very well regarded underwater robot competition. We then had land and sea robots covered. The final piece of the jigsaw fell into place when we were joined by our final partner, organisers of the workshop on Research, Development and Education on Unmanned Aerial Systems (RED-UAS 2011), with huge experience of aerial robots.

The euRathlon consortium was complete, and together we submitted our bid in April 2012. Following evaluation the bid was successful and then, from September to December 2012, we went into a phase of project negotiation, in which we worked out and agreed the details of the project with the EC. That process concluded successfully, and the project started on 1 January 2013.

So now, euRathlon is go!

Robot know thyself

Alan Winfield — Wed, 27 Jun 2012 22:10:00 +0000

How can we measure self-awareness in artificial systems?

This was a question that came up during a meeting of the Awareness project advisory board two weeks ago at Edinburgh Napier University. Awareness is a project bringing together researchers and projects interested in self-awareness in autonomic systems. In philosophy and psychology self-awareness refers to the ability of an animal to recognise itself as an individual, separate from other individuals and the environment. Self-awareness in humans is, arguably, synonymous with sentience. A few other animals, notably elephants, dolphins and some apes appear to demonstrate self-awareness. I think far more species may well experience self-awareness – but in ways that are impossible for us to discern.

In artificial systems it seems we need a new and broader definition of self-awareness – but what that definition is remains an open question. Defining artificial self-awareness as self-recognition assumes a very high level of cognition, equivalent to sentience perhaps. But we have no idea how to build sentient systems, which suggests we should not set the bar so high. And lower levels of self-awareness may be hugely useful* and interesting – as well as more achievable in the near-term.

Let’s start by thinking about what a minimally self-aware system would be like. Think of a robot able to monitor its own battery level. One could argue that, technically, that robot has some minimal self-awareness, but I think that to qualify as ‘self-aware’ the robot would also need some mechanism to react appropriately when its battery level falls below a certain level. In other words, a behaviour linked to its internal self-sensing. It could be as simple as switching on a battery-low warning LED, or as complex as suspending its current activity to go and find a battery charging station.

So this suggests a definition for minimal self-awareness:

A self-aware system is one that can monitor some internal property and react, with an appropriate behaviour, when that property changes.

So how would we measure this kind of self-awareness? Well if we know the internal mechanism because we designed it), then it’s trivial to declare the system as (minimally) self-aware. But what if we don’t? Then we have to observe the system’s behaviour and deduce that it must be self-aware; it must be reasonably safe to assume an animal visits the watering hole to drink because of some internal sensing of ‘thirst’.

But it seems to me that we cannot invent some universal test for self-awareness that encompasses all self-aware systems, from the minimal to the sentient; a kind of universal mirror test. Of course the mirror test is itself unsatisfactory. For a start it only works for animals (or robots) with vision and – in the case of animals – with a reasonably unambiguous behavioural response that suggests “it’s me!” recognition.

And it would be trivially easy to equip a robot with a camera and image processing software that compares the camera image with a (mirror) image of itself, then lights an LED, or makes a sound (or something) to indicate “that’s me!” if there’s a match. Put the robot in front of a mirror and the robot will signal “that’s me!”. Does that make the robot self-aware? This thought experiment shows why we should be sceptical about claims of robots that pass the mirror test (although some work in this direction is certainly interesting). It also demonstrates that, just as in the minimally self-aware robot case, we need to examine the internal mechanisms.

So where does this leave us? It seems to me that self-awareness is, like intelligence, not one thing that animals or robots have more or less of. And it follows, again like intelligence, there cannot be one test for self-awareness, either at the minimal or the sentient ends of the self-awareness spectrum.

Related posts:
Machine Intelligence: fake or real?
How Intelligent are Intelligent Robots?
Could a robot have feelings?

* In the comments below Andrey Pozhogin asks the question:What are the benefits of being a self-aware robot? Will it do its job better for selfish reasons?

A minimal level of self-awareness, illustrated by my example of a robot able to sense its own battery level and stop what it’s doing to go and find a recharging station when the battery level drops below a certain level, has obvious utility. But what about higher levels of self-awareness? A robot that is able to sense that parts of itself are failing and either adapt its behaviour to compensate, or fail safely is clearly a robot we’re likely to trust more than a robot with no such internal fault detection. In short, its a safer robot because of this self-awareness.

But these robots, able to respond appropriately to internal changes (to battery level, or faults) are still essentially reactive. A higher level of artificial self-awareness can be achieved by providing a robot with an internal model of itself. Having an internal model (which mirrors the status of the real robot as self-sensed, i.e. it’s a continuously updating self-model) allows a level of predictive control. By running its self-model inside a simulation of its environment the robot can then try out different actions and test the likely outcomes of alternative actions. (As an aside, this robot would be a Popperian creature of Dennett’s Tower of Generate and Test – see my blog post here.) By assessing the outcomes of each possible action for its safety the robot would be able to choose the action most likely to be the safest. A self-model represents, I think, a higher level of self-awareness with significant potential for greater safety and trustworthiness in autonomous robots.

To answer the 2nd part of Andrey’s question, the robot would do its job better, not for selfish reasons – but for self-aware reasons.
(postscript added 4 July 2012)