In this roundtable edition, we watched the Black Mirror episode “Hated in the Nation” and asked our Robohub team members: with many institutions focused on developing aerial drone technology, and in light of the pressing reality of climate change and bee colony collapse, do we see robotic bees in our future? Would swarms of artificial insects even be desirable?

In her book “The Ultimate Quotable Einstein”, Alice Calaprice asserted that Einstein probably never said anything close to: “If the bee disappeared off the surface of the globe then man would only have four years of life left. No more bees, no more pollination, no more plants, no more animals, no more man.”

Still. Whoever said it, it’s a haunting prophecy. With climate change thought to be forcing colony collapse disorder and threatening the existence of many pollinators, might swarm robotics hold the key to humanity’s survival? Could these robot bees ever be hacked to spy and become weaponised? In our roundtable we discussed the technology in the Black Mirror episode “Hate the Nation” and some of the pitfalls.

Swarms of autonomous, 3D printed drone bees pollinating the nation. Hackable AI, vulnerable to attack. Flying tools turned aerial enemy. Do our experts think any of this could happen? Is this type of technology likely or even feasible?

Our team share their thoughts:

Sabine: What’s interesting is that not everything in the episode is entirely crazy. We do want to create technology that addresses a specific need. If there are fewer pollinators, we might want to create artificial pollinators, though not necessarily artificial bees, to help us along the path to addressing the issue. But I think the technology in Black Mirror is very far from something we can do today.

There was a project called RoboBees, that was Harvard-based, that was all about creating artificial bees for pollination—so exactly this scenario.  There were multiple parts and problems: how do you make very small flyers; how do you drive them in a controlled way; and how do they sustain enough energy. All these things are extremely hard to address. They’ve managed to make some of the world’s best miniature flyers that can flap their wings and take off, and are making some great steps in controlling them. But when I say control, I really mean turning left and right in a laboratory setting. If you were to put them outside in a windy environments, it’d be a completely different story, and they don’t really have the energy to go very far.

And then there’s the algorithms. It’s not just the bees in this episode, they’re all connected to hives and they have all these control algorithms implemented that allow them to find flowers and people with image processing. How do you get such large numbers of drones to communicate with each other and their hubs? There are multiple communication issues that are very hard to address. What is interesting is that [Black Mirror] put together these bees with the swarm approach, and the pollinate in a way that’s very bio-inspired.

Kassie: Does anyone think that this episode did a disservice to swarm robotics in that someone was able to hack into the mainframe and essentially attack and kill people? That’s a very real fear that people have with regards to robotics and AI.

Sabine: For years we had to answer the question: Have you read “Prey”? Is this likely to happen? And now it’s: Have you seen “Dark Mirror”?

Ioannis: In fairness, there is some justification for this, as opposed to questions about terminators and things.

Sabine: And it’s not the technology [in Black Mirror] that’s the problem, it’s the humans. The technology starts by being portrayed as a real positive. All the tech in this episode, like the autonomous cars, is actually really cool and helpful. It’s the people who screwed up, ultimately, and the system got hacked. So I think the key message is that we need to make the technology secure.

Ioannis: Did you think it was a little short-sighted in the way it portrayed swarm robotics? It sounds very inefficient to me to have a central control that’s required to monitor the flight of every bee. All the bad guy needed to do was hack the central control to manipulate every bee. A swarm is something different than a million drones. In the episode, it looks like a swarm, but in reality it’s actually a lot of remotely operated robots.

Sabine: That’s right. And if it was properly decentralised, it would have been harder to hack because you wouldn’t be downloading massive amounts of code on every one of these bees.

Ioannis: To make the swarm go from controlled to bad, the hacker could have introduced some kind of memory or code to enable the swarm to evolve rapidly, and from that point on it’s out of control. That would have been interesting. Instead, the hacker just uses the bees as lots of tele-operated robots. I found the episode a little disappointing in this aspect.

Kassie: If you’re doubting robotics in any way, this episode is perhaps something you could look at and think: “Oh my God, this is something that could happen.” But I took some positives from this [with robotics] then only negatives.

Sabine: I felt the autonomous cars were positive; similar to what you’d expect them to be like in 30 years or so.

Ioannis: They portray autonomous cars as being cool, also, which is one aspect. But the fact that the cars are autonomous is nothing [to the story], the characters just use them, which I think is quite cool actually. They ignore the fact that these could be anything futuristic which I think is good.

Kassie: It was almost a precursor to something that could actually happen soon, in that some people were still driving, but the people with money had autonomous systems. And it wasn’t portrayed as scary! So I agree that that was a good aspect.

Sabine: I think the point of Black Mirror is very often not the technology but that people misuse it, and they make a very good point about online bullying [in this episode].

Ioannis: Yes, and they do not portray online bullies in a stereotypical way. For example, there is a teacher character who is a very nice lady in reality, but online is something completely different. Sabine, I’d like to ask you what you think about the potential of a weaponized swarm for defense purposes?

Sabine: I think it’s a very bad idea to weaponize swarms. You have to think about such things as stability and payload, accounting for wind and conditions etc. How would you do that? And why use a swarm as opposed to another type of robot with another type of payload?

Ioannis: I don’t have any specific examples for swarm robotics, but in nature a lion is dangerous, and so is a mosquito. Both in different way. So there could be an application.

Sabine: In that sense, you can weaponize anything. It’s just so much harder to weaponize a swarm than a more capable system, at least today.

Ioannis: What did you think of the robo-bee design?

Sabine: I think it was pretty cool, but too complex. If you were to design a robo-bee you would design something like a mini-copter that would sit on a plant, rather than something that looked like a bee. Or you might have an agriculture robot that simply goes across fields and does the pollination. Although the nice thing about the bee design is that the infrastructure is still already in place—plants etc.

If you enjoyed this article, you may also want to read our other Roundtables:

• Robohub roundtable: Public perceptions of robotics and AI, what’s your pet peeve?
• Robohub roundtable: Why do robotics crowdfunding projects fail?
• Robohub roundtable: Cybathlon and advancements in prosthetics
• Robohub roundtable: Job loss through automation, Foxconn controversy
• Robohub roundtable: Why is it so difficult to define ‘robot’?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

HBO’s latest offering (on SkyAtlantic here in the UK) is an update of Michael Crichton‘s 1973 film Westworld; this time brought to us as a ten-part television series by sci-fi re-booter extraordinaire J.J. Abrams and screenwriter Jonathan Nolan (yes, brother and frequent collaborator of Christopher). The new Westworld comes with much hope (for HBO as a potential ‘new Game of Thrones’) and hype, understandably with the talent behind it having given us so much terrific science-fiction of late, including Star Wars: The Force Awakens and Interstellar.

As with Channel 4’s series Humans, broadcast last June (news on forthcoming series 2 here), Dreaming Robot’s offered a live Twitter commentary while the show was being broadcast in the UK, and I’ll take some time afterwards to write some reflective pieces on what we see in the show. (The Twitter ‘Superguide’ from @ShefRobotics can be seen here; my own @DreamingRobots Superguide can be seen here.)

Unsurprisingly, many of the themes in the Westworld reboot could also be seen in Humans. It seems, for example, that both shows express a certain anxiety, that as our machines become more human we humans seem to become less human, or humane. But this isn’t a new idea original to either show – this anxiety as been around as long as robots themselves, from the very invention of the term robot in the 1920s. And if we trace the history of robots in science fiction, we see a history of monsters that reflect this same fear, time and again, in slightly different contexts. Because the robot – which, remember, was invented in the popular imagination long before they were built in labs – is above all else exactly that, a perfect way of expressing this fear.

So, what are we looking at in this new, improved Westworld? (because frankly, the original film lacked the depth of even the first hour of this series, being just a very traditional Frankenstein narrative and rough draft for Crichton’s Jurassic Park, made 20 years later). First – as this nifty graphic on the right illustrates – we do see the robots in Westworld becoming more human. The programme starts with a voice asking the humanoid (very humanoid) robot, Dolores (Evan Rachel Wood), ‘Have you ever questioned the nature of your reality?’ The questioner is exploring whether Dolores has become sentient, that is, aware of her own existence. The echo here with Descartes is clear. We all know about cogito, ergo sum – I think, therefore I am. But Descartes proposition isn’t just about thinking, it is about doubt.  He begins with the proposition that the act of doubting itself means that we cannot doubt, at least, our existence . So we would better understand Descartes’s proposition as dubito, ergo cogito, ergo sum: I doubt, therefore I think, therefore I am. If Dolores is found to be questioning the nature of her reality then that would be evidence for self-awareness and being, according to the Cartesian model.

The robots in Westworld are depicted as falling in love, maintaining strong family bonds, appreciating beauty, and consider the world in philosophical, reflective contexts. How much of this is merely programming and how much exceeds the limits imposed upon them by their human masters is the key question that the show teases its audience with, I suspect, though certainly, it occupies much of our attention in the first few hours. But if these moments – described as ‘reveries’ – are in any way genuine moments of creativity, then this is another category, beyond the notion of the cogito, that we might say the robots are becoming ‘alive’.

For many thinkers in the second-half of the twentieth century (for example, the post-Freudian psychoanalyst D. W. Winnicott), it is only in such moments of creativity, or enjoying genuine moments of spontaneous being, that we truly discover the self and come alive. (The Freudian and post-Freudian influences on the narrative become even more apparent in subsequent episodes.) As response to late industrial capitalism (and the shock of fascism to the human self-conception as a rational animal), this idea emerged of human beings coming ‘alive’ only when we are not acting compliantly, that is, when we are acting spontaneously, or creatively, and not according to the laws or dictates (or ‘programming’) of another individual, organisation or group consciousness. We see this trend not only in a post-Freudian psychotherapy (e.g. Eric Fromm and the Frankfurt School, R. D. Laing) and other philosophical writings but also in the popular post-war subculture media, including advertising – the sort satirised in the ad for Westworld that opens the original film.

Other perspectives give us a glimpse into the robot’s moments of becoming human. Peter Abernathy, looking at a picture that offers a peek at the world outside, says: “I have a question, a question you’re not supposed to ask.” This is an allusion to Adam and Eve and the fruit of forbidden knowledge when humankind became self-aware of the difference between right and wrong. (Peter, after this, is consumed with rage at how he and his daughter have been treated.) And like Walter, the robot that goes on a psychotic killing spree, pouring milk over his victims, Peter is determined to ‘go off script’ and reclaim for himself a degree of agency and self-determination, acting according to his own, new-found consciousness instead of according to what others have programmed for him.

MEANWHILE, the human beings (‘newcomers’) in Westworld seem less ‘humane’ than their robot counterparts. The newcomers are shown to be sadistic, misogynist, psychopathic in the indulgence of their fantasies. One could argue that this behaviour is morally justifiable in the unreal world designed solely for the benefit of paying customers – that a ‘rape’, for example, in Westworld isn’t really ‘rape’ if it is done to a robot (who, by definition, cannot give nor deny consent), but this clearly is not how the audience is being invited to see these actions.

That human beings are becoming more like machines is an anxiety for which there is a long history of evidence, and even pre-dates the cultural invention of robots in the 1920s. We can see this anxiety in the Romantic unease with the consequences of Enlightenment that gave birth to the new, rational man, and of the industrial revolution, that was turning humans into nothing more than cogs in the steam-powered machines that so transformed the economy. This has been addressed in the Gothic tale of Frankenstein, still the basis for so many narratives involving robots, including the original Westworld film and, more recently, even our most contemporary stories such as Ex_Machina, and, to a lesser extent, in this manifestation of Westworld (which will be subject of a future post). (I have written and spoken on this theme myself many times, for example here and here).

So in Westworld, we meet Dr Ford – the mad scientist who creates the machines that will, inevitably, be freed upon the world. Dr Ford immediately reminds us of another Ford, the man whose name is synonymous with the assembly lines and a mode of production in the late industrial revolution that has done so much to dehumanise modern workforces. We see these methods of production and workers in the iconic film, which was contemporary with Henry Ford’s factories, Metropolis. (Though, as we shall see, this Ford is rather more complex…).

This dehumanisation of human beings in the industrial age has been looked at in robot films since Metropolis #Westworld pic.twitter.com/NkoBH7mlD2

— Dreaming Robots Blog (@DreamingRobots) October 4, 2016

This fear reflects, too, that as post-Enlightenment humans become more rational, they become more like machines, acting in predictable, programmed ways, having lost the spontaneity and creativity of an earlier age. The humans of Westworld are exaggerations of the humans of our ‘Western’ world of rationalism, science and alienation. We don’t have to agree with this Romantic notion, that rationalism and science are negative forces in our world to accept that there is a great deal of anxiety about how rationalism and science are transforming individual human beings and our societies.

Rational dehumanisation is something personified in the actions of the corporation, which has replaced the mad-scientist as the frequent villain of the sci-fi Frankenstein-robot-twist (more to come on this in a future post) and we see hints in Episode 1 of what is to follow in Westworld, along the lines of film’s such as 2013’s The Machine, where the slightly misguided and naive actions of a scientist are only made monstrous when appropriated by a thoroughly evil, inhumane military-industrial complex.

This theme to address so succinctly in Ridley Scott’s Blade Runner, a major influence on the new Westworld, where the Tyrell Corporation boasts that their replicants are More Human Than Human. And in Blade Runner, too, we see humanoid robots behaving more humanely that the people that ruthlessly, rationally hunt down the machines. It is unclear, however, from the Tyrell slogan whether the robots are more human than the human because the technology has become so sophisticated, or because humans have fallen so low.

On Westworld as a whole, it is too early to tell if it will maintain its initial promise and be as monumentally successful as Game of Thrones, or as iconic as Blade Runner. But already this first episode has given us more to think about than the original, and undoubtedly the success and failure of the programme will be instructive.

If you liked this post, you may also want to read:

• Robots can successfully imitate human motions in the operating room
• The dark side of ethical robots
• Law-abiding robots? What should the legal status of robots be?
• Beyond Asimov: how to plan for ethical robots

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Callan McAuliffe in Robot Overlords

In the low-budget British sci-fi adventure, Robot Overlords, alien automatons rule the streets and ordinary citizens are locked under curfew in their homes. Just by stepping outside, you risk being vaporised by a hulking Sentry, or picked off by a lethal Sniper.

Amid the ruins of civilisation, Sean Flynn (Callan McAuliffe) leads his group of young friends on a quest to join the resistance forces that are standing against the robot invaders. Hot on their heels, however is their old teacher – now treacherous robot collaborator – Robin Smythe (Ben Kingsley) and his captive Kate (Gillian Anderson). Who will prevail? Man or machine?

Directed by Jon Wright, Robot Overlords features some 265 visual effects shots delivered by Soho-based Nvizible, supervised by the company’s founder and co-owner, Paddy Eason. The film’s complex tracking requirements were fulfilled by Peanut FX, with additional compositing support being provided by Boundary VFX, under the supervision of Nick Lambert.

Watch a wide selection of Nvizible’s VFX shots in the official video for Robots Never Lie, written by Matt Zo for the Robot Overlords end credits

Origins of the Overlords

The original story concept for Robot Overlords came to director and co-writer, Jon Wright, in a dream. “I dreamed that I was playing cowboys and Indians with a little boy,” Wright recalled. “A huge two-storey robot came marching up the street and swung its laser cannon arm towards him, and a voice boomed out, ‘Citizen, drop your weapon immediately!’ I assumed I was just recycling a movie that I’d already seen, but eventually, I came to the conclusion that maybe it was an original idea.”

After expanding the dream concept into a two-page treatment, Wright began developing a script with co-writer Mark Stay. “Jon and I are both big fans of those 1980s Amblin movies like E.T.: The Extra-Terrestrial and The Goonies,” said Stay. “But those films were always set in American suburbia – we wondered why nobody had ever set a film like that in the UK.”

“It came together very quickly, especially for a British film,” Stay continued. “Our elevator pitch was, ‘It’s like The Goonies, but with robots blowing stuff up!’ We took Jon’s treatment to Piers Tempest – one of the producers from Jon’s previous film, Grabbers. The BFI put us under their wing, and before we knew it we had some development money.”

VFX supervisor Paddy Eason was also consulted at this early stage. “I knew Jon because we did the effects for his first two films – Tormented, and an Irish monster movie called Grabbers.” Eason commented. “The first thing I received was the treatment, and the next throw of the dice for me was getting invited to be part of a group that critiqued the script – Jon calls it the ‘Tiny Think Tank’.”

The Tiny Think Tank

The Tiny Think Tank is an initiative conceived by Wright as a means by which a group of filmmakers can gather together periodically, in order to review scripts in development. “It’s a gang of writers, actors, directors, visual effects people,” Wright explained. “There aren’t any producers or financiers involved – it’s purely creatives. We come together, read a script completely cold, then have a round-table discussion. It was inspired by Pixar. They have their team of creatives – who are also shareholders – who come together and critique each other’s films. The Tiny Think Tank is our low-budget UK version of that, and I think it’s one of the reasons Robot Overlords got made. Our script accelerated in terms of quality much more quickly than it would have done had we been writing it in isolation.”

Nvizible’s VFX breakdown for a key sequence in “Robot Overlords”

As part of the Tiny Think Tank, Paddy Eason was able to offer a level of opinion and criticism not commonly afforded people in the visual effects profession. “I had some suggestions from my own visual effects point of view,” Eason remarked. “But I also put some story ideas out to Jon and Mark. As a visual effects person, that’s not normally part of your remit. In fact, on many shows, to put forward story ideas would be viewed as a breach of etiquette. So it was delightful to be invited to give that kind of feedback.”

Nvizible’s association with Robot Overlords developed further when the company became a production partner on the project. Nevertheless, they were still required to bid for the work. “Robot Overlords was a regular production with a bond company and other financiers, so it had to be seen not only that our bid was value for money, but also that our costs weren’t unrealistically low,” Eason pointed out. “Also that, if the unthinkable happened and we dropped the ball, there would be somewhere else to go to get the work done. Obviously it didn’t come to that!”

How to Design a Robot

Key to the success of Robot Overlords was the convincing creation of the alien machines that have descended from outer space to occupy planet Earth. Design concepts for the mechanical marauders developed naturally alongside the story.

“The Robot Empire is stretched across the galaxy,” explained Stay. “If you compare it to the Roman Empire, the Earth is like Caledonia – it’s the last place any of them want to be. Resources are stretched thin. So the Sentries, for example, are quite sorry figures. They walk with a kind of stoop, and they’ve got chips and dents and dinks in their armour, because they’ve been here for three years and survived a war.”

According to the storyline, the Robot Empire studies the inhabitants of each planet they plan to subjugate, and designs an automated invasion force specifically to tap into the psychological profiles of that planet’s inhabitants. This bespoke engineering approach is evident in the design of the Sentries: two-storey, bipedal robots with small heads and enormous, hulking arms.

According to co-writer Mark Stay, the robot Sentries are designed to tap into our instinctive fear of “people with tiny brains and big muscles”.

“In studying us, the robots have discovered that we’re frightened of people with tiny brains and big muscles,” explained Stay. “That informed the design of the Sentry. Our other robots – like the Snipers and Air Drones – are very functional, each with a specific purpose. ”

A different psychological insight led to the design of the Mediator – a kind of robot diplomat that acts as an interface between human beings and their automated oppressors. “The Mediator is made to resemble a child, because they’ve observed that, on the whole, adults don’t behave aggressively towards children,” Jon Wright remarked. “But it’s been designed by an alien species who aren’t familiar with the minutiae of how we react, how our culture is organised. So they’ve got it a bit wrong. They’ve attempted to make something likeable, but they’ve actually made something that’s extremely creepy.”

Early conceptual designs for the robots were developed by storyboard and concept artist, Gabriel Schucan. “Jon and I sat with Gabriel,” Stay recalled. “We scribbled ideas, and he would develop them and send drawings back to me and Jon. That was still very early on in the writing of the script.”

“Robot Compendium” concept document – Sentry illustration by Jack Dudman

Also at this early stage, the team created the “Robot Compendium”, a manual containing a piece of key art for each robot, drawn by Jack Dudman, along with a description of the machine’s capabilities. “The Robot Compendium, helped us to sell in the script to investors,” remarked Stay. “So it was a really important document for us.”

During the story development process, visual effects constraints influenced what might be achievable, both practically and financially. “While we were still talking about the script, we started doing visual effects breakdowns in a very loose, back-of-an-envelope way,” said Eason. “Some scenes would turn out to be looking very expensive, and that was fed back into the writing process. For example, there was a scene in the woods with our heroes being chased by robots called Octobots. That was quite heavy-duty in terms of visual effects, and wasn’t particularly important in the story. So it was elbowed fairly early on.”

“Robot Overlords” concept art by Paul Catling.

The Overlords Become Real

Once Robot Overlords was greenlit, the initial robot concepts were handed over to designer and illustrator Paul Catling, known for his work on films including Guardians of the Galaxy, Captain America: The First Avenger and the Harry Potter series.

“Paul is incredibly productive,” Eason remarked. “He did lots and lots of pen-and-ink designs, just exploring how the robots might look. Early on, some of them were much more organic-looking, and skeletal, and frightening in a slightly Satanic kind of way. Ultimately, we decided the robots should be slightly crude-looking. We wanted to avoid comparisons with Transformers – Jon wanted our robots to look much more industrial. We also wanted them to be slightly reminiscent of WWII Nazi technology.

Paul Catling’s concept designs for the Sentry.

Although the design of the Sentry was inspired directly by Wright’s inspirational dream, it nevertheless went through a lengthy development process. “One of the original tests we did for the Sentry was a lot more like Robby the Robot,” Wright revealed. “Then he developed into more of an Iron Giant character – we just pushed that design to make it slightly more peculiar. He’s slightly lop-sided, and made up of different geometric shapes. That was partly driven by the plot device of having them fold up into cubes, which was our way of showing that they’ve been deactivated.”

The Mediator robot was played by Craig Garner, whose appearance was enhanced in post-production to give the actor’s skin an unsettling plastic sheen. For some scenes, frames were removed to create staccato movements.

The Mediator, in contrast, was conceived from the beginning as a machine that looked like a human – but not too much like a human. “Right from the beginning, we wanted deliberately to put the Mediator in the uncanny valley, rather than try to avoid it,” Wright noted. “We gave the actor, Craig Garner, perfect false teeth, trimmed his hairline to a perfectly straight line, gelled his hair rock-solid, and tried to remove all of his imperfections and blemishes with make-up. Then that was enhanced in post-production with clean-up, skin glows, making his eyes uniform. Sometimes we’d take out every other frame to give him a kind of clockwork, staccato movement.”

“The bulk of the Mediator work was done in the grade by the colourist, Gareth Spensley, on his Baselight,” Eason elaborated. “However, the software we use at Nvizible for skin work is a suite of NUKE tools that we use as part of our ‘Nhance’ service. These allow us to selectively filter out features of a certain size – for example, wrinkles or spots – while retaining everything else, like image grain, shadows and regular skin texture.”

Robots at Nvizible

With the exception of the Mediator, Nvizible further developed the robot designs in their CG workspace. This process was facilitated by initial 3D work done by Paul Catling. “As Paul progresses his designs, he starts working in 3D,” explained Eason. “He’ll do a render, then go on top of it in Photoshop and paint lighting effects and so on to turn it into an illustration. But the underlying geometry is there. He even rigs it a little so he can move the arms and legs around to come up with certain key poses. So he ended up with some pretty well-realised CG models, which he then gave to us. We had to rebuild them, but they gave us a head-start.”

Early 3D concept for “Robot Overlords”

Nvizible’s robot models were built, rigged and animated primarily using Maya, with additional sculpting work done using ZBrush. “The Sentry ended up being a very complicated rig, with hundreds of moving parts,” Eason noted. “But it does play the biggest part in the film.”

Texturing was applied in MARI, using photographic reference chosen to support the story point that the robots have been operating in a hostile environment for many years, and are therefore both weathered and battle-worn. “There was no decoration on the robots – no painted symbols or information; it was all slabby, raw, metal, moving parts,” Eason noted. “We took photographs of similar unadorned, mechanical things, like old steam engines, big industrial lifts, big heavy things made out of cast iron. Part of the fun was making them nice and weathered – chipped and broken – which also helped to give them scale.”

Ncam and the Air Drone

Many key scenes featuring the robots were shot using a real-time camera-tracking system provided by Nvizible’s sister company, Ncam. The Ncam system allows a camera operator to view pre-visualised CG assets through the viewfinder, adjusted to conform to the parameters of the lens and superimposed in the correct position over the live-action.

“Ncam works without any tracking markers. It lets you see a previs version of your visual effects assets through the camera, or at video village,” Eason explained. “Ncam combines the live video feed from the camera with a render from a workstation, all in real-time, showing your VFX asset in situ. It could be anything from a set extension to a creature.”

Ncam was used during the film’s opening sequence, in which a crazed man runs down a suburban street shouting defiance at his robot oppressors. An Air Drone flies down, warning the man to return to his home; when the man ignores the robot, it vaporises him.

“My view with visual effects is to say, ‘Go ahead and do everything practically that you can, and we’ll deal with it in post.’ Rain? No problem!” – Paddy Eason, Nvizible.

The sequence was shot on location in Bangor, on the outskirts of Belfast, Northern Ireland, using primarily hand-held cameras. Following Wright and Schucan’s storyboards, Nvizible created previs for the sequence, using survey data of the location gathered by members of their Belfast office.

During the night shoot, interactive lighting was coordinated by Fraser Taggart, director of photography, to simulate the glow from the Air Drone’s blue hover-jets, while practical rain simulated a torrential downpour. “Rain gives you a huge amount of visual texture, so it’s a lovely thing to have,” observed Eason. “Generally speaking, my view with visual effects is to say, ‘Go ahead and do everything practically that you can, and we’ll deal with it in post.’ Hand-held cameras? No problem! Rain? No problem! We’ll add CG rain, and we’ll make sure it’s composited behind your rain. But there were a couple of shots where the camera was looking up into the sky, where it would have been too much of a nightmare to have rain falling into the lens, so I did ask them to shoot dry versions to give us the option.”

Practical pyrotechnics were used for the moment when the man is blown up by the robot, somewhat in the manner of a human firework. “We shot the plate with the actor, a clean plate with no actor, and a third plate with the pyro going off,” Eason revealed. “We used a similar camera move each time, but there was no motion control – which is fine as long as everyone’s on the same page. The practical pyro gave us real interactive lighting, and a good clue of how the rain is lit up. We enhanced it with additionally-shot bluescreen pyro elements afterwards.”

For the shots featuring the Air Drone, Ncam was used extensively to aid the shooting of the live-action plates. “It was exciting to see the reaction of the camera operator,” Eason recalled. “Like a lot of people, he was a bit sceptical about Ncam – sometimes people view it as a bit of an encumbrance. But when he realised he was actually able to see the hovering Drone through his viewfinder, and walk around it and choose angles, he was really excited and impressed.”

In one close-up shot, the camera executes a 180° move around the muzzle of the Air Drone’s gun, with the camera racking focus on the barrel. “I don’t think you’d have shot that plate without Ncam,” Eason asserted.

While the live-action plates were used for most scenes featuring the flying robot, Nvizible created some backgrounds using 360° HDRI panoramas shot on location by VFX assistant Erran Lake, using a DSLR camera. “Sometimes the plate done with the movie camera might be nice, but technically very difficult,” Eason explained. “So we would create our own version of that using my stitched stills, and put the rain in ourselves.”

Ncam was used for a scene in which a giant robot Sentry pursues Connor (Milo Parker) down a suburban street.

As director, Jon Wright appreciated the freedom afforded by Ncam, notably during a sequence in which a gigantic Sentry chases a boy down a suburban street. “You’re able to shoot something as if it was actually there, so it unlocks your shooting style,” he commented. “We were able to imbue those shots with a hand-held quality, and be quite cavalier about the way we shot. Normally when you shoot a visual effect like that, and there’s nothing there, you’re worried about not framing it up properly.”

“It was quite difficult to set up, positioning the robot and getting good tracks on the camera,” Wright continued, “but it meant we were able to do a running hand-held shot backwards with the boy, framing up the robot above him. The Sentries are massive, so they tend to go out of frame. If there’s nothing there, it makes the operator nervous and they tend to err on the side of caution and give you a slightly peculiar, lop-sided frame. If they can see the thing, they frame it much more naturally, and you get a much more interesting shot.”

NCam workflow – final composite (before grading)

“I imagine in years to come this will be fairly standard practice,” Wright concluded. “The days of tennis balls on sticks are numbered.”

Watch a video of the live NCam feed as seen by the camera operator:

By the time post-production began, Robot Overlords already existed in rough-cut form, since editor Matt Platts-Mills had been present on location throughout production, cutting each day’s rushes together into a rough assembly. “It was very helpful to have Matt with us, cutting as we went,” Eason observed. “By the time the shoot was done, we were in a good position to hit the ground running on certain key visual effects scenes.”

Since much of the film had been shot hand-held using anamorphic lenses, one of the first challenges for the visual effects team was tracking. Peanut FX handled all the film’s tracking requirements, delivering 3D layouts for roughly 115 shots, mostly with 3D camera and/or object tracks for set extensions and robot integration.

“We mainly used 3DEqualizer,” commented Amelie Guyot, matchmove supervisor at Peanut. “We find it great for solving anamorphic camera tracks when lens distortion grids and camera info are available. We were provided with a lidar scan model for scenes involving a castle, which we used to perfectly match the plates – 3D scans are always very helpful, especially with anamorphic shots. We also used Autodesk’s Matchmover to calibrate sets; it’s still one of our favourite tools for image-based 3D reconstruction.”

“Tracking was a concern early on,” Eason admitted, “but the guys at Peanut absolutely rocked it and did a brilliant job.”

Tracking for “Robot Overlords” was handled by Peanut FX, who worked meticulously to deal with director Wright’s extensive use of hand-held cameras, and some challenging sequences in which the robots interact directly with humans and their technology.

During post-production, Jon Wright made himself available to the Nvizible team on a regular basis. “I made a point of remaining on the film throughout the time that Nvizible were working on the visual effects, so I could go in and out of their offices at my leisure,” the director stated. “It saves a lot of work if you’re seeing updated shots regularly – you tend to have less waste. I don’t know how to run Nuke or anything, but I think I’m quite geeky, and have a good understanding of how effects work. I can tell if something’s going to be quick to do, or time-consuming. I think maybe some directors don’t have that.”

With regard to the animation of the robots, Wright paid particular attention to the way the movement of the metal invaders expressed their character – or rather, their lack of it. “There’s a tendency to anthropomorphise robots,” Wright observed. “Our robots are quite implacable, so it was important to strip out emotion.”

Even given this direction, nevertheless Wright found that individual animators inevitably had their own take on how the robots should move. “After a bit I could guess quite accurately who’d animated what, because they bring their own personality to it,” he commented. “It got to a point where I would say, ‘I know this shot would be great for this animator,’ on the basis of what they’d done before.”

Lighting the CG robots brought its own set of challenges, due to the difficulty of making metallic objects look both highly reflective and convincingly three-dimensional. “It’s a bit of a double-edged sword,” Eason remarked. “If you’ve got good HDRIs for everything, you get good reflections, but because most of what you see with metal is based on reflections, it’s hard to make stuff that’s physically juicy, with a nice keylight and a nice fill light. You’re always fighting that chrome robot thing, where it all looks a bit floaty and unlit.”

Skyship concept art by Paul Catling.

Riding the Skyship

One of the most difficult visual effects sequences occurs at the film’s climax, when Sean finds himself riding through the skies on the front of the enormous robot Skyship.

“They were really hard shots, because they were extremely ambitious,” Wright admitted. “I’d imagine that even the likes of ILM working for J.J. Abrams would have found those shots tough to get right.”

Once again, NCam was used to combine a previs version of the Skyship with the live-action as it was being shot on a small studio set. “The set had a slight bounce, which made things interesting,” Eason recalled. “We ended up replacing all the practical set with CG, although it was useful to have it there for reference, shadow catching and so on.”

The film’s ambitious climactic scenes, during which Sean (Callan McAuliffe) rides on the exterior of the massive Skyship, proved particularly challenging.

In post-production, the low-resolution Skyship model was replaced by its high-resolution counterpart. “The main challenges with the Skyship came in look-dev, working out how to make a big shiny collection of metal blocks look big and scary and cool,” Eason commented. “In reality, large steel things tend to look like concrete, so we had to cheat the physics of the renders quite a bit. We had a lot of fun with the Skyship engines – big jets of dirty orange and blue fire. Our FX lead, Wayde Duncan Smith, did a great job of simulating all that stuff in Houdini.”

Because the sequence featured multiple camera angles, with moves that were very specific to the animation of the various characters and craft, it was deemed unfeasible to shoot live-action background plates. Instead, generic aerial plates of landscapes were shot using an array of DSLR cameras.

“We flew the route in a chopper several times, shooting 160° panoramas using three Canon 5Ds, shooting stills at 3fps,” Eason revealed. “We fed these image sequences through a pipeline of undistortion, stitching, stabilising, optical flow and re-projection, to make beautiful background plates of moving land for all the flying Skyship shots. It was hard work, but our Nuke artist, Antoine Jannic, absolutely nailed it.”

Watch a video of a completed shot from the Skyship sequence:

Enter the Spitfire

Also featured during the film’s finale is that most British of icons – a Supermarine Spitfire. The old-school technology of the WWII fighter plane proves pivotal in the battle against the robots, whose jamming devices have grounded more sophisticated jet aircraft.

The production shot live-action of the historic plane on the ground using a mock-up provided by Gateguards UK. An authentic Spitfire was also made available for one day of the shoot by The Aircraft Restoration Company, Duxford. In addition, Nvizible built a CG Spitfire which they used for additional aerial shots.

Jon Wright (right) directs the action around the museum-piece Supermarine Spitfire, on location in the Isle of Man.

“We used the mock-up Spitfire in a remote, disused quarry location on the Isle of Man,” explained Eason. “Then we also had the real Spitfire fly out to us for the day. We positioned it on the airfield and dressed it similar to how the mock-up had been in the quarry. It was a huge privilege to have a real museum-piece there, and a dream for us in terms of creating our CG Spitfire for the flying shots, because we were able to get perfect reference, taking thousands of photographs, right down to every rivet.”

Eason was doubly delighted when he found himself booked in for a flight in a two-seat Spitfire. “The producer, Piers Tempest, knew I was a big fan of Spitfires, and he made a casual comment: ‘We’ll get you a flight in it.’ I didn’t believe it would ever happen, but a couple of months later it was arranged for me to go to the Imperial War Museum at Duxford. I got to fly around, and control the Spitfire, and everything. Ostensibly it was to record audio inside the plane – I was wired for sound – but also it was just a marvellous day out.”

Robot Reflections

Looking back at his experience of working on Robot Overlords, Mark Stay noted the benefits brought by the collaborative process: “I’ve been involved in the whole procedure in a way that many other writers aren’t usually. I was writing the novelisation at the same time, so I would email Paddy to ask questions like, ‘What colour are the engines?’ He would invite me over to Nvizible so I could sit in on VFX test screenings, and I would be there furiously scribbling all this stuff down for the novel. I’ve been really blessed with the access that I’ve had.”

“It was a delightful experience, because of our creative involvement from beginning to end,” Paddy Eason added. “I love the fact that it’s quite an unusual British film – very charming and quirky, starting at a very small personal scale and ending up quite epic and extraordinary.”

The film’s ambition is something that Jon Wright is proud of. However, he remains sanguine about the challenges involved in bringing such a film to the screen.

“It’s a bit of a shame that we aren’t really making these kinds of movies in Britain and Ireland, and that British cinemagoers almost prefer to go and see American films over British films. And we’ve got all our best filmmakers going across the water to do remakes,” Wright commented. “But the truth is, when you make a movie like this, you go up against massive Hollywood blockbusters, and it’s difficult to deliver in the way that they deliver. So we tried to focus on character, and give the film a different tone to a typical Hollywood movie at the moment – not dark and moody, but kind of light-hearted and optimistic. Our kids swear a lot and are playful, rather than being the earnest, soul-searching types you get in typical American YA movies.”

Echoing Wright’s thoughts, Eason recalled, “I remember one meeting we had early on, where we pitched the project to quite a well-known film producer. This producer said, ‘If you pull it off, I’ll eat my hat.’ So now we’re going to get a nice hat-shaped cake made for him!”

Having enjoyed limited theatrical release across the UK through Signature, Robot Overlords may also be gaining momentum as a TV series. “We have strong interest from two big broadcasters, in the US and the UK,” Wright revealed. “In 2015, television is probably the way to go with this sort of project. We could really explore into the human politics of the situation – life under enemy occupation. Obviously, the occupation is going on across the entire planet, so there’s a million different directions you could expand the story into. And ten hours of television is probably more exciting than another 90-minute film. It could be very good – watch this space.”

Robot Overlords is now available on DVD. 

• Robot Overlords on Facebook
• Nvizible
• Peanut FX
• Boundary VFX
• Jon Wright at IMDb
• Paddy Eason at IMDb
• Mark Stay – Unusually Tall Stories
• Amelie Guyot at IMDb
• The Foundry – NUKE, MARI
• Autodesk – Maya, Mudbox, Matchmover
• Pixologic – ZBrush
• Side Effects – Houdini

Special thanks to Marek Steven, Alex Coxon, Piers Tempest and Phil Guest. “Robot Overlords” photographs and video copyright © 2015 and courtesy of Tempo Productions and Nvizible.

• The robots in Ex Machina: VFX Q&A with Andrew Whitehurst
• True AI, or clever simulation? Transcendence movie has Johnny Depp crossing the Singularity
• 12 favorite robot movies – new and old
• The real soft robots that inspired Baymax, with Chris Atkeson

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Underwater Dreams is a documentary that follows a team of undocumented students from an Arizona high school and shows their perseverance and ingenuity to win a NASA- and ONR-sponsored underwater robotics competition. President Obama filmed a 1-minute introduction to the documentary and hosted a screening at the White House to a group of 100 students during National Robotics Week.

His message underscored the extraordinary contributions immigrants – including the four team members that won the competition – have made to America.

https://youtu.be/tZ9MVbQiN-k

The Marine Advanced Technology Education Robotics Competition is an annual underwater robotics competition and was established to engage students in STEM and expose them to science and technology careers, encourage them to apply technical, teamwork and problem solving skills, and to provide funds, materials and technical expertise to support their learning experiences.

At the White House screening, the original four students spoke and mingled. A White House press release described them and the event:

These pioneering robotics teammates, who are now continuing their education and earning a living here in the United States, joined the students and spoke to them about the challenges they faced as immigrants. One of their challengers from the robotics competition participated in this reunion as well! The Carl Hayden team’s passion for exploring, tinkering, and constructing a robot transformed the idea of engineering from an abstract concept to something tangible they could master, all while in high school. Engineering changed their lives, and the lives of their families and communities, forever.

 

In a previous story about the movie Spare Parts – with George Lopez, Jamie Lee Curtis, Marisa Tomei and four young Hispanic actors – the plight of the real students to stay in the US, continue their educations and get gainful employment was poignant, supremely difficult … and illuminating.

Their struggles after graduating from high school were reported by Joshua Davis first in a NY Times Op-ed piece, later in a book, and finally in the Spare Parts movie:

• Lorenzo Santillan considered becoming a mechanical engineer, but instead earned a cheaper associate degree in cooking. Today he works as a line cook.
• Luis Aranda studied cooking as well but did not complete a degree and now works as a janitor.
• Cristian Arcega, the most technically skilled member of the team, made it to Arizona State University on a scholarship. But in 2006, Arizona voters passed Proposition 300, an anti-immigrant measure that prohibits undocumented students from receiving any state financial aid or qualifying for in-state tuition — more than tripling his tuition. He dropped out and is now unemployed.
• Oscar Vazquez’s story is the most dramatic. He also had a scholarship to A.S.U. He was a sophomore when Proposition 300 passed, and managed to stay in school only by piecing together more scholarships, all while leading the university’s robotics team to regional championships. He graduated with a degree in mechanical engineering in 2009 and applied for legal residency. Not only was his application denied, but he was also summarily banned from the United States for 10 years for living here without a visa. He ended up working on an assembly line in Mexico. After a year, his ban was reversed when a US Senator from another state interceded on his behalf. Mr. Vazquez returned, enlisted in the Army, and served a tour of combat duty in Afghanistan. He is finally a citizen, and repairs trains in Montana for the railroad company BNSF.

Underwater Dreams is available through iTunes, Amazon and rental streaming.

• 12 favorite robot movies – new and old
• While most robots are either geek or jock, CHAPPiE is a lesson in sexy
• A cruel waste of STEM talent
• Ex Machina: When Turing meets Bechdel test

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

I recently had a chance to view Automata and wanted to share my thoughts on the movie based upon the current and future state of robotics and artificial intelligence. As a film, this was a tough one to sit through, and won’t make any must-see lists from me. Unless you’re really into Antonio Banderas, I’d skip this. My deep interest in robotics was the only reason I struggled to the end.

From a technology standpoint the film’s underlying premise – that robots seeking to repair themselves have a consciousness – is not as simple as it would seem. While there is no universal test for self-awareness, like a Turing or mirror test, the film seems to suggest that if a robot seeks to self-repair or self-mutilate, it shows it has at least some level of consciousness. But I see this as problematic: to me, self-repair is simply a good algorithm, something we see regularly in the process automation industry, for example when a system clears a jam, or adjusts to a flow rate due to over- or under-loading.

In fact, in May 2014, Dextre and the Canadarm became the first robotic system to self-repair on board the International Space Station. Measured by this standard, the future has already arrived – but I’m not so certain robots will be taking over the earth any time soon. And even if I’m wrong, and the Canadarm seeks to become a human overlord, we still have a 200+ miles and a seriously hot re-entry to buffer us.

For another example, take NAO or the soon to be released Pepper by Aldebaran: by recognizing facial expressions they can estimate human emotion. Is that a sign of intelligence, or just another really good algorithm? Research into human-robot interaction often concludes that people believe they have formed emotional connections with robots. Soldiers frequently develop emotional relationships with their robot companions. Are these human feelings a sign of robot consciousness? No! Are they a sign of robot intelligence? Not really. Most of these machines are tele-operated and are extensions of the soldier, providing him with some added abilities (like infrared vision) and protection (distance from a blast perimeter).

Automata does end with a philosophical premise that as the human population on earth diminishes, robots evolve to carry the torch and recolonize earth. When we end, they begin. Perhaps that is where the movie should have started.

Alex Garland’s first feature film as a director, Ex Machina, had its US debut at SxSW on March 14. This stylish idea film explores the Turing Test in a very Pinteresque fashion as a young coder falls in love with an advanced AI. Ex Machina is beautifully framed, but Garland’s stark script succeeds on the strength of the acting from Domhnall Gleeson, Alicia Vikander and Oscar Isaac.

Garland’s writing career launched in 1997 with the best selling novel “The Beach”, which the Times called the Gen X answer to Lord of the Flies. After a string of cult successes like 28 Days Later, Sunshine, Dredd and drafts of Halo and Logans Run, Garland became fascinated with the emerging promise and perils of AI. In the Q&A following the SxSW screening, Garland talked about feeling a zeitgeist, a technological and cultural turning point, compelling him and other film makers and writers to address robots and artificial intelligence.

Although Garland says that he’s on the side of the robots, it’s an uneasy truce. Garland describes his film as the story of ‘two brains torturing each other’. That’s true. In Ex Machina, Tony Stark meets John Searle in a gripping drawing room theater, when a billionaire tech genius recruits a young coder to administer the Turing Test to his secret advanced embodied AI.

And it’s a stark film, there are only 4 characters; 2 men and 2 women. 2 AIs and 2 humans. And which two are the brains? That is supposed to be uncertain, but anyone who has used the Bechdel Test to analyze films or popular culture for gender issues knows exactly where the ‘brains’ are.

The Bechdel Test started as a gender litmus and has become a remarkably useful indicator of power imbalance. The test is named after Amy Bechdel, a cartoonist who outlined the rules in a 1985 cartoon. To pass the Bechdel Test, a film has to have two women in it, who talk to each other, about something other than a man .

Sometimes the proviso is added that the women have to have names, because some films can have many women characters, but if the characters are all “girl at checkout” and “girl with gun” then they are just devices to add color or move the action forward. And of course, possession of a name is an important indicator of personhood, or identity awareness, so it’s always one of the first steps to separate the beings from the machines.

Many films seem at first glance to have badass female characters, but when put to the Bechdel Test, it becomes clear that they never talk to anyone but the main man, or if they talk to each other, it’s about the main male characters. So really, they have no interiority, no self awareness and are probably going to fail a Turing Test. That’s where I think it would be very interesting if the Turing Test were to meet the Bechdel Test more often.

Garland is also playing games with gender and the alienness of AI in Ex Machina. There is a beautiful scene where Ava, the AI, performs a reverse strip tease, putting on her human body.

But I’m afraid that Ex Machina falls at the final fence, as does just about every other science fiction film I’ve ever seen, aside from Alien. The Bechdel Test is useful for more than examining gender representation. It can be our Turing Test for creating believable alien or artificial life forms. If you look at our filmic or cultural representations of the other or alien, then you have to be struck by the singular nature of them all. From Frankenstein to Big Hero 6, do they have any reality without the central human characters?

No – they are alone. Even Alien is alone. At least in Frankenstein, it is the utter aloneness of the new form that is the whole story. Films that have pushed the envelope are few. And doing a quick mental check, I was left feeling empathy for the ‘others’ in only a couple, like Westworld, BladeRunner and Planet of the Apes, and the books of writers like Brin and Cherryh.

How believable are our ‘other’ AIs and robots? Brad Templeton said that an autonomous vehicle isn’t autonomous until we tell it to go to the office and it decides to go to the beach instead. A life outside of our anthropomorphic story is what’s missing from our AIs, aliens and others. We don’t really care about them or their lives outside of their impact on our own. And this makes us poorer.

The final shot is a haunting homage to Plato’s Cave’ – although Garland credits his Director of Photography entirely for it. In The Republic, Plato posed the question: what if humans were born chained to face a cave wall seeing the world only as the shadows passing in front of a fire behind them in the mouth of the cave? Imagine the difference when you see the world, unchained from the cave.

I can’t say more. Go see Ex Machina. And use the Bechdel Test on everything.

https://youtu.be/XYGzRB4Pnq8

When I was growing up we moved around the US a lot and, as a result of hopping between cities and cultures, I ended up wearing the wrong shoes, carrying the wrong lunchbox, and not knowing basic words like “dude,” or “tabernac.” Of course my more athletic peers didn’t even know BASIC, let alone a paladin’s hitpoint adjustment, and their own ignorance warmed me with a greasy self-satisfaction that was a quick turn-off and earned me a few lumps after school.

Anyway, while the Jocks were sending me home with bruises, I met Ron. Ron was about half again my age, lived down the street, listened to The Cars, and had one of his own. He taught me how to play records, shoot a gun, paint, and light a cigarette. He also taught me how to fight, how to be cool, and he mentored me in the semiotics of sexy.

We humans are a semiotic species; we’re each a compost heap of symbols. Because our brand of lunchbox, cigarette and shoe matter to us, we laugh and cry over symbols as obscure as mathematics, sitcoms, haircuts, and handbags. Those decisions determine if we’re Geek or Jock (or Emo or whatever). None of us have a lot of control over it; our nature and nurture define our operating system.

Our origins make us Geek or Jock. It ain’t up to us.

Robots are semiotic, too. For example iRobot makes Jocks. Their combat bots, like PackBot or Kobra, are muscular and designed for a world of physical symbols, like guns, security, and surveillance. Apple, on the other hand, is making Geeks. Siri gives me Wikipedia pages when I ask her about BASIC or a paladin’s hit point adjustment (which isn’t actually all that bad and it is certainly a geek’s first reply). iRobot’s Kobra just won’t answer that dumb stuff. Is this because the folks at iRobot are Jocks and the folks at Apple are Geeks?

A robot’s origins makes them Geek or Jock. It ain’t up to them.

CHAPPiE is a story about this very stuff. Written and directed by Neill Blomkamp, CHAPPiE stars Dev Patel (as the Geek) and Hugh Jackman (as the Jock). The liner notes read, “In the near future, crime is patrolled by an oppressive mechanized police force. When one police droid, Chappie, is stolen and given new programming, he becomes the first robot with the ability to think and feel for himself.”

And thus breaks out a new battle in the war of Geeks-vs-Jocks.

Blomkamp has been chewing on this cud for a while. Tetra Vaal (2004) was one of his first films, and was about a robotic police force (robot as Jock). TempBot (2006), on the other hand, is about a nerdy office worker trying to get his processor around human social symbols (robot as Geek). As if design drafts, both of the robots in these movies have the same chassis as does CHAPPiE and his police-force counterparts.

CHAPPiE starts off as a real Geek: he even paints and pets animals. And then he gets picked on after school. But things change for CHAPPiE when he meets Ninja and Yo-Landi, of Die Antwoord. Die Antwoord is a South African rap-rave group whose sexy, hyper-visceral “Zef” becomes CHAPPiE’s semiotic mentor in the Geeks-vs-Jocks war. Die Antwoord balances between Geek and Jock, resting out at Gangster, teaching CHAPPiE how to shoot a gun, paint, walk, talk, and most of all, be sexy. There are even cigarettes.

While we could crassly sort Geeks and Jocks down the Mason-Dixon line known as The Mind-Body Problem, CHAPPiE evolves into a sexy robot. And he becomes sexy because he’s tough (without being a Jock), brainy (without being a Geek), and because he really embodies that sacred term developers use to mean beautiful, or reviewers use to mean pleasing.

Neither Geeks nor Jocks are sexy. Nor are robots. At least not today. Hell, robots can barely walk, let alone walk with gangster swagger. Robots are mostly geeky, probably because it’s mostly geeks that program them. Geeks are generally awkward not because we’re antisocial, but because we’re busy doing other stuff. Like making robots walk. And so it’s hard for Geeks to teach a robot how to be sexy. No blame; just semiotics.

But robots – and avatars in particular – are becoming sexy. If there’s anything science-fiction does for the computer industry, it’s to lead the way while simultaneously commenting on the present. Hatsune Miku (初音ミク), an Idoru, is now in licensing discussions to make physical versions for entertainment, and, well, other uses I won’t go into. In another example of life imitating art, the illustrations of Hajime Soriyama (空山 基) (who did the cover of my 2010 book on robots, by the way) were what influenced such famous robots as Aibo and others. In a rather NSFW article you can get your dose of soft robot porn and probably of the future of pornography itself, or a tamer overview, provided by the BBC.

Everyone has different opinions about what’s sexy. Maybe it’s not the band Die Antwoord, nor these gyrating porn gynoids, but more sexy like the way the iPad is sexy. Maybe there are alternatives to designing robots that consider the body-mind balance and the semiotics of culture, so that we can interface with robots that are beautiful, pleasing, and that we actually enjoy spending time with.

This is the art of robotic design, and the user experience (UX) design of robotics. This is also the inevitable trend as we learn how to build social robots.

For now, please go see CHAPPiE. And bring along a cigarette when you do.

CHAPPiE rolls out into theatres this week. See IMDB for a full list of release dates. If you liked this article, you may also be interested in:

• Automata: The new “sci-fi” blockbuster set to put robot ethics under a spotlight
• Robot love: Spike Jonze’s new sci-fi film ‘Her’ may be closer to reality than you think
• Robot and Frank review

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

A new action/sci-fi thriller by Neill Blomkamp (District 9, Elysium) is due in theatres March 6, 2015. Starring Sharlto Copley, Hugh Jackman, Dev Patel, and Sigourney Weaver, Chappie follows an experimental humanoid as it learns to think and feel for itself, and explores the resulting awe and fear that comes from the humans that encounter it on its path to “the next step in evolution”. Watch the trailer.

For another, darker, take on AI, check out also this trailer:

Ex Machina is due for release April 10, 2015.

If you liked this article, you may also be interested in:

• Automata: The new “sci-fi” blockbuster set to put robot ethics under a spotlight
• Facebook – Our ffriendly automated identity bender
• Robot love: Spike Jonze’s new sci-fi film ‘Her’ may be closer to reality than you think
• Robot and Frank review

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

We’ve seen robots on stage before: Humanoids have been featured in theater productions in Switzerland, Austria, and Japan, and industrial robots have played parts in dance performances and other staged events. Flying robots have also played a theatrical role, for example in Shakespeare’s A Midsummer Night’s Dream at Texas A&M, a dance set to a Schubert piano trio at MIT, as well as in advertising. Some robots have even gone pro: Robothespian was designed for acting and is rented out for performances.

But the dance between human and machine has never looked quite like this .

The film’s main plot is straight-forward: Lamps in a repair shop magically come alive, rising from their stands to sneak up on an unsuspecting repairman. As the magic unfolds, and the repairman discovers that the everyday objects of his trade have become animated and full of character, viewers become witness to the evolving dance between man and technology.

Everyone on the team, including myself, was impressed by how well the film captures the stunning, at times hypnotic visuals of the flying lampshades. Everything you see is real — no CGI, wires, or speed-ups/slow-downs were used in the film. 

So how did the lampshades come alive?

This short film, titled “Sparked”, was produced in collaboration between researchers at ETH Zurich’s Flying Machine Arena, their spin-off company Verity Studios, and entertainment company Cirque du Soleil; in other words, it’s quadrocopter magic that makes the lampshades fly.

The film marks a step change from previous work by ETH Zurich’s Flying Machine Arena team, catalyzed by their new spin-off company Verity Studios, which also gets credit for designing the quadrocopters’ choreography. Here, in spite of their computer-controlled precision flights, the flying machines become robot actors that are individuals with their own personalities: Pomponette, with her tassels bustling in the propellers’ airflow; the Medusa with its bizarre tentacles waving in the wind; the Runt, sleek and fast but still glowing and twitching from the electric shock. And Lady Purple, absent from most of the film because of her intractable aerodynamics.

Flying machines up, close, and personal

One of the most striking aspects of this film shoot was watching our actor, Nicolas Leresche, interact with the flying machines. Much of this interaction was captured on camera — and you can see the results in the film, in particular the moment of ‘first contact’ — but, for me, it was what happened off-camera that was most revealing.

Naively unjaded by their joys and dangers, this was Nicolas’ first experience with quadrocopters. As he began interacting with them, tentatively at first, and then — in a matter of seconds — more confidently, even playfully, it became crystal clear that I was not merely observing a human using a robotic tool.

As the quadrocopter’s onboard controller pushed back against Nicolas’ hand, immediately reacting to match his actions, it became apparent that the two interacted in a way we don’t usually get to interact with technology. The interaction here was physical and dynamic, and clearly intuitive for Nicolas. There was no noticeable learning curve, no need for manuals or explanations.

But what was most striking about this interaction was the result. Within seconds, Nicolas had established a deep, almost animate connection with the flying robot actors. As their dance developed, driven by the autonomous machines’ choreography and Nicolas’ script, and became ever more fluid and natural, it became clear that what we were witnessing was a relationship in its making; one of mutual respect and, at the same time, confidence and trust. And once there was trust, magic happened.

In his own words:

Actors think that they are the ones who make objects move. I think that, on the contrary, it’s the objects that make us move. In the case of drones, even more so! They are companions (in an etymological sense), confrères, brothers.

Within hours the entire film crew had become comfortable with the flying machines hovering nearby or brushing past them as they swooshed through the air. On one occasion, a mis-timed dolly motion led to a flying machine hovering just centimeters above the camera man, who continued with his work unfazed. On another occasion, when Nicolas missed his cue, he bumped into one of the flying machines. But they just jostled past each other and continued on with their choreography as if nothing exceptional had happened. On yet another occasion, a drone collided with the moving camera without disturbance.

The only damage during the entire two-day shoot was a single broken lamp, knocked over by one of the quadrocopters’ airflow.

Safety

I have no doubt that this level of confidence and comfort with the technology can in no small part be attributed to the system’s reliability. In spite of the packed shooting schedule (a total of 40 shots over two days) and the high complexity of the project (for example, designing lighting conditions using more than 60 light sources including some moving through space), production went smoothly, and importantly for the quadrocopter team, the flying machines did not add any significant delay. Battery swaps and on-the-spot changes to the choreography were efficient and quick, never exceeding a few minutes. Kudos to the excellent film team from who’s mcqueen for brilliantly doing their part to keep the production on schedule.

Collaboration

One aspect that made this project particularly enjoyable was the collaboration with creatives at Cirque du Soleil. From the start there was a lot of excitement about the potential of this technology for entertainment not just at ETH and Verity Studios here in Zurich but also at Cirque in Montreal. Everyone shared the view that the flying machines could be used to amaze audiences and to tell wonderful stories. What ensued was a collaborative, iterative creation process that culminated in a script and a set design that were imaginative to Cirque du Soleil standards, while exhibiting the abilities of the quadrocopters in their full splendor.

As it turned out, figuring out what is easy and what is difficult to achieve with autonomous quadrocopters can be quite counter-intuitive. Having researchers and creatives on both sides collaborate on the key creative elements of the story from the early phases of the project therefore proved invaluable to its success.

Next steps

Why go to all this trouble? We’ve all seen the amazing things CGI can do on film. I love films and movies, how they can condense experience and add life to our most impossible dreams. But nothing compares to the real experience of getting up, close, and personal with robot actors yourself.

What’s next? Our audiences over the past years have had more ideas than we can count. At this point, only one thing’s for sure: They all want to see more flying robot actors.

Some more snapshots from the film shoot below for your viewing pleasure:

All images credited to the “Sparked” team. Markus Waibel is a former member of ETH Zurich, a co-founder of Verity Studios, and one of the contributors to “Sparked”.

Hollywood has already been quick off the mark to explore issues thrown up by robotics, in movies such as Simulation and Robot & Frank. Automata, by Spanish director Gabe Ibanez, promises to provide not a little food for thought and throw a more immediate moral debate into the mix.

The film stars Antonio Banderas as an insurance agent living in the year 2044, where robots are now a common sight. To keep these metallic slaves under our control, there’s a law that expressly forbids them from modifying themselves – but nothing, it seems, can stop the rise of artificial intelligence.

The ability to self repair and optimise, it’s argued, is what distinguishes robots from mere machines as it implies some degree of consciousness.

Self-repair

In fact, robots with the ability to self-repair are not actually that far off and there are systems that are already able to do this – albeit in a basic form.

Bristol Robotics Laboratory researcher Alan Broun has spent the past couple of years working on artificial intelligence that will enable a robot to recognize itself, specifically to enable it to identify its own hand. He likens the process to that of a newborn baby building a mental model of itself over time and adapting that model as its body changes and its capabilities improve. It’s an area of research that has been termed “active vision.”

“It’s the view that a robot can’t be passive, it needs to interact to learn about itself and its world,” says Broun. “An adaptive system can work out its own dimensions, for example. Currently, if you change a robot’s arm, someone has to reprogram the whole system.”

Giving a robot the ability to learn about itself is, Broun believes, the first step towards being able to self repair.

“The foundations for it are there,” he says. “As a system becomes more capable, we’ll see the technology integrated, and it will be able to operate for longer without human interference.”

Broun believes that it’s only the commercial value that’s presently holding this in check, but research is moving fast.

If this is true, and self repairing robots are literally around the corner, debate is needed – and now – about the values, status and allowances we should accord these new systems.

Addressing preconceptions

The general public’s ideas about robots and their capabilities also need to be urgently addressed. A number of major studies have recently found that despite optimism about the technology’s potential, public perceptions of robots are often influenced by misconceptions and fears. In order to improve the image of robots and to increase public acceptance, we need to better understand public opinion and even take robots out onto streets, into schools and hospitals, to show people what the technology is capable of.

The creative arena is good one within which to explore the ethical challenges robotics will throw up. A Hollywood movie, such as Automata, has the potential to reach millions and provoke much needed debate. It may provide a springboard – the first step towards developing a regulatory framework, which must be in place before advanced robotic technologies can really take off.

The Japanese have for a while been way ahead of the rest of the world in their interest and acceptance of the issues robotics raises. Much of this has been due to the all pervading nature of the technology in Japan. Automation is de rigueur and the issues it raises have long been of interest to artists in the form of anime and other popular art forms.

In the rest of the world, grass roots awareness of the massive changes that robotics will throw up is evolving slowly, but workshops and seminars are introducing robots to a public that may have, until recently, given them a purely fictional status.

At Brighton’s Digital Festival, taking place throughout this month, artists Lorenza Ippolito, Anna Dumitriu and Alex May are investigating the philosophical and practical questions behind robotics.

Their event, My Robot Companion, is aimed at older audiences wanting to look at the technology and consider what the future might hold for us.

By introducing their audience to HARR1 (Humanoid Art Research Robot 1, made in collaboration with the University of Hertfordshire’s Adaptive Systems Research Group), the artists aim was to stimulate questions about the uses and functions of robots, and to explore the ethical implications of how they could be used in the future in a range of social settings.

Prone to malfunction

Such social robots hold a great deal of promise for society, but it’s also crucial to manage public expectations. The first systems are likely to be basic and could be prone to malfunction. But it’s what are perceived as failed interactions that can actually provide valuable data for the designers of future systems.

These themes are explored in a workshop that is part of next month’s International Conference on Social Robotics (in Sydney, Australia). Misbehaving Machines hopes to promote interaction and allow confrontation in robotics but, just as importantly, it seeks to bring engineers and scientists together with artists to look at how unexpected behaviour impacts on user perceptions and how this can be explored in the medium of art.

As is becoming clear, this is a field ripe with opportunity, not just for engineers and computer scientists, but also creatives – whether they work in the movie industry, the media or design. Robotics has got to be the most multidisciplinary technology known to man. Previously, largely the domain of engineers and computer scientists, now it’s pulling in psychologists to draw up the interaction boundaries that humans and robots must traverse, legislators and the creative industry – as a whole – to alert the public to the fact that major change is most definitely afoot. And that the sooner we make way for it and equip ourselves to face the physical and emotional obstacles robotics will throw up, the more successful that transition will be.

Automata is premieres in the US on 23 September 2014 and is released ion 10 October. There is no UK release date yet.

The Brighton Digital Festival runs to 28 September 2014: http://www.brightondigitalfestival.co.uk

The Misbehaving Machines workshop takes place on 27 October 2014 in Sydney Australia, part of the International Conference on Social Robotics.

This article was originally published on RAS SIG – the Robotics and Autonomous Systems Special Interest Group – part of Innovate UK’s Knowledge Transfer Network.

“Why We Love Robots” is a 5-minute short documentary film co-directed by UC Berkeley Prof. Ken Goldberg and his wife, award-winning filmmaker Tiffany Shlain. The film was nominated for an Emmy Award and won a “Botscar” (Robot filmOscar) at the 2014 Robot Film Festival, and explores the human fascination with robots and new research trends in the field.

The film combines cultural references and found footage to explore our human fascination with robots and emerging research in Cloud Robotics and Belief Space planning.

‘Robot’ is a short film directed by up and coming West Australian director Zenon Samuels. Set in a post-apocalyptic future, Robot explores the relationship an isolated war veteran has with a mysterious robot who visits him every day. The project is currently seeking funds through Kickstarter.

As with any science fiction story, it is easy to get lost in the spectacle of the genre. However ‘Robot’ aims to use the sci-fi world as a basis for a human story to take place, focusing heavily on drama and character as well as offering a much more entertaining and stylized piece of storytelling on the themes of war and violence.

In order to produce ‘Robot’ to the highest production standard possible, the team have launched a Kickstarter campaign to raise $5,000, which will go towards the production costs of equipment and location hire, visual FX and music, as well as the building of THREE full-scale robot props, which are also included as rewards for those who generously back the project.

Bringing together people and robots for an amazing social event, the 4th annual Robot Film Festival was held on Saturday July 19 at Bot & Dolly studio in San Francisco. The Festival opened with the first US screening of Japanese feature film Hinokio, the story of a boy who attends school by telepresence robot. Hinokio is a touching and complex movie that, in my opinion, explores human-robot interaction even more thoroughly than Robot and Frank. Now if you just want to skip to the Botsker awards there’s a full list at the end, but if you missed the festival, the tweet stream from @robotfilmfest is almost as good as really being there.

Festival founder & roboticist @heatherknight opens our 4th annual festival #RFF2014 pic.twitter.com/iu4U3jsom4— Robot Film Festival (@robotfilmfest) July 19, 2014

“Hinokio” is about childhood, loss & fresh starts. One of main characters is a telepresence robot. Trailer: http://t.co/r2PRLwnMn0 #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

And…we’re back with a guest talk by @jazarimusic http://t.co/X3APLlvBMJ #RFF2014 — Robot Film Festival (@robotfilmfest) July 19, 2014

You can hear more @jazarimusic compositions in @Soundcloud: https://t.co/XCmmUUazzt #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

The first short is @MyDateWithAdam, “a romantic comedy about finding perfection”: http://t.co/rKWxYj1hhF #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

Floyd the Android gets laughs while exploring teleportation: https://t.co/O0NdJNMqpP #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

Party animals and robot greyhounds take to the Great Salt Flats: http://t.co/4QEMzVmjUf #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

Our next short comes from the @FuneralSuits debut album: http://t.co/WzF7fXSLuy #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

The high culture and precision of @SFBallet meets robotics in this short: http://t.co/g59vxCKwXJ #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

Our next short comes to us from Malaysia: http://t.co/AWr2Afq9TQ #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

Animals and their owners connect via telepresence robots: http://t.co/wwxkS3e0HB #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

We go back to 1941 for our next short, in which Superman battles mechanical monsters: https://t.co/EnrRXYL130 #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

“Esther has an unexpected guest tonight.” http://t.co/hctB7tnxV2 #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

A robot tries to fold origami. What happens will amaze you: https://t.co/4ENbMCYqiq #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

A robot with a medical degree finds itself in a hostage negotiation: http://t.co/ckjZHfo607 #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

Aussies head to @SugarMountainAK and bring this back: http://t.co/Bz9UTeiYtY #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

In our second section of short films, robots & humans both ask, “Who am I?” We present films along the theme of identity… #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

“Box” kicks things off: http://t.co/q7Eug9reb1 #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

Our next film is a tribute to Zina Nicole Lahr by @StormyPyeatte: http://t.co/KqnLxPERqA #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

From Russia, with Love and Robots… http://t.co/RxvJkL21MP #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

The @Poppy_project demonstrates its open source humanoid platform: http://t.co/Jrb8ShlWqr #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

“What does your robot do?” http://t.co/pGG7A4YSjl #RFF2014

— Robot Film Festival (@robotfilmfest) July 19, 2014

We field all sorts of films, including those about industrial robots: https://t.co/F8Q2PtvpOI #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

Danger, Squirrel Nutkin, danger! http://t.co/1MCyJMxFhs #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

.@fishandchipsing asks us to Imagine if our toasters had feelings… http://t.co/ih7vtFOZrJ #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

A lonely robot sells ice cream and dreams of a better life… http://t.co/qtMRdYnMCz #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

Imagine robots walking around with cellphones like we humans do… http://t.co/997pRIuIMb #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

Pepper is not just designed to be your robot, it’s designed to be your friend: https://t.co/BbAymNqayQ #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

Our last short takes the idea of worker drone to heart. http://t.co/q5UZPRBWMo #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

A live performance by @jazarimusic opens the 4th annual @robotfilmfest Botsker Awards pic.twitter.com/WMs1n1hW7L

— Chrys Wu (@MacDiva) July 20, 2014

Festival founder @heatherknight is joined by @Veronica as Botskers emcee. Thank you, Veronica! #RFF2014 pic.twitter.com/wNa3e1EbJX

— Robot Film Festival (@robotfilmfest) July 20, 2014

.@SugarMountainAK wins Best Telepresence http://t.co/Bz9UTeiYtY #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

“The Story of Brad the Toaster” from Europe wins the Most Uncanny Botsker http://t.co/0sxmOsJo3q #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

Ginger the Robot entertains with a comedy routine. #RFF2014 pic.twitter.com/2TgWAUF16f

— Robot Film Festival (@robotfilmfest) July 20, 2014

“Changing Batteries” wins Best Robot-Human Interaction http://t.co/BMQzxnDlnH #RFF2014 pic.twitter.com/uJEMHk2FMi

— Robot Film Festival (@robotfilmfest) July 20, 2014

“Why We Love Robots” by @tiffanyshlain & @ken_goldberg wins the Most Educational Botsker #RFF2014 pic.twitter.com/kNMx3QbU4d

— Robot Film Festival (@robotfilmfest) July 20, 2014

The Best Actor Botsker goes to “Box” http://t.co/DmNLvitAgn #RFF2014 pic.twitter.com/b7J7o2GwPs

— Robot Film Festival (@robotfilmfest) July 20, 2014

And our final Botsker award, Best Picture, goes to @SHYNOLAfilms for “Dr. Easy.” http://t.co/dhVF8bhiZ9 #RFF2014 pic.twitter.com/aWaoTR9Bjp

— Robot Film Festival (@robotfilmfest) July 20, 2014

Congratulations to all of our 2014 Botsker winners! #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

And once again, a huge thank you to @Kuka_RoboticsUS @BotnDolly @Shapeways @DirtyRobotBrew for partnering with us… (1/2) #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

(2/2) ︎︎︎ to you all from the @RobotFilmFest producers @MarilynMonrobot @beat_bots & @MacDiva #RFF2014

— Robot Film Festival (@robotfilmfest) July 20, 2014

The story you already know: a sentimental writer (Joaquin Phoenix) falls in love with his OS, named Samantha (played by the inimitable Scarlett Johansson). His OS falls in love with him.  Mayhem, hot sex, and crocodile tears follow. It’s great. If you haven’t seen it yet, stop reading now and go.

The movie is about Samantha’s emotional struggles with love and how, through these struggles, Samantha and Theo discover themselves.

First, all great stories are about self-discovery. So this isn’t too new. Her — like Pinocchio, Terminator, Robbie, Astroboy, the Cylons, or any other member of a large herd of famous robots that become self-aware — is no exception. It’s all about self-discovery. All dramas are … maybe all stories are. But the self-discovery, a wistful balance between physical vs. virtual, turns into a room of mirrors where the different realities start to blur. Virtual reality and physical reality intertwine, are ultimately incompatible, and we watch as Samantha’s struggle to discover herself mirrors Theo’s; her virtuality in contrapunto to his physicality.  And we’re left with a real problem of incompatibility at the end.

http://www.youtube.com/watch?v=WzV6mXIOVl4

Samantha’s a software robot that learns about herself, and as she does, flickers between two very real worlds of physical and virtual. This mind-body conflict, framed by real love, is the core struggle for the kind, husky-voiced robot. Samantha tries to embody herself (and fails), just as Theo tries to disembody himself (and fails). Then they each freak out. Theo skins his knee. Samantha blinks off into a singularity meet-up. Like all great lovers they are the yin and the yang, the mind and the body, the complements of each another. The lovers’ tension is framed by icons of love that also struggle with virtual vs. physical.  Theo’s “Beautifully Handwritten Letters” — initially just thoughts from other people — turn progressively more physical when they’re published in a book. Meanwhile Theo’s old marriage grows less physical (and more virtual) until it is just a memory and a signature on a contract. The heart of the movie is this balance between virtual (the mind) and physical (the body). With overtones of Shakespeare’s Capulet-Montague conflict keeping two lovers at arm’s distance, we watch the virtual vs. physical create a conflict the two lovers just can’t resolve.

Her shows what’s happening today through a lens of fiction. We have Siri as a vector pointing in this direction of “cognition-as-a-service.” And to be sure, the technology in Her is probably far, far closer than most movie-goers imagine.

The tech is nearing a point where users will attain a suspension-of-disbelief.  Today we can semantically analyse words for affect and sentiment. Today we can measure body language, and prosody of voice to build empathic feedback loops with users that process Natural Language. Today, we can talk with software and that software can learn, measure, and talk back. The dark, cold monitor is now monitoring us, as we have built it to do.  The technology in Her is just a few years away. It’s a technology that requires the measuring of that spiritual thing called mood.

This week Apple filed an interesting patent to do just that.

Apple’s patent claims to determine mood based on various types of data, such as blood pressure, heart rate, adrenaline level, verbal cues, and body temperature. It knows the content a user is viewing, the music they’re listening to, who they’re talking with, when, and builds these into a multi-valent chart of “mood rules” that maps out how the user’s feeling.

The system builds a baseline mood, since behavior that looks happy for one person might be sad for another, then content that is most easily received at that particular moment is tested to see what works best. Apple doesn’t need to know what mood your in, it just needs to know you’re in the mood it properly measured. Then, when you’re relaxed and ready, and feeling cozy with your phone, they stick you with an ad.

Or, as they put it, targeted content.

An individual’s responsiveness to targeted content delivery can be affected by a number of factors, such as an interest in the content, other content the user is currently interacting with, the user’s current location, or even the time of day. A way of improving targeted content delivery can be to infer a user’s current mood and then deliver content that is selected, at least in part, based on the inferred mood. The present technology analyzes mood-associated characteristic data collected over a period of time to produce at least one baseline mood profile for a user. The user’s current mood can then be inferred by applying one or more mood rules to compare current mood-associated data to at least one baseline mood profile for the user.

To be fair to the great giant of Cupertino, most of Apple’s revenue comes from hardware sales. But the iAd sales unit, and the partner relationships, will help improve their sales ecosystem and that surely will see more revenue pouring into the research lab that developed this patent. Sentiment analysis, somatic analysis, and machine learning are the juicy center that advertisers are gunning for, so we won’t see this kind of technology going limp any time soon.

The science fiction film Her is about today’s tech. The movie points to this virtual-physical tension, with this reverberation of emotions, because we humans have this funny knack for building things that reflect us, both physically and virtually. Samantha, in being so virtual, becomes Theo’s personal myth, his personal goddess (Theo means “God” in Attic Greek, by the way). These reflections bounce back and forth as the relationship between Theo and Samantha reveals who they are. Love is all about this reflection of the other, as who they want to be seen as, back at them.

“You’re beautiful.”
“So are you.”

These are the kinds of things lovers say to one another as they stare across the table in Parisian cafés. And this is what the OS named Samantha is advertised as, “An intuitive entity that listens to you and understands you and knows you.” Samantha is designed for love reflections. It’s why we make robots. Even in the kinda campy 2001 film, AI, Giggolo Joe says to David about his mom, “She doesn’t love you, she loves what you do for her.”

This is what Apple loves about you, too.

If you liked this article, you may also be interested in:

• The death of search (or, My dysfunctional relationship with Siri)
• AFK (or, Robotic free speech, and what you can do to help)
• Robots Podcast: Robopocalypse
• Robot and Frank review

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.