A new system called Data Civilizer automatically finds connections among many different data tables and allows users to perform database-style queries across all of them. The results of the queries can then be saved as new, orderly data sets that may draw information from dozens or even thousands of different tables.

The age of big data has seen a host of new techniques for analyzing large data sets. But before any of those techniques can be applied, the target data has to be aggregated, organized, and cleaned up.

That turns out to be a shockingly time-consuming task. In a 2016 survey, 80 data scientists told the company CrowdFlower that, on average, they spent 80 percent of their time collecting and organizing data and only 20 percent analyzing it.

An international team of computer scientists hopes to change that, with a new system called Data Civilizer, which automatically finds connections among many different data tables and allows users to perform database-style queries across all of them. The results of the queries can then be saved as new, orderly data sets that may draw information from dozens or even thousands of different tables.

“Modern organizations have many thousands of data sets spread across files, spreadsheets, databases, data lakes, and other software systems,” says Sam Madden, an MIT professor of electrical engineering and computer science and faculty director of MIT’s bigdata@CSAIL initiative. “Civilizer helps analysts in these organizations quickly find data sets that contain information that is relevant to them and, more importantly, combine related data sets together to create new, unified data sets that consolidate data of interest for some analysis.”

The researchers presented their system last week at the Conference on Innovative Data Systems Research. The lead authors on the paper are Dong Deng and Raul Castro Fernandez, both postdocs at MIT’s Computer Science and Artificial Intelligence Laboratory; Madden is one of the senior authors. They’re joined by six other researchers from Technical University of Berlin, Nanyang Technological University, the University of Waterloo, and the Qatar Computing Research Institute. Although he’s not a co-author, MIT adjunct professor of electrical engineering and computer science Michael Stonebraker, who in 2014 won the Turing Award — the highest honor in computer science — contributed to the work as well.

Pairs and permutations

Data Civilizer assumes that the data it’s consolidating is arranged in tables. As Madden explains, in the database community, there’s a sizable literature on automatically converting data to tabular form, so that wasn’t the focus of the new research. Similarly, while the prototype of the system can extract tabular data from several different types of files, getting it to work with every conceivable spreadsheet or database program was not the researchers’ immediate priority. “That part is engineering,” Madden says.

The system begins by analyzing every column of every table at its disposal. First, it produces a statistical summary of the data in each column. For numerical data, that might include a distribution of the frequency with which different values occur; the range of values; and the “cardinality” of the values, or the number of different values the column contains. For textual data, a summary would include a list of the most frequently occurring words in the column and the number of different words. Data Civilizer also keeps a master index of every word occurring in every table and the tables that contain it.

Then the system compares all of the column summaries against each other, identifying pairs of columns that appear to have commonalities — similar data ranges, similar sets of words, and the like. It assigns every pair of columns a similarity score and, on that basis, produces a map, rather like a network diagram, that traces out the connections between individual columns and between the tables that contain them.

Tracing a path

A user can then compose a query and, on the fly, Data Civilizer will traverse the map to find related data. Suppose, for instance, a pharmaceutical company has hundreds of tables that refer to a drug by its brand name, hundreds that refer to its chemical compound, and a handful that use an in-house ID number. Now suppose that the ID number and the brand name never show up in the same table, but there’s at least one table linking the ID number and the chemical compound, and one linking the chemical compound and the brand name. With Data Civilizer, a query on the brand name will also pull up data from tables that use just the ID number.

Some of the linkages identified by Data Civilizer may turn out to be spurious. But the user can discard data that don’t fit a query while keeping the rest. Once the data have been pruned, the user can save the results as their own data file.

“Data Civilizer is an interesting technology that potentially will help data scientists address an important problem that arises due to the increasing availability of data — identifying which data sets to include in an analysis,” says Iain Wallace, a senior informatics analyst at the drug company Merck. “The larger an organization, the more acute this problem becomes.”

“We are currently exploring how to use Civilizer as a harmonization layer on top of a variety of chemical-biology datasets,” Wallace continues. “These datasets typically link compounds, diseases, and targets together. One use case is to identify which table contains information about a specific compound and what additional information is available about that compound in other related datasets. Civilizer helps us by allowing full text search over all the columns and then identifying related columns automatically. By using Civilizer, we should be easily able to add additional data sources and update our analysis very quickly.”

Read the paper here.

The population of the scenic ski-resort Davos, nestled in the Swiss Alps, swelled by nearly +3,000 people between the 17^th and 20^th of January. World leaders, academics, business tycoons, press and interlopers of all varieties were drawn to the 2017 World Economic Forum (WEF) Annual Meeting. The WEF is the foremost creative force for engaging the world’s top leaders in collaborative activities to shape the global, regional and industry agendas for the coming year and beyond. Perhaps unsurprisingly given recent geopolitical events, the theme of this year’s forum was Responsive and Responsible Leadership.

With the onset of the fourth industrial revolution, increasingly discontented segments of society not experiencing congruous economic and social progress are in danger of existential uncertainty and exclusion. Responsive and Responsible Leadership entails inclusive development and equitable growth, both nationally and globally. It also involves working rapidly to close generational divides by exercising shared stewardship of those systems that are critical to our prosperity.

In the end, leaders from all walks of life at the Annual Meeting 2017 must be ready to react credibly and responsibly to societal and global concerns that have been neglected for too long.”

Developing last year’s theme—“The fourth industrial revolution”—this year’s luminaries posited questions, among many others, concerning incipient robotics and artificial intelligence technologies set to have a pronounced impact on the global economy and global consciousness alike. What can we learn from the first wave of AI? How can the humanitarian sector benefit from big data algorithms? How will drone technology change the face of warfare? Can AI and computational tech help foster responsive and responsible leadership? What are the downsides of technology in the fourth industrial revolution?

Enjoy a selection of tech-themed videos below.

And a bit about global science including big data, open source science and education.

MIT researchers have developed a new approach to training speech-recognition systems that doesn’t depend on transcription. Instead, their system analyzes correspondences between images and spoken descriptions of those images, as captured in a large collection of audio recordings. Image: MIT News

Speech recognition systems, such as those that convert speech to text on cellphones, are generally the result of machine learning. A computer pores through thousands or even millions of audio files and their transcriptions, and learns which acoustic features correspond to which typed words.

But transcribing recordings is costly, time-consuming work, which has limited speech recognition to a small subset of languages spoken in wealthy nations.

At the Neural Information Processing Systems conference this week, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) are presenting a new approach to training speech-recognition systems that doesn’t depend on transcription. Instead, their system analyzes correspondences between images and spoken descriptions of those images, as captured in a large collection of audio recordings. The system then learns which acoustic features of the recordings correlate with which image characteristics.

“The goal of this work is to try to get the machine to learn language more like the way humans do,” says Jim Glass, a senior research scientist at CSAIL and a co-author on the paper describing the new system. “The current methods that people use to train up speech recognizers are very supervised. You get an utterance, and you’re told what’s said. And you do this for a large body of data.

“Big advances have been made — Siri, Google — but it’s expensive to get those annotations, and people have thus focused on, really, the major languages of the world. There are 7,000 languages, and I think less than 2 percent have ASR [automatic speech recognition] capability, and probably nothing is going to be done to address the others. So if you’re trying to think about how technology can be beneficial for society at large, it’s interesting to think about what we need to do to change the current situation. And the approach we’ve been taking through the years is looking at what we can learn with less supervision.”

Joining Glass on the paper are first author David Harwath, a graduate student in electrical engineering and computer science (EECS) at MIT; and Antonio Torralba, an EECS professor.

Visual semantics

The version of the system reported in the new paper doesn’t correlate recorded speech with written text; instead, it correlates speech with groups of thematically related images. But that correlation could serve as the basis for others.

If, for instance, an utterance is associated with a particular class of images, and the images have text terms associated with them, it should be possible to find a likely transcription of the utterance, all without human intervention. Similarly, a class of images with associated text terms in different languages could provide a way to do automatic translation.

Conversely, text terms associated with similar clusters of images, such as, say, “storm” and “clouds,”  could be inferred to have related meanings. Because the system in some sense learns words’ meanings — the images associated with them — and not just their sounds, it has a wider range of potential applications than a standard speech recognition system.

To test their system, the researchers used a database of 1,000 images, each of which had a recording of a free-form verbal description associated with it. They would feed their system one of the recordings and ask it to retrieve the 10 images that best matched it. That set of 10 images would contain the correct one 31 percent of the time.

“I always emphasize that we’re just taking baby steps here and have a long way to go,” Glass says. “But it’s an encouraging start.”

The researchers trained their system on images from a huge database built by Torralba; Aude Oliva, a principal research scientist at CSAIL; and their students. Through Amazon’s Mechanical Turk crowdsourcing site, they hired people to describe the images verbally, using whatever phrasing came to mind, for about 10 to 20 seconds.

For an initial demonstration of the researchers’ approach, that kind of tailored data was necessary to ensure good results. But the ultimate aim is to train the system using digital video, with minimal human involvement. “I think this will extrapolate naturally to video,” Glass says.

Merging modalities

To build their system, the researchers used neural networks, machine-learning systems that approximately mimic the structure of the brain. Neural networks are composed of processing nodes that, like individual neurons, are capable of only very simple computations but are connected to each other in dense networks. Data is fed to a network’s input nodes, which modify it and feed it to other nodes, which modify it and feed it to still other nodes, and so on. When a neural network is being trained, it constantly modifies the operations executed by its nodes in order to improve its performance on a specified task.

The researchers’ network is, in effect, two separate networks: one that takes images as input and one that takes spectrograms, which represent audio signals as changes of amplitude, over time, in their component frequencies. The output of the top layer of each network is a 1,024-dimensional vector — a sequence of 1,024 numbers.

The final node in the network takes the dot product of the two vectors. That is, it multiplies the corresponding terms in the vectors together and adds them all up to produce a single number. During training, the networks had to try to maximize the dot product when the audio signal corresponded to an image and minimize it when it didn’t.

For every spectrogram that the researchers’ system analyzes, it can identify the points at which the dot-product peaks. In experiments, those peaks reliably picked out words that provided accurate image labels — “baseball,” for instance, in a photo of a baseball pitcher in action, or “grassy” and “field” for an image of a grassy field.

In ongoing work, the researchers have refined the system so that it can pick out spectrograms of individual words and identify just those regions of an image that correspond to them.

“Possibly, a baby learns to speak from its perception of the environment, a large part of which may be visual,” says Lin-shan Lee, a professor of electrical engineering and computer science at National Taiwan University. “Today, machines have started to mimic such a learning process. This work is one of the earliest efforts in this direction, and I was really impressed when I first learned of it.”

“Perhaps even more exciting is just the question of how much we can learn with deep neural networks,” adds Karen Livescu, an assistant professor at the Toyota Technological Institute at the University of Chicago. “The more the research community does with them, the more we realize that they can learn a lot from big piles of data. But it is hard to label big piles of data, so it’s really exciting that in this work, Harwath et al. are able to learn from unlabeled data. I am really curious to see how far they can take that.”

Read the paper here.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have devised a way to train neural networks so that they provide not only predictions and classifications but rationales for their decisions. Illustration: Christine Daniloff/MIT

In recent years, the best-performing systems in artificial-intelligence research have come courtesy of neural networks, which look for patterns in training data that yield useful predictions or classifications. A neural net might, for instance, be trained to recognize certain objects in digital images or to infer the topics of texts.

But neural nets are black boxes. After training, a network may be very good at classifying data, but even its creators will have no idea why. With visual data, it’s sometimes possible to automate experiments that determine which visual features a neural net is responding to. But text-processing systems tend to be more opaque.

At the Association for Computational Linguistics’ Conference on Empirical Methods in Natural Language Processing, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) will present a new way to train neural networks so that they provide not only predictions and classifications but rationales for their decisions.

“In real-world applications, sometimes people really want to know why the model makes the predictions it does,” says Tao Lei, an MIT graduate student in electrical engineering and computer science and first author on the new paper. “One major reason that doctors don’t trust machine-learning methods is that there’s no evidence.”

“It’s not only the medical domain,” adds Regina Barzilay, the Delta Electronics Professor of Electrical Engineering and Computer Science and Lei’s thesis advisor. “It’s in any domain where the cost of making the wrong prediction is very high. You need to justify why you did it.”

“There’s a broader aspect to this work, as well,” says Tommi Jaakkola, an MIT professor of electrical engineering and computer science and the third coauthor on the paper. “You may not want to just verify that the model is making the prediction in the right way; you might also want to exert some influence in terms of the types of predictions that it should make. How does a layperson communicate with a complex model that’s trained with algorithms that they know nothing about? They might be able to tell you about the rationale for a particular prediction. In that sense, it opens up a different way of communicating with the model.”

Virtual brains

Neural networks are so called because they mimic — approximately — the structure of the brain. They are composed of a large number of processing nodes that, like individual neurons, are capable of only very simple computations but are connected to each other in dense networks.

In a process referred to as “deep learning,” training data is fed to a network’s input nodes, which modify it and feed it to other nodes, which modify it and feed it to still other nodes, and so on. The values stored in the network’s output nodes are then correlated with the classification category that the network is trying to learn — such as the objects in an image, or the topic of an essay.

Over the course of the network’s training, the operations performed by the individual nodes are continuously modified to yield consistently good results across the whole set of training examples. By the end of the process, the computer scientists who programmed the network often have no idea what the nodes’ settings are. Even if they do, it can be very hard to translate that low-level information back into an intelligible description of the system’s decision-making process.

In the new paper, Lei, Barzilay, and Jaakkola specifically address neural nets trained on textual data. To enable interpretation of a neural net’s decisions, the CSAIL researchers divide the net into two modules. The first module extracts segments of text from the training data, and the segments are scored according to their length and their coherence: The shorter the segment, and the more of it that is drawn from strings of consecutive words, the higher its score.

The segments selected by the first module are then passed to the second module, which performs the prediction or classification task. The modules are trained together, and the goal of training is to maximize both the score of the extracted segments and the accuracy of prediction or classification.

One of the data sets on which the researchers tested their system is a group of reviews from a website where users evaluate different beers. The data set includes the raw text of the reviews and the corresponding ratings, using a five-star system, on each of three attributes: aroma, palate, and appearance.

What makes the data attractive to natural-language-processing researchers is that it’s also been annotated by hand, to indicate which sentences in the reviews correspond to which scores. For example, a review might consist of eight or nine sentences, and the annotator might have highlighted those that refer to the beer’s “tan-colored head about half an inch thick,” “signature Guinness smells,” and “lack of carbonation.” Each sentence is correlated with a different attribute rating.

Validation

As such, the data set provides an excellent test of the CSAIL researchers’ system. If the first module has extracted those three phrases, and the second module has correlated them with the correct ratings, then the system has identified the same basis for judgment that the human annotator did.

In experiments, the system’s agreement with the human annotations was 96 percent and 95 percent, respectively, for ratings of appearance and aroma, and 80 percent for the more nebulous concept of palate.

In the paper, the researchers also report testing their system on a database of free-form technical questions and answers, where the task is to determine whether a given question has been answered previously.

In unpublished work, they’ve applied it to thousands of pathology reports on breast biopsies, where it has learned to extract text explaining the bases for the pathologists’ diagnoses. They’re even using it to analyze mammograms, where the first module extracts sections of images rather than segments of text.

“There’s a lot of hype now — and rightly so — around deep learning, and specifically deep learning for natural-language processing,” says Byron Wallace, an assistant professor of computer and information science at Northeastern University. “But a big drawback for these models is that they’re often black boxes. Having a model that not only makes very accurate predictions but can also tell you why it’s making those predictions is a really important aim.”

“As it happens, we have a paper that’s similar in spirit being presented at the same conference,” Wallace adds. “I didn’t know at the time that Regina was working on this, and I actually think hers is better. In our approach, during the training process, while someone is telling us, for example, that a movie review is very positive, we assume that they’ll mark a sentence that gives you the rationale. In this way we train the deep-learning model to extract these rationales. But they don’t make this assumption, so their model works without using direct annotations with rationales, which is a very nice property.”

Talking Machines is in the process of raising funds to defray the cost of producing our first season and to help us start production on our second season. On the show we’ve talked about how we’ll use the money (to pay for studio time, editing, and the cost of travel to get our great interviews). But we haven’t gotten to the heart of the question yet: Do we even need journalism about machine learning?

We need journalism about machine learning, artificial intelligence, and data science desperately. Not just to calm the public conversation, which always seems to be full of hype on these topics, but to make sure that work in our field is sustainable. And no one is going to make the case for our industry unless we do it ourselves.

I live in Cambridge, MA. A lot of the people here are scientists or are training to enter the field. From the vantage point of Cambridge, the answer seems to be a resounding yes, we do need journalism about these topics, and Talking Machines is a way for those in the field to access each other’s ideas, and for those in training to get exposure to work they might not have heard of.

But not all of our listeners live in Cambridge, or come from an academic background. We get letters from all over the world saying that Talking Machines has allowed them to better understand ideas they’d like to use in their business, helped them talk with their data teams, or helped them make the right hire.

Most importantly though, not all of our listeners think they live a life that has anything to do with machine learning, or computer science .. or science at all. We started Talking Machines because we wanted to open the world of machine learning up to a wider audience, to help them understand the reality of research in the field and the industry, and how that impacts their lives in a real way on a daily basis.

The public conversation around machine learning (and by extension artificial intelligence) is filled with extreme hype, both positive and negative. These extremes have lead to a crippling pattern of “winters” where interest, activity, and funding in the field dries up. If we present the reality of what is happening in the field in a way that invites the public to be part of the conversation, that arms them with the knowledge they need to participate, and we will create a more sustainable industry for ourselves. For our own benefit, and for the good of those who use the tools we make, it’s our responsibility to play a bigger role than we have before in the public conversation.

Talking Machines does just that. By introducing machine learning to a wide audience in a way that allows people in, we ensure realistic expectations of work coming out of both in the industry and the field. More than that, we allow people to understand the tools that they use every day and the impact that they have. It is our responsibility to make sure we are accurately represented, and only we can do that. Our project has been going on for a little under a year now, and we’ve made a difference in the accessibility of the field.

But if we’re going to keep going, we need your help to do so.

When Google bought Boston Dynamics last December, the news made headlines, but it was not the first time the Internet giant has invested in DARPA-funded robotics. As part of Robohub’s Big Deals series, we asked Gill Pratt, Program Manager of DARPA’s Defense Sciences Office, to shed some light on what DARPA thinks about Google’s robotics acquisitions , and what it might mean to the robotics and open source communities.

Congratulations, Gill, on the DARPA Robotics Trials – what was your personal assessment of the event?

I was very pleased with how well the teams performed, and in particular that the hardware for the majority of the teams worked so reliably. In general, the teams did slightly better than we had expected and this was probably the result of a lot of very careful preparation. The infrastructure that we set up also worked reliably in the trials. Still, the DRC Trials were really just a beginning and showed us the potential of what can be done in disaster scenarios with robots.

Cloud computing was an important facet of the DARPA Robotics Challenge and especially the Virtual Robotics Challenge … How do you see cloud computing helping the development of robots?

The importance of cloud computing for Robotics falls into two different categories. The first has to do with simulation environments. We used cloud computing for the first time with the Virtual Robotics Challenge, which we had back in June as a precursor to the DRC Trials, to do real-time simulation of tasks in unstructured environments. We rented a whole lot of cloud computing space, we ran a very good simulator developed for DARPA by the Open Source Robotics Foundation, and we adjusted the latency – the time delay – so that it would be the same for all the teams, even though they were located in different parts of the world. It was a tremendous enabler to run a real-time competition based on simulation that also involves human-robot collaboration. Doing so required that the simulation run in real-time. That was a big first and we felt it was very successful.

Another important aspect of cloud computing is something that we have not used to date, but I think has a lot of potential for the future: the ability to have a robot’s computing and data hosted remotely. There’s a lot of potential for this, in particular for doing perception. Forty percent of the human brain is used for perception, and perception is one of the most difficult tasks for an autonomous robot. It’s very difficult to fit a computer with the size, weight, and power that you need to achieve really good perception onto a robot: the computer just gets too large, it consumes too much power, and it weighs too much. However, if you have access to cloud resources with lots of data and lots of computing cycles, they can help move that burden off of the physical robot.

Perhaps most exciting for the future of cloud computing in robotics is that when one robot learns how to perceive something, or learns how to do a particular task, that learning can be instantly shared with other robots. This sharing could have a catalytic effect on the capabilities of robots, particularly in structured environments.

This is really an up and coming field. How likely are we to see commercial interest come out of cloud computing for robotics?

If you look at the research being done we see a lot of possibilities with cloud computing for robots. In the commercial world, I think we are going to see it applied first to structured environments where there are lots of human artifacts — things like doors, stairs, furniture, tools — and where a large database of stored examples of these artifacts can help the recognition problem.

The Atlas robot, created by Bostron Dynamics and DARPA, was used by several teams in the DRC Trials. Source: DARPA

This is an interview for our Big Deals series, so we’re wondering, what does DARPA think about Google buying up some of the technologies that DARPA has spent years building? We’re thinking of the cars, the humanoids …

We are thrilled to see commercial interest. It’s one of the signs of success for the investments that we have made in future technology.

We think that it takes commercial investment to drive down the cost of technology. A very well known example is the cellphone. Cellphones have microprocessors in them that many years ago began with investments by DARPA’s Microsystems Technology Office and others. Cell phones have inertial measurement units to figure out the tilt of the cellphone. They have displays; they have GPS receivers; they have radio devices that work at low power. I can point to any number of DARPA investments that helped start those technologies, and of course cellphones now talk to the Internet, which is probably the best known output of all of DARPA’s investments. But if the Department of Defense wanted to produce a cellphone without the commercial world having picked it up and turned it into a product that billions of people use, it would cost many orders of magnitude more than it does now. It’s really because of the commercial world that we’ve seen the price of cellphones go down.

But DARPA is not only interested in seeding technologies with these early investments; we are also committed to solving real problems for national security. In fact, we have a variety of programs that use cellphones. One of these is called TransApps, where soldiers use smart phones to plan and carry out missions and layer and share data between teams. This is an example where investment by the commercial world made possible a national security capability that otherwise might not be feasible because of excessive cost.

By analogy, what’s going on in robotics is that we’ve seen some interest in the commercial world in taking this to the next step. And they have resources far beyond what DARPA has to further develop a technology and, most importantly, to drive down cost.

So these big deals are a good thing for DARPA?

Yes, they are a wonderful thing for DARPA and, more broadly, for our nation.

Do you think it’s a good thing for the field of robotics in general?

Absolutely, but I understand why there’s some anxiety.  It’s true that, for a very short time, some talented people in the field will be working for a commercial firm rather than for DARPA. However, that effect is transient and very small compared to the much more important thing, which is that these corporate investments both accelerate the field and show that it is going to be real.

This anxiety is a lot like the kind you get when your kids leave home. It’s sad, and for a while you’re going to miss them, but wait a while and then the grandkids come. The important thing is for Robotics to become a field that is not only about prototypes or lab demonstrations, but a field where significant resources are invested to make products that really improve people’s lives. We saw it with the Roomba vacuum cleaner, but we want to go beyond that. These kinds of commercial investments will make the field far more attractive to startups and to students who are planning their careers. The net effect is strongly positive.

How do you think this corporate interest in robotics will impact the open source community?

Open source is very important. It allows us to develop pre-competitive catalysts to help the whole field move forward.

In the integrated circuits world it was recognized that you couldn’t build experimental integrated circuits as easily as you could wire up a circuit with discrete components like capacitors and resistors and transistors that you solder by hand. As a result, people working in the area at that time needed to develop a simulator to allow for the experimentation of how a chip would work without actually having to make the chip. The idea was to do the design and the debug cycle on the computer in simulation instead of having to do it on the bench: when you were done you could press a button and have confidence that the chip would actually work. The open-source simulator SPICE filled that niche.

The very same kind of thing is true in the robotics field. The key is to recognize that simulation is really a pre-competitive technology now.  We don’t need to have proprietary closed simulation systems and keep reinventing the simulator over and over again. We’ve done it enough times now to know that it would be useful to apply open source. In the same way, I think that an operating system is precompetitive – it has been done enough times that a new company should not try to distinguish itself based on a proprietary version of an OS.

Rather, let’s just move on to the next thing where the real innovation is going to occur. DARPA funded the Open Source Robotics Foundation to develop an open source simulator for the DARPA Robotics Challenge because we think that it will catalyze the whole field to everyone’s advantage.

With the purchase of Boston Dynamics, some headlines were saying that Google bought military robotics. Is this an accurate assessment?

I think it’s very inaccurate. It’s based on the confusion – and the conflation – of two aspects of robotics: remote bodies and remote brains. A robot allows you to have a mechanical body take the place of a human being; for example, a robot can work in a dangerous environment while a human supervisor is in a safe place. And a robot also allows you to perform some of the function of the human brain autonomously in a remote location. When you conflate bodies and brains, and say that there is only one idea here, it’s like saying, “The robot looks like person, so it must be as intelligent as a person, and therefore I should be scared of all robots, particularly those funded by the Department of Defense.”

All of the robots that I’m aware of, except for the ones that actually have weapons or specialized sensors on them, are generic. They are neither military nor commercial. They are just robots that have mobility and perhaps some manipulation, but there are not classed to one type or the other. And they are all almost completely empty-headed: their bodies may look like ours, but their heads are almost totally empty.

In terms of autonomy, the technology has such a long way to go that the fears that have been generated are way out of proportion to the state of the art. At the DRC Trials, for example, we had people supervising the robots in pretty tight loops, telling them what to do. While autonomy may be able to handle very simple environments, close human supervision is going to be needed in unstructured environments for some time.

What do you say to people who are uncomfortable with the use of military robots in a commercial context?

I think there are legitimate concerns, but from a rational point of view it’s important to distinguish between a weapon, which none of the robots we are talking about are, and a machine that makes us feel scared because it looks like us but it’s hard to understand exactly how it works. Of course, science fiction has stoked that fear with movies like Terminator and so forth, but that reaction actually goes back a long, long way to stories like Frankenstein – anything that sort of looks like a person but is hard to understand will generate fear. It’s an emotional reaction. However, from a rational perspective, if you consider how little autonomy these real machines actually have, you realize that the worry is way out of proportion to reality.

What are some of the benefits that could come out of combining Google’s expertise in data with hardware from Boston Dynamics?

I don’t know the specific applications that Google is thinking of – it’s their prerogative to discuss them or not, and they haven’t told DARPA.

Generally speaking, when you think about data and robotics, there are a lot of exciting things that could happen. For example, you could do perception better by using a lot of data and sophisticated search. This is not necessarily the kind of search we think of when we type words into our browsers, but the kind that involves images, where a robot could say, “Okay, I see something and because of many prior examples I know what to do,” and share that data in a useful way.

Would voice commands play into this dynamic, in addition to visual search? Would it make it easier for humans to interface with robots?

I’m not an expert on voice, so I don’t want to speculate, but I know that you can train better when you have a lot of data. Voice and natural language recognition have the potential to help with planning and perception, and I think they could also facilitate more intuitive interfaces for sharing information between robots and humans.

Team SCHAFT raises the arms of its S-One robot in victory after successfully completing the Climb Industrial Ladder task at the DRC Trials. SCHAFT won that task and three others, and scored the most points of any team at the event. Source: DARPA.

What new robotic technologies will DARPA be pushing that might appeal to big companies that are not traditionally robotics focused?

We’re really interested in cooperation directly between robots and between human beings and machines. We are also interested in dealing with dynamic environments. What I would like to do for the future of the DRC, and what I think that DARPA is going to push even beyond that, is to answer questions like “What if the communication is intermittent?” and “What if the environment is beyond a set of familiar things that I’ve seen before?” … What do we do if we’re actually outdoors? How do we handle this piece of vegetation that’s in front of us? Do we go through it or around it? If we go through it, do we move the branches out of the way?

We also still have a lot of research to do to figure out how to apply cloud computing when the environment is unstructured, like it would be in disasters or outdoor scenarios.  And if the communications are intermittent, we have to figure out how to cache enough of the intelligence on the robot itself so that it can continue to do what it needs to even when it’s disconnected from the cloud. Of course, disconnected operation may be at a reduced level of effectiveness, but we don’t want the robot to stop altogether when communications go down for a little while. We want the robot to have at least some ability to complete the tasks it was given to do.

How do you actually put behavior in the cloud? How do you take a data driven approach to behavior? I think that’s very much an unknown thing right now. Data driven approaches to perception are easier to understand, but we are going to try to take the next step and look at behavior as well.

If you liked this article, you may also be interested in:

• Snowden BEAMs into TED: How robotic telepresence disrupts borders
• Google’s robot and artificial intelligence acquisitions are anything but scary
• Will companies like Google and Amazon usher in a new era for robotics?
• The robots behind your Internet shopping addiction: How your wallet is driving retail automation
• What do Google, Amazon, IBM, Apple, Microsoft and all the airlines and car companies want?
• Robots Podcast: Privacy, Google, and big deals
• Robots, soldiers and cyborgs: The future of warfare

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.