Image: Alex Waller, MIT CSAIL

Being able to both walk and take flight is typical in nature – many birds, insects and other animals can do both. If we could program robots with similar versatility, it would open up many possibilities: picture machines that could fly into construction areas or disaster zones that aren’t near roads, and then be able to squeeze through tight spaces to transport objects or rescue people.

The problem is that usually robots that are good at one mode of transportation are, by necessity, bad at another. Drones are fast and agile, but generally have too limited of a battery life to travel for long distances. Ground vehicles, meanwhile, are more energy efficient, but also slower and less mobile.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) are aiming to develop robots that can do both. In a new paper, the team presented a system of eight quadcopter drones that can both fly and drive through a city-like setting with parking spots, no-fly zones and landing pads.

“The ability to both fly and drive is useful in environments with a lot of barriers, since you can fly over ground obstacles and drive under overhead obstacles,” says PhD student Brandon Araki, lead author on a paper about the system out of CSAIL director Daniela Rus’ group. “Normal drones can’t maneuver on the ground at all. A drone with wheels is much more mobile while having only a slight reduction in flying time.”

Araki and Rus developed the system along with MIT undergraduate students John Strang, Sarah Pohorecky and Celine Qiu, as well as Tobias Naegeli of ETH Zurich’s Advanced Interactive Technologies Lab. The team presented their system at IEEE’s International Conference on Robotics and Automation (ICRA) in Singapore earlier this month.

How it works

The project builds on Araki’s previous work developing a “flying monkey” robot that crawls, grasps, and flies. While the monkey robot could hop over obstacles and crawl about, there was still no way for it to travel autonomously.

To address this, the team developed various “path-planning” algorithms aimed at ensuring that the drones don’t collide. To make them capable of driving, the team put two small motors with wheels on the bottom of each drone. In simulations the robots could fly for 90 meters or drive for 252 meters before their batteries ran out.

Adding the driving component to the drone slightly reduced its battery life, meaning that the maximum distance it could fly decreased 14 percent to about 300 feet. But since driving is still much more efficient than flying, the gain in efficiency from driving more than offsets the relatively small loss in efficiency in flying due to the extra weight.

“This work provides an algorithmic solution for large-scale, mixed-mode transportation and shows its applicability to real-world problems,” says Jingjin Yu, a computer science professor at Rutgers University who was not involved in the paper.

The team also tested the system using everyday materials like pieces of fabric for roads and cardboard boxes for buildings. They tested eight robots navigating from a starting point to an ending point on a collision-free path, and all were successful.

Rus says that systems like theirs suggest that another approach to creating safe and effective flying cars is not to simply “put wings on cars,” but to build on years of research in drone development to add driving capabilities to them.

“As we begin to develop planning and control algorithms for flying cars, we are encouraged by the possibility of creating robots with these capabilities at small scale,” says Rus. “While there are obviously still big challenges to scaling up to vehicles that could actually transport humans, we are inspired by the potential of a future in which flying cars could offer us fast, traffic-free transportation.”

Click here to read the paper.

New algorithms power a prototype system for helping visually impaired users avoid obstacles and identify objects. Courtesy of the researchers.

Computer scientists have been working for decades on automatic navigation systems to aid the visually impaired, but it’s been difficult to come up with anything as reliable and easy to use as the white cane, the type of metal-tipped cane that visually impaired people frequently use to identify clear walking paths.

White canes have a few drawbacks, however. One is that the obstacles they come in contact with are sometimes other people. Another is that they can’t identify certain types of objects, such as tables or chairs, or determine whether a chair is already occupied.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a new system that uses a 3-D camera, a belt with separately controllable vibrational motors distributed around it, and an electronically reconfigurable Braille interface to give visually impaired users more information about their environments.

The system could be used in conjunction with or as an alternative to a cane. In a paper they’re presenting this week at the International Conference on Robotics and Automation, the researchers describe the system and a series of usability studies they conducted with visually impaired volunteers.

“We did a couple of different tests with blind users,” says Robert Katzschmann, a graduate student in mechanical engineering at MIT and one of the paper’s two first authors. “Having something that didn’t infringe on their other senses was important. So we didn’t want to have audio; we didn’t want to have something around the head, vibrations on the neck — all of those things, we tried them out, but none of them were accepted. We found that the one area of the body that is the least used for other senses is around your abdomen.”

Katzschmann is joined on the paper by his advisor Daniela Rus, an Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science; his fellow first author Hsueh-Cheng Wang, who was a postdoc at MIT when the work was done and is now an assistant professor of electrical and computer engineering at National Chiao Tung University in Taiwan; Santani Teng, a postdoc in CSAIL; Brandon Araki, a graduate student in mechanical engineering; and Laura Giarré, a professor of electrical engineering at the University of Modena and Reggio Emilia in Italy.

Parsing the world

The researchers’ system consists of a 3-D camera worn in a pouch hung around the neck; a processing unit that runs the team’s proprietary algorithms; the sensor belt, which has five vibrating motors evenly spaced around its forward half; and the reconfigurable Braille interface, which is worn at the user’s side.

The key to the system is an algorithm for quickly identifying surfaces and their orientations from the 3-D-camera data. The researchers experimented with three different types of 3-D cameras, which used three different techniques to gauge depth but all produced relatively low-resolution images — 640 pixels by 480 pixels — with both color and depth measurements for each pixel.

The algorithm first groups the pixels into clusters of three. Because the pixels have associated location data, each cluster determines a plane. If the orientations of the planes defined by five nearby clusters are within 10 degrees of each other, the system concludes that it has found a surface. It doesn’t need to determine the extent of the surface or what type of object it’s the surface of; it simply registers an obstacle at that location and begins to buzz the associated motor if the wearer gets within 2 meters of it.

Chair identification is similar but a little more stringent. The system needs to complete three distinct surface identifications, in the same general area, rather than just one; this ensures that the chair is unoccupied. The surfaces need to be roughly parallel to the ground, and they have to fall within a prescribed range of heights.

Tactile data

The belt motors can vary the frequency, intensity, and duration of their vibrations, as well as the intervals between them, to send different types of tactile signals to the user. For instance, an increase in frequency and intensity generally indicates that the wearer is approaching an obstacle in the direction indicated by that particular motor. But when the system is in chair-finding mode, for example, a double pulse indicates the direction in which a chair with a vacant seat can be found.

The Braille interface consists of two rows of five reconfigurable Braille pads. Symbols displayed on the pads describe the objects in the user’s environment — for instance, a “t” for table or a “c” for chair. The symbol’s position in the row indicates the direction in which it can be found; the column it appears in indicates its distance. A user adept at Braille should find that the signals from the Braille interface and the belt-mounted motors coincide.

In tests, the chair-finding system reduced subjects’ contacts with objects other than the chairs they sought by 80 percent, and the navigation system reduced the number of cane collisions with people loitering around a hallway by 86 percent.

D’Arpino Perez discusses her work teaching Optimus to pick up a bottle. Image: Jason Dorfman, MIT CSAIL

Most robots are programmed using one of two methods: learning from demonstration, in which they watch a task being done and then replicate it, or via motion-planning techniques like optimization or sampling, which require a programmer to explicitly specify a task’s goals and constraints.

Both methods have drawbacks. Robots that learn from demonstration can’t easily transfer one skill they’ve learned to another situation and remain accurate. On the other hand, motion planning systems that use sampling or optimization can adapt to these changes, but are time-consuming, since they usually have to be hand-coded by expert programmers.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have recently developed a system that aims to bridge the two techniques: C-LEARN, which allows non-coders to teach robots a wider range of tasks simply by providing some information about how objects are typically manipulated and then showing the robot a single demo of the task. Importantly, this enables users to teach robots skills that can be automatically transferred to other robots with different “kinematics” (ways of moving) – a key time- and cost-saving measure for companies that want a range of robots to perform similar actions.

“By combining the intuitiveness of learning from demonstration with the precision of motion-planning algorithms, this approach can help robots do new types of tasks that they haven’t been able to learn before, like multi-step assembly using both of their arms,” says Claudia Pérez-D’Arpino, a PhD student who wrote a paper on C-LEARN with MIT professor Julie Shah.

The team tested the system on Optimus, a new two-armed robot designed for bomb disposal that they programmed to perform tasks like opening doors, transporting objects and extracting objects from containers.

(The robot is made of components from multiple manufacturers, including a manipulation system by RE2, a Husky unmanned ground vehicle by Clearpath, 3-finger grippers by Robotiq and the multisense sensor from Carnegie Robotics that is used in the Atlas robot.)

In simulations they showed that Optimus’ learned skills could be seamlessly transferred to Atlas, CSAIL’s six-foot-tall, 400-pound humanoid robot.

The paper was recently accepted to the IEEE International Conference on Robotics and Automation (ICRA), which takes place May 29-June 3 in Singapore.

How it works

With C-LEARN the user first gives the robot a knowledge base of information on how to reach and grasp different objects that have different constraints. (The C in C-LEARN stands for “constraints.”) The operator then uses a 3D interface to show the robot a single demonstration of the specific task, which is represented by a sequence of relevant moments known as “keyframes.” By matching these keyframes with the knowledge base, the robot can automatically suggest motion plans for the operator to approve or edit as needed.

“This approach is actually very similar to how humans learn in terms of seeing how something’s done and connecting it to what we already know about the world,” says Pérez-D’Arpino. “We can’t magically learn from a single demonstration, so we take new information and match it to previous knowledge about our environment.”

One challenge was that existing constraints that could be learned from demonstrations weren’t accurate enough to enable robots to precisely manipulate objects. To overcome that, the researchers developed constraints inspired by computer-aided design (CAD) programs that can tell the robot if its hands should be parallel or perpendicular to the objects it is interacting with.

The team also showed that the robot performed significantly better in collaboration with humans. While the robot successfully executed tasks 87.5 percent of the time on its own, it did so 100 percent of the time when it had a operator that could correct minor sensory inaccuracies.

“Having a knowledge base is fairly common, but what’s not common is integrating it with learning from demonstration,” says Dmitry Berenson, an assistant professor of computer science at the University of Michigan who was not involved in the research. “That’s very helpful, because if you are dealing with the same objects over and over again, you don’t want to then have to start from scratch to teach the robot every new task.”

The system is part of a larger wave of research focused on making LfD approaches more adaptive. If you’re a robot that has learned to take an object out of a tube from a demonstration, you might not be able to do it if there’s an obstacle in the way that requires you to move your arm differently. However, C-LEARN can do this, because it does not learn one specific way to perform the action.

“It’s good for the field that we’re moving away from directly imitating motion, towards actually trying to infer the principles behind the motion,” says Berenson. “By using these learned constraints in a motion planner, we can make systems that are far more flexible than those which just try to mimic what’s being demonstrated”

Shah says that advanced LfD methods could prove important in time-sensitive scenarios like bomb disposal and disaster response, where robots are currently tele-operated at the level of individual joint movements.

“Something as simple as picking up a box could take 20-30 minutes, which is significant for an emergency situation,” says Pérez-D’Arpino.

C-LEARN can’t yet handle certain advanced tasks, like avoiding collisions or planning for different step sequences for a given task. But the team is hopeful that incorporating more insights from human learning will give robots an even wider range of physical capabilities.

“Traditional programming of robots in real-world scenarios is difficult, tedious and requires a lot of domain knowledge,” says Shah. “It would be much more effective if we could train them more like how we train people: by giving them some basic knowledge and a single demonstration. This is an exciting step towards teaching robots to perform complex multi-arm and multi-step tasks necessary for assembly manufacturing and ship or aircraft maintenance.”

Click here to access the full paper.

MIT Professor Daniela Rus, director of CSAIL, said the goal of a new SystemsThatLearn@CSAIL initiative is “to create a new generation of AI tools that are deeply rooted in systems.” Photo: Jason Dorfman/MIT CSAIL

From self-driving cars to the internet of things, artificial intelligence (AI) has reached new levels of sophistication in recent years. With that in mind, this week MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) launched an industry collaboration focused on using machine learning to create functional human-like systems.

Nearly 40 senior researchers will participate in the new “SystemsThatLearn@CSAIL” (STL) initiative alongside a range of organizations that include founding members BT, Microsoft, Nokia Bell Labs, Salesforce, and Schlumberger. Member companies will work with CSAIL scientists to suggest new lines of research and develop real-world applications.

“Developing capabilities in AI and machine learning are key to the future of fields like finance, energy, manufacturing, and health care,” says STL Executive Director Lori Glover. “While the demand for expertise is great, the supply of talent remains small and unevenly distributed. By democratizing the field of AI, SystemsThatLearn@CSAIL is an effort to address that skills gap.”

STL builds on CSAIL’s Big Data Initiative, which developed tools for handling complex datasets. While many machine-learning solutions are trained and deployed in separate phases, STL aims to integrate these processes, focusing on a range of resources to handle distributed data and computing power.

Another goal is to make key aspects of data science less laborious. A 2016 report found that data scientists spend 80 percent of their time collecting and organizing data, and only 20 percent analyzing it.

“By promoting industry interaction with academia, we’re hoping to create new tools and systems that can increase productivity by automating much of the tedious work of data science,” says MIT Professor Samuel Madden, one of STL’s two faculty leads alongside Professor Tommi Jaakkola. “We are already seeing that areas like autonomous vehicles and personalized health care have the potential to transform entire industries.”

In her opening remarks, CSAIL Director Daniela Rus described AI and systems researchers as two communities that would benefit from stronger collaboration.

“Our grand vision is to create a new generation of AI tools that are deeply rooted in systems and that can make those systems better,” Rus said. “My aspiration is to get to a place where machine learning becomes a normal part of what an operating system does.”

One STL project is the data discovery tool “Data Civilizer,” which allows organizations to discover related datasets from thousands of distinct business databases and files. Another is “Model DB,” a machine-learning management system that saves time for data scientists and lets them easily correlate performance on particular training examples with specific model features.

According to STL Technical Director Stephen Buckley, many of the software tools they develop will be released under MIT’s open source license.

“The ultimate aim is to democratize access and use of machine learning tools, without requiring advanced knowledge of the underlying technologies,” says Buckley.

The feedback system enables human operators to correct the robot’s choice in real-time – Jason Dorfman, MIT CSAIL

For robots to do what we want, they need to understand us. Too often, this means having to meet them halfway: teaching them the intricacies of human language, for example, or giving them explicit commands for very specific tasks. But what if we could develop robots that were a more natural extension of us and that could actually do whatever we are thinking?

A team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and Boston University is working on this problem, creating a feedback system that lets people correct robot mistakes instantly with nothing more than their brains.

Using data from an electroencephalography (EEG) monitor that records brain activity, the system can detect if a person notices an error as a robot performs an object-sorting task. The team’s novel machine-learning algorithms enable the system to classify brain waves in the space of 10 to 30 milliseconds.

While the system currently handles relatively simple binary-choice activities, the paper’s senior author says that the work suggests that we could one day control robots in much more intuitive ways.

“Imagine being able to instantaneously tell a robot to do a certain action, without needing to type a command, push a button or even say a word,” says CSAIL director Daniela Rus. “A streamlined approach like that would improve our abilities to supervise factory robots, driverless cars and other technologies we haven’t even invented yet.”

In the current study the team used a humanoid robot named “Baxter” from Rethink Robotics, the company led by former CSAIL director and iRobot co-founder Rodney Brooks.

The paper presenting the work was written by BU PhD candidate Andres F. Salazar-Gomez, CSAIL PhD candidate Joseph DelPreto, and CSAIL research scientist Stephanie Gil under the supervision of Rus and BU professor Frank H. Guenther.  The paper was recently accepted to the IEEE International Conference on Robotics and Automation (ICRA) taking place in Singapore this May.

Past work in EEG-controlled robotics has required training humans to “think” in a prescribed way that computers can recognize. For example, an operator might have to look at one of two bright light displays, each of which corresponds to a different task for the robot to execute.

The downside to this method is that the training process and the act of modulating one’s thoughts can be taxing, particularly for people who supervise tasks in navigation or construction that require intense concentration.

Rus’ team wanted to make the experience more natural. To do that, they focused on brain signals called “error-related potentials” (ErrPs), which are generated whenever our brains notice a mistake. As the robot indicates which choice it plans to make, the system uses ErrPs to determine if the human agrees with the decision.

The team uses EEG brain signals to detect if the person notices a mistake – Jason Dorfman, MIT CSAIL

“As you watch the robot, all you have to do is mentally agree or disagree with what it is doing,” says Rus. “You don’t have to train yourself to think in a certain way –  the machine adapts to you, and not the other way around.”

ErrP signals are extremely faint, which means that the system has to be fine-tuned enough to both classify the signal and incorporate it into the feedback loop for the human operator.

In addition to monitoring the initial ErrPs, the team also sought to detect “secondary errors” that occur when the system doesn’t notice the human’s original correction.

“If the robot’s not sure about its decision, it can trigger a human response to get a more accurate answer,” says Gil. “These signals can dramatically improve accuracy, creating a continuous dialogue between human and robot in communicating their choices.”

The feedback system enables human operators to correct the robot’s choice in real-time – Jason Dorfman, MIT CSAIL2

While the system cannot yet recognize secondary errors in real time, Gil expects the model to be able to improve to upwards of 90 percent accuracy once it can.

In addition, since ErrP signals have been shown to be proportional to how egregious the robot’s mistake is, the team believes that future systems could extend to more complex multiple-choice tasks.

Salazar-Gomez notes that the system could even be useful for people who can’t communicate verbally: a task like spelling could be accomplished via a series of several discrete binary choices, which he likens to an advanced form of the blinking that allowed stroke victim Jean-Dominique Bauby to write his memoir “The Diving Bell and the Butterfly.”

“This work brings us closer to developing effective tools for brain-controlled robots and prostheses,” says Wolfram Burgard, a professor of computer science at the University of Freiburg who was not involved in the research. “Given how difficult it can be to translate human language into a meaningful signal for robots, work in this area could have a truly profound impact on the future of human-robot collaboration.”

Read the paper here.

The project was funded in part by Boeing and the National Science Foundation.

If you enjoyed this article from CSAIL, you might also be interested in:

• MIT CSAIL’s 6-foot-tall NASA humanoid robot has landed
• Researchers add a splash of human intuition to planning algorithms
• Wearable AI that can detect the tone of a conversation
• Ingestible robots, glasses-free 3-D, and computers that explain themselves
• Design, simulate and build a custom drone

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory are trying to improve automated planners by giving them the benefit of human intuition. By encoding the strategies of high-performing human planners in a machine-readable form, they were able to improve the performance of competition-winning planning algorithms by between 10 and 15 percent on a challenging set of problems. Image: Jose-Luis Olivares/MIT

Every other year, the International Conference on Automated Planning and Scheduling hosts a competition in which computer systems designed by conference participants try to find the best solution to a planning problem, such as scheduling flights or coordinating tasks for teams of autonomous satellites.

On all but the most straightforward problems, however, even the best planning algorithms still aren’t as effective as human beings with a particular aptitude for problem-solving — such as MIT students.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory are trying to improve automated planners by giving them the benefit of human intuition. By encoding the strategies of high-performing human planners in a machine-readable form, they were able to improve the performance of competition-winning planning algorithms by 10 to 15 percent on a challenging set of problems.

The researchers are presenting their results this week at the Association for the Advancement of Artificial Intelligence’s annual conference.

“In the lab, in other investigations, we’ve seen that for things like planning and scheduling and optimization, there’s usually a small set of people who are truly outstanding at it,” says Julie Shah, an assistant professor of aeronautics and astronautics at MIT. “Can we take the insights and the high-level strategies from the few people who are truly excellent at it and allow a machine to make use of that to be better at problem-solving than the vast majority of the population?”

The first author on the conference paper is Joseph Kim, a graduate student in aeronautics and astronautics. He’s joined by Shah and Christopher Banks, an undergraduate at Norfolk State University who was a research intern in Shah’s lab in the summer of 2016.

The human factor

Algorithms entered in the automated-planning competition — called the International Planning Competition, or IPC — are given related problems with different degrees of difficulty. The easiest problems require satisfaction of a few rigid constraints: For instance, given a certain number of airports, a certain number of planes, and a certain number of people at each airport with particular destinations, is it possible to plan planes’ flight routes such that all passengers reach their destinations but no plane ever flies empty?

A more complex class of problems — numerical problems — adds some flexible numerical parameters: Can you find a set of flight plans that meets the constraints of the original problem but also minimizes planes’ flight time and fuel consumption?

Finally, the most complex problems — temporal problems — add temporal constraints to the numerical problems: Can you minimize flight time and fuel consumption while also ensuring that planes arrive and depart at specific times?

For each problem, an algorithm has a half-hour to generate a plan. The quality of the plans is measured according to some “cost function,” such as an equation that combines total flight time and total fuel consumption.

Shah, Kim, and Banks recruited 36 MIT undergraduate and graduate students and posed each of them the planning problems from two different competitions, one that focused on plane routing and one that focused on satellite positioning. Like the automatic planners, the students had a half-hour to solve each problem.

“By choosing MIT students, we’re basically choosing the world experts in problem solving,” Shah says. “Likely, they’re going to be better at it than most of the population.”

Encoding strategies

Certainly, they were better than the automatic planners. After the students had submitted their solutions, Kim interviewed them about the general strategies they had used to solve the problems. Their answers included things like “Planes should visit each city at most once,” and “For each satellite, find routes in three turns or less.”

The researchers discovered that the large majority of the students’ strategies could be described using a formal language called linear temporal logic, which in turn could be used to add constraints to the problem specifications. Because different strategies could cancel each other out, the researchers tested each student’s strategies separately, using the planning algorithms that had won their respective competitions. The results varied, but only slightly. On the numerical problems, the average improvement was 13 percent and 16 percent, respectively, on the flight-planning and satellite-positioning problems; and on the temporal problems, the improvement was 12 percent and 10 percent.

“The plan that the planner came up with looked more like the human-generated plan when it used these high-level strategies from the person,” Shah says. “There is maybe this bridge to taking a user’s high-level strategy and making that useful for the machine, and by making it useful for the machine, maybe it makes it more interpretable to the person.”

In ongoing work, Kim and Shah are using natural-language-processing techniques to make the system fully automatic, so that it will convert users’ free-form descriptions of their high-level strategies into linear temporal logic without human intervention.

Read the preprint accepted at AAAI here. 

Samsung Simband. Image: Jason Dorfman, MIT CSAIL

It’s a fact of nature that a single conversation can be interpreted in very different ways. For people with anxiety or conditions like Asperger’s, this can make social situations extremely stressful. But what if there was a more objective way to measure and understand our interactions?

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) say that they’ve gotten closer to a potential solution: an artificially intelligent wearable system that can predict if a conversation is happy, sad or neutral based on a person’s speech patterns and vitals.

“Imagine if, at the end of a conversation, you could rewind it and see the moments when the people around you felt the most anxious,” says graduate student Tuka Alhanai, who co-authored a related paper with PhD candidate Mohammad Ghassemi that they will present at next week’s Association for the Advancement of Artificial Intelligence (AAAI) conference in San Francisco. “Our work is a step in this direction, suggesting that we may not be that far away from a world where people can have an AI social coach right in their pocket.”

As a participant tells a story, the system can analyze audio, text transcriptions and physiological signals to determine the overall tone of the story with 83 percent accuracy. Using deep-learning techniques, the system can also provide a specific sentiment score for five-second intervals within a conversation.

“As far as we know, this is the first experiment that collects both physical data and speech data in a passive but robust way, even while subjects are having natural, unstructured interactions,” says Ghassemi. “Our results show that it’s possible to classify the emotional tone of conversations in real-time.”

The team is keen to point out that they developed the system with privacy strongly in mind: the algorithm runs locally on a user’s device as a way of protecting personal information. (Alhanai says that a consumer version would obviously need clear protocols for getting consent from the people involved in the conversations.)

How it works

Past studies in this area would often show participants “happy” and “sad” videos, or ask them to artificially act out specific emotive states. But in an effort to elicit more organic emotions, the team instead asked subjects to tell a happy or sad story of their own choosing.

Subjects wore a Samsung Simband, a device that captures high-resolution physiological waveforms to measure features like movement, heart rate, blood pressure, blood flow and skin temperature,. The system also captured audio data and text transcripts to analyze the speaker’s tone, pitch, energy, and vocabulary.

Mohammad Ghassemi and Tuka Alhanai converse with the wearable. Image: Jason Dorfman MIT CSAIL

“The team’s usage of consumer market devices for collecting physiological data and speech data shows how close we are to having such tools in everyday devices,” says Björn Schuller, professor and chair of Complex and Intelligent Systems at the University of Passau in Germany who was not involved in the research. “Technology could soon feel much more emotionally intelligent, or even ‘emotional’ itself.”

After capturing 31 different conversations of several minutes each, the team trained two algorithms on the data: one classified the overall nature of a conversation as either happy or sad, while the second classified each five-second block of every conversation as either positive, negative or neutral.

Alhanai notes that, in traditional neural networks, all features about the data are provided to the algorithm at the base of the network. In contrast, her team found that they could improve performance by organizing different features at the various layers of the network.

“The system picks up on how, for example, the sentiment in the text transcription was more abstract than the raw accelerometer data, says Alhanai. “It’s quite remarkable that a machine could approximate how we humans perceive these interactions, without significant input from us as researchers.”

Results

Indeed, the algorithm’s findings align well with what we humans might expect to observe. For instance, long pauses and a monotonous vocal tones were associated with sadder stories, while more energetic, varied speech patterns were associated with happier ones. In terms of body language, sadder stories were also strongly associated with increased fidgeting and cardiovascular activity, as well as certain postures like putting one’s hands on one’s face.

Graph showing real-time emotion detection. Image: MIT CSAIL

On average, the model could classify the mood of each five-second interval with an accuracy that was approximately 18 percent above chance, and a full 7.5 percent better than existing approaches. In future work, the team hopes to collect data on a much larger scale, potentially using commercial devices like the Apple Watch that would allow them to more easily deploy the system out in the world.

“Our next step is to improve the algorithm’s emotional granularity so it can call out boring, tense, and excited moments with greater accuracy instead of just labeling interactions as ‘positive’ or ‘negative’,” says Alhani. “Developing technology that can take the pulse of human emotions has the potential to dramatically improve how we communicate with each other.”

Click here to download the research paper.

You might also enjoy the following articles:

• Artificial intelligence and ethics: Who does the thinking?
• Soft exosuit economies: Understanding the costs of lightening the load
• MIT Media Lab to participate in $27 million initiative on AI ethics and governance
• How artificial intelligence is changing our Christmas shop
• White House report: Artificial intelligence, automation, and the economy

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In 2016, MIT CSAIL researchers worked on a range of projects in robotics, theory, wireless technology, software systems, and other disciplines. Image: CSAIL

Machines that predict the future, robots that patch wounds and wireless emotion-detectors are just a few of the exciting projects that came out of MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) this year. Here’s a sampling of 16 highlights from 2016 that span the many computer science disciplines that make up CSAIL.

Robots for exploring Mars — and your stomach

• A team led by CSAIL director Daniela Rus developed an ingestible origami robot that unfolds in the stomach to patch wounds and remove swallowed batteries.
• Researchers are working on NASA’s humanoid robot, “Valkyrie,” who will be programmed for trips into outer space and to autonomously perform tasks.
• A 3-D printed robot was made of both solids and liquids and printed in one single step, with no assembly required.

Keeping data safe and secure

• CSAIL hosted a cyber summit that convened members of academia, industry, and government, including featured speakers Admiral Michael Rogers, director of the National Security Agency; and Andrew McCabe, deputy director of the Federal Bureau of Investigation.
• Researchers came up with a system for staying anonymous online that uses less bandwidth to transfer large files between anonymous users.
• A deep-learning system called AI2 was shown to be able to predict 85 percent of cyberattacks with the help of some human input.

Advancements in computer vision

• A new imaging technique called Interactive Dynamic Video lets you reach in and “touch” objects in videos using a normal camera.
• Researchers from CSAIL and Israel’s Weizmann Institute of Science produced a movie display called Cinema 3D that uses special lenses and mirrors to allow viewers to watch 3-D movies in a theater without having to wear those clunky 3-D glasses.
• A new deep-learning algorithm can predict human interactions more accurately than ever before, by training itself on footage from TV shows like “Desperate Housewives” and “The Office.”
• A group from MIT and Harvard University developed an algorithm that may help astronomers produce the first image of a black hole, stitching together telescope data to essentially turn the planet into one large telescope dish.

Tech to help with health

• A team produced a robot that can help schedule and assign tasks by learning from humans, in fields like medicine and the military.
• Researchers came up with an algorithm for identifying organs in fetal MRI scans to extensively evaluate prenatal health.
• A wireless device called EQ-Radio can tell if you’re excited, happy, angry, or sad, by measuring breathing and heart rhythms.

Algorithms, systems and networks

• A system called “Polaris” was found to load web pages 34 percent faster by decreasing network trips.
• A team analyzed ant-colony behavior to create better algorithms for network communication, for applications such as social networks and collective decision-making among robot swarms.
• Researchers trained neural networks to explain themselves by providing rationales for their decisions.

PhD student Tao Du watching the bunnycopter take off . Image credit: Jason Dorfman, MIT CSAIL

This fall’s new FAA regulations have made drone flight easier than ever for both companies and consumers. But what if the drones out on the market aren’t exactly what you want?

A new system from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) is the first to allow users to design, simulate and build their own custom drone. Users can change the size, shape and structure of their drone based on the specific needs they have for payload, cost, flight time, battery usage and other factors.
To demonstrate, researchers created a range of unusual-looking drones, including a five-rotor “pentacopter” and a rabbit-shaped “bunnycopter” with rotors of different sizes and heights.

“This system opens up new possibilities for how drones look and function,” says MIT professor Wojciech Matusik, who oversaw the project in CSAIL’s Computational Fabrication Group. “It’s no longer a one-size-fits-all approach for people who want to make and use drones for particular purposes.”

The interface lets users design drones with different rotors and rods. It also provides guarantees that its drones can take off, hover and land (which is no simple task considering the intricate technical trade-offs associated with drone weight, shape and control).

“For example, adding more rotors generally lets you carry more weight, but you also need to think about how to balance the drone to make sure it doesn’t tip,” says PhD student Tao Du, who is first author on a related paper about the system. “Irregularly-shaped drones are very difficult to stabilize, which means that they require establishing very complex control parameters.”

Du and Matusik co-authored a paper with PhD student Adriana Schulz, postdoctoral researcher Bo Zhu and assistant professor Bernd Bickel of IST Austria. It will be presented this week at the annual SIGGRAPH Asia conference in Macao, China.

How it works
Today’s commercial drones only come in a small range of options, typically with an even number of upward-facing rotors. But there are many emerging use cases for other kinds of drones. For example, having an odd number of rotors might create a clearer view for a drone’s camera, or allow the drone to carry objects with unusual shapes.

Designing these less conventional drones, however, often requires expertise in multiple disciplines, including control systems, fabrication and electronics.

“Developing multicopters like these that are actually flyable involves a lot of trial-and-error, tweaking the balance between all the propellers and rotors,” says Du. “It would be more or less impossible for an amateur user, especially one without any computer-science background.”

But the CSAIL group’s new system makes the process much easier. Users design drones by choosing from a database of parts and specifying their needs for things like payload, cost and battery usage. The system computes the sizes of design elements like rod lengths and motor angles, and looks at metrics such as torque and thrust to determine whether the design will actually work. It also uses an “LQR controller” that takes information about a drone’s characteristics and surroundings to optimize its flight plan.

One of the project’s core challenges stemmed from the fact that a drone’s shape and structure (its “geometry”) is usually strongly tied to how it has been programmed to move (its “control”). To overcome this, researchers used what’s called an “alternating direction method,” which means that they reduced the number of variables by fixing some of them and optimizing the rest. This allowed the team to decouple the variables of geometry and control in a way that optimizes the drone’s performance.

“Once you decouple these variables, you turn a very complicated optimization problem into two easy sub-problems that we already have techniques for solving,” says Du.

Du envisions future versions of the system that could proactively give design suggestions, like recommending where a rotor should go to accommodate a desired payload.

“This is the first system in which users can interactively design a drone that incorporates both geometry and control,” says Nobuyuki Umetani, a research scientist at Autodesk, Inc who was not involved in the paper. “This is very exciting work that has the potential to change the way people design.”

The project was supported in part by the National Science Foundation, the Air Force Research Laboratory and the European Union’s Horizon 2020 research and innovation program.

Click here to read the full paper.

If you enjoyed this article, you might also be interested in:

• Emerging patchwork of regulation of UAS operations near critical infrastructure
• Protecting infrastructure and innovation, under Section 2209 of the FAA extension
• Why DJI is succeeding in making and selling drones and 3DR didn’t
• MIT CSAIL’s 6-foot-tall NASA humanoid robot has landed

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Credit: Carl Vondrick, MIT CSAIL

Living in a dynamic physical world, it’s easy to forget how effortlessly we understand our surroundings. With minimal thought, we can figure out how scenes change and objects interact.

But what’s second nature for us is still a huge problem for machines. With the limitless number of ways that objects can move, teaching computers to predict future actions can be difficult.

Recently, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have gotten a step closer, developing a deep-learning algorithm that, given still images from a scene, can create brief videos that simulate the future of that scene.

Trained on two million unlabeled videos that include a year’s worth of footage, the algorithm generated videos that human subjects deemed to be realistic 20 percent more often than a baseline model.

To be clear, at this point the videos are still relatively low-resolution and only 1-1.5 seconds in length. But the team is hopeful that future versions could be used for everything from improved security tactics to safer self-driving cars.

According to CSAIL PhD student and first author Carl Vondrick, the algorithm can also help machines recognize people’s activities without expensive human annotations.

“These videos show us what computers think can happen in a scene,” says Vondrick. “If you can predict the future, you must have understood something about the present.”

Vondrick wrote the paper with MIT professor Antonio Torralba and Hamed Pirsiavash, a former CSAIL postdoctoral associate who is now a professor at the University of Maryland, Baltimore County. The work will be presented at next week’s Neural Information Processing Systems (NIPS) conference in Barcelona.

How it works
Multiple researchers have tackled similar topics in computer vision, including MIT professor Bill Freeman, whose new work on “visual dynamics” also creates future frames in a scene. But where his model focuses on extrapolating videos into the future, Torralba’s model can also generate completely new videos that haven’t been seen before.

Credit: Carl Vondrick, MIT CSAIL

Previous systems build up scenes frame by frame, which creates a large margin for error. In contrast, this work focuses on processing the entire scene at once, with the algorithm generating as many as 32 frames from scratch per second.

“Building up a scene frame-by-frame is like a big game of ‘Telephone,’ which means that the message falls apart by the time you go around the whole room,” says Vondrick. “By instead trying to predict all frames simultaneously, it’s as if I’m talking to everyone in the room at once.”

Of course, there’s a trade-off to generating all frames simultaneously: while it becomes more accurate, the computer model also becomes more complex for longer videos.

To create multiple frames, researchers taught the model to generate the foreground separate from the background, and to then place the objects in the scene to let the model learn which objects move and which objects don’t.

The team used a deep-learning method called “adversarial learning” that involves training two competing neural networks. One network generates video, and the other discriminates between the real and generated videos. Over time, the generator learns to fool the discriminator.

From that, the model can create videos resembling scenes from beaches, train stations, hospitals, and golf courses.  For example, the beach model produced beaches with crashing waves, and the golf model had people walking on grass.

Testing the scene
The team compared the videos against a baseline of generated videos and asked subjects which they thought were more realistic. From over 13,000 opinions of 150 users, subjects chose the generative model videos 20 percent more often than the baseline.

To be clear, the the model still lacks some fairly simple common-sense principles. For example, it often doesn’t understand that objects are still there when they move, like when a train passes through a scene. The model also tends to make humans and objects look much larger in size than reality.

As mentioned before, another limitation is that the generated videos are just one and a half seconds long, which the team hopes to be able to increase in future work. The challenge is that this requires tracking longer dependencies to ensure that the scene still makes sense over longer time periods. One way to do this would be to add human supervision.

“It’s difficult to aggregate accurate information across long time periods in videos,” says Vondrick. “If the video has both cooking and eating activities, you have to be able to link those two together to make sense of the scene.”

These types of models aren’t limited to predicting the future. Generative videos can be used for adding animation to still images, like the animated newspaper from the Harry Potter books. They could also help detect anomalies in security footage and compress data for storing and sending longer videos.

“In the future, this will let us scale up vision systems to recognize objects and scenes without any supervision, simply by training them on video,” says Vondrick.

This work was supported by the National Science Foundation, the START program at UMBC, and a Google PhD fellowship.

Read the research paper.

An autonomous mobility scooter and related software were designed by researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), the National University of Singapore, and the Singapore-MIT Alliance for Research and Technology (SMART). Courtesy of the Autonomous Vehicle Team of the SMART Future of Urban Mobility Project

By: Larry Hardesty

At MIT’s 2016 Open House last spring, more than 100 visitors took rides on an autonomous mobility scooter in a trial of software designed by researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), the National University of Singapore, and the Singapore-MIT Alliance for Research and Technology (SMART).

The researchers had previously used the same sensor configuration and software in trials of autonomous cars and golf carts, so the new trial completes the demonstration of a comprehensive autonomous mobility system. A mobility-impaired user could, in principle, use a scooter to get down the hall and through the lobby of an apartment building, take a golf cart across the building’s parking lot, and pick up an autonomous car on the public roads.

The new trial establishes that the researchers’ control algorithms work indoors as well as out. “We were testing them in tighter spaces,” says Scott Pendleton, a graduate student in mechanical engineering at the National University of Singapore (NUS) and a research fellow at SMART. “One of the spaces that we tested in was the Infinite Corridor of MIT, which is a very difficult localization problem, being a long corridor without very many distinctive features. You can lose your place along the corridor. But our algorithms proved to work very well in this new environment.”

The researchers’ system includes several layers of software: low-level control algorithms that enable a vehicle to respond immediately to changes in its environment, such as a pedestrian darting across its path; route-planning algorithms; localization algorithms that the vehicle uses to determine its location on a map; map-building algorithms that it uses to construct the map in the first place; a scheduling algorithm that allocates fleet resources; and an online booking system that allows users to schedule rides.

https://www.youtube.com/watch?v=h7ehanUmDhQ

Uniformity

Using the same control algorithms for all types of vehicles — scooters, golf carts, and city cars — has several advantages. One is that it becomes much more practical to perform reliable analyses of the system’s overall performance.

“If you have a uniform system where all the algorithms are the same, the complexity is much lower than if you have a heterogeneous system where each vehicle does something different,” says Daniela Rus, the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science at MIT and one of the project’s leaders. “That’s useful for verifying that this multilayer complexity is correct.”

Furthermore, with software uniformity, information that one vehicle acquires can easily be transferred to another. Before the scooter was shipped to MIT, for instance, it was tested in Singapore, where it used maps that had been created by the autonomous golf cart.

Similarly, says Marcelo Ang, an associate professor of mechanical engineering at NUS who co-leads the project with Rus, in ongoing work the researchers are equipping their vehicles with machine-learning systems, so that interactions with the environment will improve the performance of their navigation and control algorithms. “Once you have a better driver, you can easily transplant that to another vehicle,” says Ang. “That’s the same across different platforms.”

Finally, software uniformity means that the scheduling algorithm has more flexibility in its allocation of system resources. If an autonomous golf cart isn’t available to take a user across a public park, a scooter could fill in; if a city car isn’t available for a short trip on back roads, a golf cart might be.

“I can see its usefulness in large indoor shopping malls and amusement parks to take [mobility-impaired] people from one spot to another,” says Dan Ding, an associate professor of rehabilitation science and technology at the University of Pittsburgh, about the system.

Changing perceptions

The scooter trial at MIT also demonstrated the ease with which the researchers could deploy their modular hardware and software system in a new context. “It’s extraordinary to me, because it’s a project that the team conducted in about two months,” Rus says. MIT’s Open House was at the end of April, and “the scooter didn’t exist on February 1st,” Rus says.

The researchers described the design of the scooter system and the results of the trial in a paper they presented last week at the IEEE International Conference on Intelligent Transportation Systems. Joining Rus, Pendleton, and Ang on the paper are You Hong Eng, who leads the SMART autonomous-vehicle project, and four other researchers from both NUS and SMART.

The paper also reports the results of a short user survey that the researchers conducted during the trial. Before riding the scooter, users were asked how safe they considered autonomous vehicles to be, on a scale from one to five; after their rides, they were asked the same question again. Experience with the scooter brought the average safety score up, from 3.5 to 4.6.

Whitman Richards. Photo: Webb Chappell/MIT Media Lab

Whitman Richards ’53, PhD ’65, professor emeritus of cognitive sciences and of media arts and sciences and principal investigator in the Computer Science and Artificial Intelligence Laboratory, died on Sept. 16 after a long battle with myelofibrosis. One of the first four PhD graduates of the Department of Brain and Cognitive Sciences (BCS), his more than 60 years at MIT were marked by a dedication to the experimental and theoretical study of vision, perception, and cognition.

Richards began his affiliation with MIT as an undergraduate, matriculating in 1950. His decision to return to MIT for graduate work was greatly inspired by a meeting with BCS founder and then department head Professor Hans-Lukas Teuber.

“In the 1960’s, with the advent of accessible computer technology, the development of information theory, and the single electrode, there was renewed excitement about prospects for modeling and understanding mind and brain,” Richards said in a 2004 interview. “Teuber’s charisma and broad vision for a new psychol­ogy was a powerful draw [to the department]. …There was a unique opportunity for a non-traditional grounding in a discipline otherwise mired in tradition.”

Richards’ early research pursued traditional psychophysical experimental methods to study the mechanisms of color perception and stereovision. In the 1970s, his research direction and methodology shifted dramatically after meeting noted physiologist David Marr, who he eventually recruited to MIT. Instead of relying on the traditional experimental methods that had characterized his early career, Richards, Marr, and colleagues began to look for the deep, underlying mathematical principles that allowed a human or artificial visual system to look at the world and make accurate inferences about what the system saw or perceived.

“The breadth of his research was really quite remarkable,” says Josh Tenenbaum, MIT professor of computational cognitive science and former Richards graduate student. “As his career developed, he transitioned from studying the parts of vision that are very close to neural mechanisms, to computational representations of perception, to Bayesian statistical models of perception and cognition. He became almost a computational social scientist — he was incredibly flexible in his thinking.”

Richards’ passionate advocacy for the computational approach to studying visual perception helped to create and nurture the department’s early computational research initiatives.

“Whit’s connection with David Marr back in the late ’70s is really the genesis of modern computational social science today,” says MIT Professor Alex Pentland, the Toshiba Professor of Media Arts and Science and a former Richards graduate student.

Alongside his impressive research legacy, which includes the publication of eight books and over 200 articles, Richards was also regarded by his students and colleagues as a superlative mentor. Many of his former students have found success in a variety of different fields, including psychology, cognitive science, computer science, media, computer graphics, and the defense industry.

“Whitman was an incredibly dedicated advisor. His strategy was to have very few students and make a huge personal investment in each of them,” says John Rubin, a former graduate student of Richards and current executive producer with Tangled Bank Studios at the Howard Hughes Medical Institute. “He was really great at keeping enthusiasm high in his lab, which took all kinds of forms, but included croquet parties at his home, which were terrifically fun. He was always available and, in fact, it was hard for me to keep up with the amount of time he wanted to devote to our joint work! He was indefatigable and devoted.”

Richards is survived by his wife of 54 years, Waltraud Weller Richards, and three daughters: Diana Richards Doyle and husband Mark S. Doyle of Green Cove Springs, Florida; Sylvia Richards-Gerngross and husband Tillman Gerngross of Hanover, New Hampshire; and Eleanor “Nora” Richards Bender and husband Thomas A. Bender of Dedham, Massachusetts. He is also survived by his two siblings: Lincoln K. Richards and wife Gerda of Wellesley, Massachusetts, and Sylvia Richards Messner of Cave Creek, Arizona; and by two grandchildren, Morgan Kelly Doyle and Serafina Richards-Gerngross. Memorial services will be private.

To demonstrate Foundry, MIT researchers designed and fabricated skis with retro-reflective surfaces. Image: Kiril Vimidce/MIT CSAIL

3-D printing has progressed over the last decade to include multi-material fabrication, enabling production of powerful, functional objects. While many advances have been made, it still has been difficult for non-programmers to create objects made of many materials (or mixtures of materials) without a more user-friendly interface.

But this week, a team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) will present “Foundry,” a system for custom-designing a variety of 3-D printed objects with multiple materials.

“In traditional manufacturing, objects made of different materials are manufactured via separate processes and then assembled with an adhesive or another binding process,” says PhD student Kiril Vidimče, who is first author on the paper. “Even existing multi-material 3-D printers have a similar workflow: parts are designed in traditional CAD [computer-aided-design] systems one at a time and then the print software allows the user to assign a single material to each part.”

In contrast, Foundry allows users to vary the material properties at a very fine resolution that hasn’t been possible before.

“It’s like Photoshop for 3-D materials, allowing you to design objects made of new composite materials that have the optimal mechanical, thermal, and conductive properties that you need for a given task,” says Vidimče. “You are only constrained by your creativity and your ideas on how to combine materials in novel ways.”

To demonstrate, the team designed and fabricated a ping-pong paddle, skis with retro-reflective surfaces, a tricycle wheel, a helmet, and even a bone that could someday be used for surgical planning.

Image: Kiril Vimidce/MIT CSAIL

Redesigning multi-material objects in existing design tools would take experienced engineers many days — and some designs would actually be completely infeasible. With Foundry, you can create these designs in minutes.

“3-D printing is about more than just clicking a button and seeing the product,” Vidimče says. “It’s about printing things that can’t currently be made with traditional manufacturing.”

The paper’s co-authors include MIT Professor Wojciech Matusik and students from his Computational Fabrication Group: PhD student Alexandre Kaspar and former graduate student Ye Wang. The paper will be presented later this week at the Association for Computing Machinery’s User Interface Software and Technology Symposium (UIST) in Tokyo.

How it works

Today’s multi-material 3-D printers are mostly used for prototyping, because the materials currently used are not very functional. Users typically create preliminary models, make rapid adjustments, and then print them again. New platforms such as MIT’s MultiFab are developing highly functional materials appropriate for volume manufacturing.

Foundry, meanwhile, serves as the interface to help create such objects. To use it, you first design your object in a traditional CAD package like SolidWorks. Once the file is exported, you can determine the object’s composition by creating an “operator graph” that can include any of approximately 100 fine-tuned actions called “operators.”

Bike seat design. Image: Kiril Vimidce/MIT CSAIL

Operators can “subdivide,” “remap,” or “assign” materials. Some operators cleanly divide an object into two or more different materials, while others provide more of a gradual shift from one material to another.

Foundry lets you mix and match any combination of materials and also assign specific properties to different parts of the object, combining operators together to make new ones.

For example, if you want to make a cube that is both rigid and elastic, you would assign a “rigid operator” to make one part rigid and an “elastomer operator” to the other part elastic; a third “gradient operator” connects the two and introduces a gradual transition between materials.

Users can preview their design in real-time, rather than having to wait until the final steps in the printing process to see what it will look like.

Testing the system

To test Foundry, the team tried the system on non-designers. They were given three different objects to reproduce: a teddy bear, a bone structure, and an integrated “tweel” (tire and wheel). With just an hour’s explanation, users could design the bone, tire wheel, and teddy bear in an average of 56, 48, and 26 minutes, respectively.

In addition to the user study, the team also fabricated a custom wheel for a toddler tricycle. The wheel had an improved structure to maximize lateral strength, and a foam outer wheel for improved suspension.

Image: Kiril Vimidce/MIT CSAIL

Using Foundry to exploit the full capabilities of the 3-D printing platform enables many practical applications in medicine and more. Surgeons could create high-quality replicas of objects like bones to practice on, while doctors could also develop more comfortable dentures and other products that would benefit from having both soft and rigid components.

Vidimče’s ultimate dream is for Foundry to create a community of designers who can share new operators with each other to expand the possibilities of what can be produced. He also hopes to integrate Foundry into the workflow of existing CAD systems.

“The user should be able to iterate on the material composition in a similar manner to how they iterate on the geometry of the part being designed,” Vidimče says. “Integrating physics simulations to predict the behavior of the part will allow rapid iteration on the final design.”

The research was supported by the National Science Foundation. Click here to read the research paper.

Credit: Jason Dorfman, MIT CSAIL

Anyone who’s watched drone videos or an episode of BattleBots knows that robots can break – and often it’s because they don’t have the proper padding to protect themselves.

But this week researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) will present a new method for 3-D printing soft materials that make robots safer, more resilient, and more precise in their movements – and that could be used to improve the durability of drones, phones, shoes, helmets and more.

The team’s “Programmable Viscoelastic Material” (PVM) technique allows users to program every single part of a 3D-printed object to the exact levels of stiffness and elasticity they want, depending on the task they need for it.

For example, after 3-D printing a cube robot that moves by bouncing, the researchers outfitted it with shock-absorbing “skins” that reduce the amount of energy it transfers to the ground 250 percent.

“That reduction makes all the difference for preventing a rotor from breaking off of a drone or a sensor from cracking when it hits the floor,” says CSAIL director Daniela Rus, who oversaw the project and co-wrote a related paper. “These materials allow us to 3-D print robots with visco-elastic properties that can be inputted by the user at print-time as part of the fabrication process.”

The skins also allow the robot to land nearly four times more precisely, suggesting that similar shock absorbers could be used to help extend the lifespan of delivery drones like the ones being developed by Amazon and Google.

The new paper will be presented at next week’s IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) in Korea. It was written by Rus alongside three postdoctoral associates: lead authors Robert MacCurdy and Jeffrey Lipton, as well as third author Shuguang Li.

Putting a damper on things

There are many reasons for dampers, from controlling the notes of a piano, to keeping car tires on the ground, to protecting structures like radio towers from storms.

The most common damper materials are “viscoelastics” like rubber and plastic that have both solid and liquid qualities. Viscoelastics are cheap, compact and easy to find, but are generally only commercially available in specific sizes and at specific damping levels because of how time-consuming it is to customize them.

The solution, the team realized, was 3-D printing. By being able to deposit materials with different mechanical properties into a design, 3-D printing allows users to “program” material to their exact needs for every single part of an object.

“It’s hard to customize soft objects using existing fabrication methods, since you need to do injection moulding or some other industrial process,” says Lipton. “3-D printing opens up more possibilities and lets us ask the question, ‘can we make things we couldn’t make before?”

Using a standard 3-D printer, the team used a solid, a liquid and a flexible rubber-like material called TangoBlack+ to print both the cube and its skins. The PVM process is related to Rus’ previous 3-D printed robotics work, with an inkjet depositing droplets of different material layer-by-layer and then using UV light to solidify the non-liquids.

The cube robot includes a rigid body, two motors, a microcontroller, battery and IMU sensors. Four layers of looped metal strip serve as the springs that propel the cube.

“By combining multiple materials to achieve properties that are outside the range of the base material, this work pushes the envelope of what’s possible to print,” says Hod Lipson, a professor of engineering at Columbia University and co-author of “Fabricated: The New World of 3-D Printing.” “On top of that, being able to do this in a single print-job raises the bar for additive manufacturing.”

Rus says that PVMs could have many other protective uses, including shock-absorbing running shoes and headgear. By damping the motion brought about by robots’ motors, for example, PVMs are not only able to protect sensitive parts like cameras and sensors, but can also actually make the robots easier to control.

“Being able to program different regions of an object has important implications for things like helmets,” says MacCurdy. “You could have certain parts made of materials that are comfortable for your head to rest on, and other shock-absorbing materials for the sections that are most likely to be impacted in a collision.”

This work was supported by a grant from the National Science Foundation.

Read the research paper here.

From L-R: PhD Fadel Adib, PhD Mingmin Zhao and Professor Dina Katabi demonstrating different ’emotions’ like the picture. Credit: Jason Dorfman, MIT CSAIL

By measuring your heartbeat and breath, this device from MIT’s Computer Science and Artificial Intelligence Lab can tell if you’re excited, happy, angry or sad
.

As many a relationship book can tell you, understanding someone else’s emotions can be a difficult task. Facial expressions aren’t always reliable: a smile can conceal frustration, while a poker face might mask a winning hand.

But what if technology could tell us how someone is really feeling?

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed “EQ-Radio,” a device that can detect a person’s emotions using wireless signals.

By measuring subtle changes in breathing and heart rhythms, EQ-Radio is 87 percent accurate at detecting if a person is excited, happy, angry or sad – and can do so without on-body sensors or facial-recognition software.

MIT professor and project lead Dina Katabi envisions the system being used in entertainment, consumer behavior and health-care. Film studios and ad agencies could test viewers’ reactions in real-time, while smart homes could use information about your mood to adjust the heating or suggest that you get some fresh air.

Professor Dina Katabi (middle) explains how PhD Fadel Adib’s face (right) is neutral, but that EQ-Radio’s analysis of his heartbeat and breathing show that he is sad. Credit: Jason Dorfman MIT CSAIL

“Our work shows that wireless signals can capture information about human behavior that is not always visible to the naked eye,” says Katabi, who co-wrote a paper on the topic with PhD students Mingmin Zhao and Fadel Adib. “We believe that our results could pave the way for future technologies that could help monitor and diagnose conditions like depression and anxiety.”

EQ-Radio builds on Katabi’s continued efforts to use wireless to measure human behavior like breathing and falling. She says that she will incorporate emotion-detection into her spin-off company Emerald, which makes a device that is aimed at detecting and predicting falls among the elderly.

Using wireless signals reflected off people’s bodies, the device measures heartbeats as accurately as an ECG monitor, with a margin of error of approximately 0.3 percent. It then studies the waveforms within each heartbeat to match a person’s behavior to how they previously acted in one of the four emotion-states.

The team will present the work next month at the Association of Computing Machinery’s International Conference on Mobile Computing and Networking (MobiCom).

How it works

Existing emotion-detection methods rely on audiovisual cues or on-body sensors, but there are downsides to both techniques. Cues like facial expressions are famously unreliable, while on-body sensors like chest bands and ECG monitors are inconvenient to wear and become inaccurate if they change position over time.

EQ-Radio instead sends wireless signals that reflect off of a person’s body and back to the device. Its beat-extraction algorithms break the reflections into individual heartbeats and analyze the small variations in heartbeat intervals to determine their levels of arousal and positive affect.

These measurements are what allow EQ-Radio to detect emotion. For example, a person whose signals correlate to low arousal and negative affect is more likely to tagged as sad, while someone whose signals correlate to high arousal and positive affect would likely be tagged as excited.

The exact correlations vary from person to person, but are consistent enough that EQ-Radio could detect emotions with 70 percent accuracy even when it hadn’t previously measured the target person’s heartbeat.

“Just by generally knowing what human heartbeats look like in different emotional states, we can look at a random person’s heartbeat and reliably detect their emotions,” says Zhao.

For the experiments, subjects used videos or music to recall a series of memories that each evoked one the four emotions, as well as a no-emotion baseline. Trained just on those five sets of two-minute videos, EQ-Radio could then accurately classify the person’s behavior among the four emotions 87 percent of the time.

Compared to Microsoft’s vision-based “Emotion API”, which focuses on facial expressions,  EQ-Radio was found to be significantly more accurate in detecting joy, sadness and anger. The two systems performed similarly with neutral emotions, since a face’s absence of emotion is generally easier to detect than its presence.

One of the CSAIL team’s toughest challenges was to tune out irrelevant data. In order to get the heart-rate, for example, the team had to dampen the breathing, since the distance that a person’s chest moves from breathing is much greater than the distance that their heart moves to beat.

To do so, the team focused on wireless signals that are based on acceleration rather than distance traveled, since the rise and fall of the chest with each breath tends to be much more consistent –  and therefore have a lower acceleration – than the motion of the heartbeat.

Although the focus on emotion-detection meant analyzing the time between heartbeats, the team says that the algorithm’s ability to captured the heartbeat’s entire waveform means that in the future it could be used for non-invasive health monitoring and diagnostic settings.

“By recovering measurements of the heart valves actually opening and closing at a millisecond time-scale, this system can literally detect if someone’s heart skips a beat,” says Adib. “This opens up the possibility of learning more about conditions like arrhythmia, and potentially exploring other medical applications that we haven’t even thought of yet.”

Read more about the research here.

Robot from Computer Science and Artificial Intelligence Lab suggests where to move patients and who should do C-sections.

By: Adam Conner-Simons

Today’s robots are awkward co-workers because they are often unable to predict what humans need. In hospitals, robots are employed to perform simple tasks like delivering supplies and medications but they have to be explicitly told what to do.

A team from MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) thinks that this will soon change, and that robots might be most effective by helping humans perform one of the most complex tasks of all: scheduling.

In a pair of new papers, CSAIL researchers demonstrate a robot that, by learning from human workers, can help assign and schedule tasks in fields ranging from medicine to the military.

In one paper, the team demonstrated a robot that assisted nurses in a labor ward, where it made recommendations on everything from where to move a patient to which nurse to assign to a C-section.

“The aim of the work was to develop artificial intelligence that can learn from people about how the labor and delivery unit works, so that robots can better anticipate how to be helpful or when to stay out of the way – and maybe even help by collaborating in making challenging decisions,” says MIT professor Julie Shah, the senior author on both papers.

In a second paper, the same system was put to the test in a videogame that simulates missile-defense scenarios. In the game, which was developed by Lincoln Laboratory researchers and involves using decoy missiles to ward off enemy attacks, the system even occasionally outperformed human experts at reducing both the number of missile attacks and the overall cost of decoys.

The labor-ward paper was presented at the recent Robotics: Science and Systems (RSS) Conference and was co-written by PhD student Matthew Gombolay, CSAIL postdocs Xi Jessie Yang and Brad Hayes, and Dr. Neel Shah and Toni Golen from Beth Israel Deaconess Medical Center, which is where the study took place.

The Navy-simulation paper is being presented at this week’s International Joint Conference on Artificial Intelligence (IJCAI), and was co-written by Gombolay and the  Lincoln Lab’s Reed Jensen, Jessica Stigile and Sung-Hyun Son.

Right on schedule

From visiting hospitals and factories, Shah and Gombolay found that a subset of workers are extremely strong schedulers, but can’t easily transfer that knowledge to colleagues.

“Figuring out what makes certain people good at this often seems like a mystery,” Gombolay says. “Being able to automate the task of learning from experts – and to then generalize it across industries – could help make many businesses run more efficiently.”

A particularly tough place for scheduling are hospitals. Labor wards’ head nurses have to try to predict when a woman will arrive in labor, how long labor will take, and which patients will become sick enough to require C-sections or other procedures.

They are deluged with an endless stream of challenging split-second decisions that include assigning nurses to patients, patients to beds, and technicians to surgeries. At Beth Israel, the head nurse has to coordinate 10 nurses, 20 patients and 20 rooms at the same time, meaning that the number of distinct scheduling possibilities adds up to a staggering 2 to the one millionth power, which is more than the number of atoms in the universe.

“We thought a complex environment like a labor ward would be a good place to try to automate scheduling and take this significant burden off of workers,” says Gombolay.

How it works

Like many AI systems, the team’s robot was trained via “learning from demonstration,” which involves observing humans’ performances of tasks. But Gombolay says that researchers have never been able to apply this technique to scheduling, because of the complexity of coordinating multiple actions that can be very dependent on each other.

To overcome this, the team trained its system to look at several actions that human schedulers make, and compare them to all the possible actions that are not made at each of those moments in time. From there, it developed a scheduling policy that can respond dynamically to new situations that it has not seen before.

“Rather than considering actions in isolation of each other, we crafted a model that understands why one action is better than the alternatives,” says Shah. “By considering all such comparisons, you can learn to recommend which action will be most helpful.”

The policy is “model-free,” meaning that the nurses do not have to train the robot by painstakingly ranking each possible action in each possible scenario by hand.

“You can put the robot on the labor floor, and, just by watching humans doing the different tasks, it will understand how to coordinate an efficient schedule,” says Gombolay.

The results

With this framework, the system – which the team has dubbed “apprenticeship scheduling” – can anticipate room assignments and suggest which nurses to assign to patients for C-sections and other procedures.

The approach was evaluated through experiments in which a robot provided decision-support to nurses and doctors as they made decisions on a labor floor.

Using the system on a Nao robot, nurses accepted the robot’s recommendations 90 percent of the time. The team also demonstrated that human subjects weren’t just blindly accepting advice – the robot delivered consciously bad feedback that was rejected at the same rate of 90 percent, showing that the system was trained to distinguish between good and bad recommendations.

Nurses had almost uniformly positive feedback about the robot. One said that it would “allow for a more even dispersion of workload,” while another said that it would be particularly helpful for “new nurses [who] may not understand the constraints and complexities of the role.”

“A great potential of this technology is that good solutions can be spread more quickly to many hospitals and workplaces,” says Dana Kulic, an associate professor of computer engineering at the University of Waterloo. “ For example, innovative improvements can be distributed rapidly from research hospitals to regional health centres.”

Shah says that the new techniques have many uses, from turning robots into better collaborators to helping train new nurses, but the goal is not to develop robots that fully make decisions on their own.

“These initial results show there is tremendous potential for machines to collaborate with us in rich ways that will enhance many sectors of the economy,” says Shah. “The awkward robots of the past will be replaced by valued team members.”

The RSS paper was supported by the National Science Foundation, CRICO Harvard Risk Management Foundation, and Aldebaran Robotics Inc. The IJCAI paper was supported the National Science Foundation and the U.S. Navy.

Read the paper here. 

Action-Prediction Algorithms. Source: MIT CSAIL

When we see two people meet, we can often predict what happens next: a handshake, a hug, or maybe even a kiss. Our ability to anticipate actions is thanks to intuitions born out of a lifetime of experiences.

Machines, on the other hand, have trouble making use of complex knowledge like that. Computer systems that predict actions would open up new possibilities ranging from robots that can better navigate human environments, to emergency response systems that predict falls, to Google Glass-style headsets that feed you suggestions for what to do in different situations.

This week researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have made an important new breakthrough in predictive vision, developing an algorithm that can anticipate interactions more accurately than ever before.

Trained on YouTube videos and TV shows like “The Office” and “Desperate Housewives,” the system can predict whether two individuals will hug, kiss, shake hands or slap five. In a second scenario, it could also anticipate what object is likely to appear in a video five seconds later.

Source: MIT CSAIL

While human greetings may seem like arbitrary actions to predict, the task served as a more easily controllable test case for the researchers to study.

“Humans automatically learn to anticipate actions through experience, which is what made us interested in trying to imbue computers with the same sort of common sense,” says CSAIL PhD student Carl Vondrick, who is first author on a related paper that he will present this week at the International Conference on Computer Vision and Pattern Recognition (CVPR). “We wanted to show that just by watching large amounts of video, computers can gain enough knowledge to consistently make predictions about their surroundings.”

Vondrick’s co-authors include MIT professor Antonio Torralba and former postdoc Hamed Pirsiavash, now a professor at the University of Maryland.

How it works
Past attempts at predictive computer-vision have generally taken one of two approaches.

The first method is to look at an image’s individual pixels and use that knowledge to create a photorealistic “future” image, pixel by pixel – a task that Vondrick describes as “difficult for a professional painter, much less an algorithm.” The second is to have humans label the scene for the computer in advance, which is impractical for being able to predict actions on a large scale.

Source: CSAIL

The CSAIL team instead created an algorithm that can predict “visual representations,” which are basically freeze-frames showing different versions of what the scene might look like.

Source: MIT CSAIL

“Rather than saying that one pixel value is blue, the next one is red, and so on, visual representations reveal information about the larger image, such as a certain collection of pixels that represents a human face,” Vondrick says.

The team’s algorithm employs techniques from deep-learning, a field of artificial intelligence that uses systems called “neural networks” to teach computers to pore over massive amounts of data to find patterns on their own.

Each of the algorithm’s networks predicts a representation is automatically classified as one of the four actions – in this case, a hug, handshake, high-five or kiss. The system then merges those actions into one that it uses as its prediction. For example, three networks might predict a kiss, while another might use the fact that another person has entered the frame as a rationale for predicting a hug instead.

“A video isn’t like a ‘Choose Your Own Adventure’ book where you can see all of the potential paths,” says Vondrick. “The future is inherently ambiguous, so it’s exciting to challenge ourselves to develop a system that uses these representations to anticipate all of the possibilities.”

How it did
After training the algorithm on 600 hours of unlabeled video, the team tested it on new videos showing both actions and objects.

When shown a video of people who are one second away from performing one of the four actions, the algorithm correctly predicted the action more than 43 percent of the time, which compares to existing algorithms that could only do 36 percent of the time.

In a second study, the algorithm was shown a frame from a video and asked to predict what object will appear five seconds later. For example, seeing someone open a microwave might suggest the future presence of a coffee mug. The algorithm predicted the object in the frame 30 percent more accurately than baseline measures, though the researchers caution that it still only has an average precision of 11 percent.

It’s worth noting that even humans make mistakes on these tasks: for example, human subjects were only able to correctly predict the action 71 percent of the time.

“There’s a lot of subtlety to understanding and forecasting human interactions,” says Vondrick. “We hope to be able to work off of this example to be able to soon predict even more complex tasks.”

Looking forward
While the algorithms aren’t yet accurate enough for practical applications, Vondrick says that future versions could be used for everything from robots that develop better action plans to security cameras that can alert emergency responders when someone who has fallen or gotten injured.

“I’m excited to see how much better the algorithms get if we can feed them a lifetime’s worth of videos,” says Vondrick. “We might see some significant improvements that would get us closer to using predictive-vision in real-world situations.””

The work was supported by a grant from the National Science Foundation, along with a Google faculty research award for Torralba and a Google PhD fellowship for Vondrick.

Source: CC0

Nearly 70 years after the “Turing Test” was first proposed, the question remains: can we create intelligent machines that exhibit behavior indistinguishable from humans?

In December MIT researchers helped develop a system that passed a “visual” Turing Test, producing written characters that fool humans. Now, researchers from MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) have demonstrated a deep-learning algorithm that passes the Turing Test for sound: when shown a silent video clip of an object being hit, the algorithm can produce a sound for the hit that is realistic enough to fool human viewers.

The project represents much more than just a clever computer trick: researchers envision future versions of similar algorithms being used to automatically produce sound effects for movies and TV shows, as well as to help robots better understand objects’ properties.

“When you run your finger across a wine glass, the sound it makes reflects how much liquid is in it,” says CSAIL PhD student Andrew Owens, who was lead author on the paper. “An algorithm that models such sounds can reveal key information about objects’ shapes and material types, as well as the force and motion of their interactions with the world.”

The team used techniques from the field of “deep learning,” which involves teaching computers to sift through huge amounts of data to find patterns on their own. Deep learning approaches are especially useful because they free computer scientists from having to hand-design algorithms and supervise their progress.

The paper’s co-authors include recent PhD graduate Phillip Isola and MIT professors Edward Adelson, Bill Freeman, Josh McDermott and Antonio Torralba. The paper will be presented later this month at the annual conference on Computer Vision and Pattern Recognition (CVPR) in Las Vegas.

How it works
The first step to training a sound-producing algorithm is to give it sounds to study. Over several months, the researchers recorded roughly 1,000 videos of an estimated 46,000 sounds that represent various objects being hit, scraped and prodded with a drumstick. (They used a drumstick because it provided a consistent way to produce a sound.)

Credit: MIT CSAIL

Next, the team fed those videos to a deep-learning algorithm that deconstructed the sounds and analyzed their pitch, loudness and other features.

Credit: MIT CSAIL

“To then predict the sound of a new video, the algorithm looks at the sound properties of each frame of that video, and matches them to the most similar sounds in the database,” says Owens. “Once the system has those bits of audio, it stitches them together to create one coherent sound.”

The result is that the algorithm can accurately simulate the subtleties of different hits, from the staccato taps of a rock to the longer waveforms of rustling ivy. Pitch is no problem either, as it can synthesize hit-sounds ranging from the low-pitched “thuds” of a  soft couch to the high-pitched “clicks” of a hard wood railing.

An additional benefit of the work is that the team’s library of 46,000 sounds is free and available for other researchers to use. The name of the dataset: “Greatest Hits.”

Fooling humans
To test how realistic the fake sounds were, the team conducted an online study in which subjects saw two videos of collisions – one with the actual recorded sound, and one with the algorithm’s – and were asked which one was real.

The result: subjects picked the fake sound over the real one twice as often as a baseline algorithm. They were particularly fooled by materials like leaves and dirt that tend to have less “clean” sounds than, say, wood or metal.

On top of that, the team found that the materials’ sounds revealed key aspects of their physical properties: an algorithm they developed could tell the difference between hard and soft materials 67 percent of the time.

The team’s work aligns with recent CSAIL research on audio and video amplification. Freeman has helped develop algorithms that amplify movements captured by video that are invisible to the naked eye, which has allowed his groups to do things like make the human pulse visible and even recover speech using nothing more than video of a potato chip bag.

Looking ahead
Researchers say that there’s still room to improve the system. For example, if the drumstick moves especially erratically in a video, the algorithm is more likely to miss or hallucinate a false hit. It is also limited by the fact that it applies only to “visually indicated sounds” – sounds that are directly caused by the physical interaction that is being depicted in the video.

“From the gentle blowing of the wind to the buzzing of laptops, at any given moment there are so many ambient sounds that aren’t related to what we’re actually looking at,” says Owens. “What would be really exciting is to somehow simulate sound that is less directly associated to the visuals.”

The team believe that future work in this area could improve robots’ abilities to interact with their surroundings.

“A robot could look at a sidewalk and instinctively know that the cement is hard and the grass is soft, and therefore know what would happen if they stepped on either of them,” says Owens. “Being able to predict sound is an important first step toward being able to predict the consequences of physical interactions with the world.”

The work was funded in part by the National Science Foundation and Shell. Owens was also supported by a Microsoft Research Fellowship.

Read the paper here. 

By Adam Conner-Simons, MIT CSAIL

This week MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) received an unusual package: a six-foot-tall, 300-pound humanoid robot that NASA hopes to have serve on future space missions to Mars and beyond.

A team of researchers led by CSAIL principal investigator Russ Tedrake will program their new “Valkyrie” robot to autonomously perform a variety of challenging tasks that would allow it to help or even replace astronauts on missions.

Valkyrie is fully electric, with four body cameras, 28 torque-controlled joints and 44 degrees of freedom. The robot boasts more than 200 individual sensors, including 38 on each hand (six on each palm, and eight along each of its four fingers).

Other researchers participating in the project include professors Leslie Kaelbling and Tomas Lozano-Perez, who will conduct work on high-level autonomy.

“Our work is about vetting the robot and seeing what it is capable of,” says Tedrake, whose team received a two-year research grant from NASA for the project. “If we can integrate the autonomy work with our planning and control algorithms, it could result in an unprecedented level of autonomous capabilities for a humanoid robot.’

Tedrake’s team at CSAIL’s Robot Locomotion Group has extensive experience developing autonomous robots. The group spent the last three years doing research as part of the DARPA Robotics Challenge, where they programmed another six-foot-tall robot named Atlas to complete a series of tasks that included opening doors, turning valves, drilling holes, climbing stairs and driving a car.

Besides the CSAIL team, NASA also awarded a Valkyrie robot to Northeastern University in conjunction with the University of Massachusetts at Lowell.

This 3-D hexapod robot moves via a single motor, which spins a crankshaft that pumps fluid to the robot’s legs. Besides the motor and battery, every component is printed in a single step with no assembly required. Among the robot’s key parts are several sets of “bellows” 3-D printed directly into its body. To propel the robot, the bellows uses fluid pressure that is translated into a mechanical force. (As an alternative to the bellows, the team also demonstrated they could 3-D print a gear pump that can produce continuous fluid flow.)
Photo: Robert MacCurdy/MIT CSAIL

By Adam Conner-Simons | CSAIL

One reason we don’t yet have robot personal assistants buzzing around doing our chores is because making them is hard. Assembling robots by hand is time-consuming, while automation — robots building other robots — is not yet fine-tuned enough to make robots that can do complex tasks.

But if humans and robots can’t do the trick, what about 3-D printers?

In a new paper, researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) present the first-ever technique for 3-D printing robots that involves printing solid and liquid materials at the same time.

The new method allows the team to automatically 3-D print dynamic robots in a single step, with no assembly required, using a commercially-available 3-D printer.

“Our approach, which we call ‘printable hydraulics,’ is a step towards the rapid fabrication of functional machines,” says CSAIL Director Daniela Rus, who oversaw the project and co-wrote the paper. “All you have to do is stick in a battery and motor, and you have a robot that can practically walk right out of the printer.”

To demonstrate the concept, researchers 3-D printed a tiny six-legged robot that can crawl via 12 hydraulic pumps embedded within its body. They also 3-D printed robotic parts that can be used on existing platforms, such as a soft rubber hand for the Baxter research robot.

The paper, which was recently accepted to this summer’s IEEE International Conference on Robotics and Automation (ICRA), was co-written by MIT postdoc Robert MacCurdy and PhD candidate Robert Katzschmann, as well as Harvard University undergraduate Youbin Kim.

The printing process

For all of the progress in 3-D printing, liquids continue to be a big hurdle. Printing liquids is a messy process, which means that most approaches require an additional post-printing step such as melting it away or having a human manually scrape it clean. That step makes it hard for liquid-based methods to be employed for factory-scale manufacturing.

The team’s method of printing solid and liquid materials simultaneously allows them to create robotic structures that can be hydraulically powered.
Photo: Robert MacCurdy/MIT CSAIL

With “printable hydraulics,” an inkjet printer deposits individual droplets of material that are each 20 to 30 microns in diameter, or less than half the width of a human hair. The printer proceeds layer-by-layer from the bottom up. For each layer, the printer deposits different materials in different parts, and then uses high-intensity UV light to solidify all of the materials (minus, of course, the liquids). The printer uses multiple materials, though at a more basic level each layer consist of a “photopolymer,” which is a solid, and “a non-curing material,” which is a liquid.

“Inkjet printing lets us have eight different print-heads deposit different materials adjacent to one another, all at the same time,” MacCurdy says. “It gives us very fine control of material placement, which is what allows us to print complex, pre-filled fluidic channels.”

Another challenge with 3-D printing liquids is that they often interfere with the droplets that are supposed to solidify. To handle that issue, the team printed dozens of test geometries with different orientations to determine the proper resolutions for printing solids and liquids together.

While it’s a painstaking process, MacCurdy says that printing both liquids and solids is even more difficult with other 3-D printing methods, such as fused-deposition modeling and laser-sintering.

“As far as I’m concerned,” he says, “inkjet-printing is currently the best way to print multiple materials.”

The results

To demonstrate their method, researchers 3-D printed a small hexapod robot that weighs about 1.5 pounds and is less than 6 inches long. To move, a single DC motor spins a crankshaft that pumps fluid to the robot’s legs. Aside from its motor and power supply, every component is printed in a single step with no assembly required.

Among the robot’s key parts are several set of “bellows” that are 3-D printed directly into its body. To propel the robot, the bellows uses fluid pressure that is then translated into a mechanical force. (As an alternative to the bellows, the team also demonstrated they could 3-D print a gear pump that can produce continuous fluid flow.)

Photo: Robert MacCurdy/MIT CSAIL

Lastly, the team 3-D printed a silicone-rubber robotic hand with fluid-actuated fingers. This “soft gripper” was developed for Baxter, a robot that was designed by former CSAIL director Rodney Brooks as part of his spinoff company Rethink Robotics.

“The CSAIL team has taken multi-material printing to the next level by printing not just a combination of different polymers or a mixture of metals, but essentially a self-contained working hydraulic system,” says Hod Lipson, a professor of engineering at Columbia University and co-author of “Fabricated: The New World of 3-D Printing.” “It’s an important step towards the next big phase of 3-D printing — moving from printing passive parts to printing active integrated systems.”

Compatible with any multimaterial 3-D inkjet printer, “printable hydraulics” allows for a customizable design template that can create robots of different sizes, shapes and functions.

“If you have a crawling robot that you want to have step over something larger, you can tweak the design in a matter of minutes,” MacCurdy says. “In the future, the system will hardly need any human input at all; you can just press a few buttons, and it will automatically make the changes.”

MacCurdy envisions many potential applications, including disaster relief in dangerous environments. Many nuclear sites, for example, need to be remediated to reduce their radiation levels. Unfortunately, the sites are not only lethal to humans, but radioactive enough to destroy conventional electronics.

“Printable robots like these can be quickly, cheaply fabricated, with fewer electronic components than traditional robots,” MacCurdy says.

Looking ahead

The team is eager to further build on their work. While the hexapod’s 22-hour print-time is relatively short for its complexity, researchers say that future hardware advances would improve the speed.

“Accelerating the process depends less on the particulars of our technique, and more on the engineering and resolution of the printers themselves,” says Rus, the Viterbi Professor of Electrical Engineering and Computer Science at MIT. “Printing ultimately takes as long as the printer takes, so as printers improve, so will the manufacturing capabilities.”

This isn’t Rus’ group’s first foray into 3-D printed robots. This past fall her team developed a similar gripper, while in 2014 they created an arm that can snake through a pipe and grasp an object. But where these projects still required multiple non-3-D printed objects, “printable hydraulics” gets even closer to printing all components in one step.

“Building robots doesn’t have to be as time-consuming and labor-intensive as it’s been in the past,” Rus says. “3-D printing offers a way forward, allowing us to automatically produce complex, functional, hydraulically-powered robots that can be put to immediate use.”

Read the research here

The team’s work was funded, in part, by a grant from the National Science Foundation.

According to CSAIL researchers, a new wireless technology they’ve developed could mean safer drones, smarter homes and password-free WiFi. Source: MIT/CSAIL

By Adam Conner-Simons | CSAIL

We’ve all been there, impatiently twiddling our thumbs while trying to locate a WiFi signal. But what if, instead, the WiFi could locate us?

According to researchers at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL), it could mean safer drones, smarter homes and password-free WiFi.

In a new paper, a research team led by professor Dina Katabi presents a system called Chronos that enables a single WiFi access point to locate users to within tens of centimeters, without any external sensors.

The group demonstrated Chronos in an apartment and a cafe, while also showing off a drone that maintains a safe distance from its user with a margin of error of about four centimeters.

“From developing drones that are safer for people to be around, to tracking where family members are in your house, Chronos could open up new avenues for using WiFi in robotics, home automation and more,” says PhD student Deepak Vasisht, who is first author on the paper alongside Katabi and former PhD student Swarun Kumar, who is now an assistant professor at Carnegie Mellon University. “Designing a system that enables one WiFi node to locate another is an important step for wireless technology.”

Experiments conducted in a two-bedroom apartment with four occupants show that Chronos can correctly identify which room a resident is in 94 percent of the time. For the cafe demo, the system was 97 percent accurate in distinguishing in-store customers from out-of-store intruders – meaning it could be used by small businesses to prevent non-customers from stealing their WiFi. (32 percent of Americans have copped to this cyber-crime.)

Chronos locates users by calculating the “time-of-flight” that it takes for data to travel from the user to an access point. The system is 20 times more accurate than existing systems, computing time-of-flight with an average error of 0.47 nanoseconds, or half than one-billionth of a second.

Vasisht presented the paper at this month’s USENIX Symposium on Networked Systems Design and Implementation (NSDI ’16).

How it works

Existing localization methods have required four or five WiFi access points. This is because today’s WiFi devices don’t have wide enough bandwidth to measure time-of-flight, and so researchers have only been able to determine someone’s position by triangulating multiple angles relative to the person.

What Chronos adds is the ability to calculate not just the angle, but the actual distance from a user to an access point, as determined by multiplying the time-of-flight by the speed of light.

“Knowing both the distance and the angle allows you to compute the user’s position using just one access point,” says Deepak Vasisht. “This is encouraging news for the many small businesses and consumers that don’t have the luxury of owning several access points.”

Exploiting the fact that WiFi lets you hop on different frequency channels, the team programmed the system to jump from channel to channel, gathering many different measurements of the distance between access points and the user. Chronos then automatically “stitches” together these measurements to determine the distance.

“By devising a method to rapidly hop across these channels that span almost one gigahertz of bandwidth, Chronos can measure time-of-flight with sub-nanosecond accuracy, emulating with commercial WiFi what has previously needed an expensive ultra-wideband radio,” says Venkat Padmanabhan. a principal researcher at Microsoft Research India. “This is an impressive breakthrough and promises to be a key enabler for applications such as high-accuracy indoor localization.”

That said, getting an accurate time-of-flight with this method still isn’t easy, due to three sets of delays that happen during the transfer.

First, when you wirelessly send a piece of web data, there is a delay in detecting the presence of the “packet” that is hard to distinguish from the actual time-of-flight. To account for it, the team exploits the fact that WiFi uses an encoding method that transmits bits of packets on several even smaller frequencies.

Secondly, if you’re indoors the WiFi signals can bounce off walls and furniture, meaning that the receiver gets several copies of the signal that each experience different times-of-flight. To identify the actual direct path, researchers developed a mechanism to algorithmically determine the delays experienced by all of these copies. From there, they can identify the path with the smallest time-of-flight as the direct path.

Lastly, the team’s channel-hopping approach leads to one other complication: every time Chronos hops to a new band, the hardware resets, adding a delay known as a “phase offset.” To address this the team used the fact that in WiFi, you get an acknowledgement back for each data packet that your phone sends. The team uses these acknowledgements to intelligently cancel out the phase offsets.

The success of Chronos suggests that WiFi-based positioning could help for other situations where there are limited or inaccessible sensors, like finding lost devices or controlling large fleets of drones.

“Imagine having a system like this at home that can continuously adapt the heating and cooling depending on number of people in the home and where they are” says Katabi. “Eliminating the need for cooperation between WiFi routers opens up many exciting new applications for localization.”

In a forest simulation, algorithms segment space into “obstacle-free regions” and then link them together to find a single collision-free route.
Photo courtesy of CSAIL researchers.

Adam Conner-Simons | CSAIL

Getting drones to fly around without hitting things is no small task. Obstacle-detection and motion-planning are two of computer science’s trickiest challenges, because of the complexity involved in creating real-time flight plans that avoid obstacles and handle surprises like wind and weather. In a pair of projects announced this week, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) demonstrated software that allow drones to stop on a dime to make hairpin movements over, under, and around some 26 distinct obstacles in a simulated “forest.”

One team’s video shows a small quadrotor doing donuts and figure-eights through an obstacle course of strings and PVC pipes. Weighing just over an ounce and clocking in at 3 and a half inches from rotor to rotor, the drone can fly through the 10-square-foot space at speeds upwards of 1 meter per second.

The team’s algorithms — which are available online and were previously used to plan footsteps for CSAIL’s Atlas robot at last year’s DARPA Robotics Challenge — segment space into “obstacle-free regions” and then link them together to find a single collision-free route.

“Rather than plan paths based on the number of obstacles in the environment, it’s much more manageable to look at the inverse: the segments of space that are ‘free’ for the drone to travel through,” says recent graduate Benoit Landry ’14 MNG ’15, who was first author on a related paper just accepted to the IEEE International Conference on Robotics and Automation (ICRA). “Using free-space segments is a more ‘glass-half-full’ approach that works far better for drones in small, cluttered spaces.”

In a second CSAIL project, PhD student Anirudha Majumdar showed off a fixed-wing plane that is guaranteed to avoid obstacles without any advanced knowledge of the space, and even in the face of wind gusts and other dynamics. His approach was to pre-program a library of dozens of distinct “funnels” that represent the worst-case behavior of the system, calculated via a rigorous verification algorithm.

“As the drone flies, it continuously searches through the library to stitch together a series of paths that are computationally guaranteed to avoid obstacles,” says Majumdar, who was lead author on a related technical report. “Many of the individual funnels will not be collision-free, but with a large-enough library you can be certain that your route will be clear.”

Both papers were co-authored by MIT professor Russ Tedrake; the ICRA paper, which will be presented in May in Sweden, was also co-written by PhD students Robin Deits and Peter R. Florence.

Drones in high-density

A bird might make it seem simple, but flight is a highly complicated endeavor. A flying object can change position in six distinct directions — forward/backward (“surge”), up/down (“heave”), left/right (“sway”), and by rotating front-to-back (“pitch”), side-to-side (“roll”), and horizontally (“yaw”).

“At every moment in time there are 12 distinct numbers needed to describe where the system it is and how quickly it is moving, on top of simultaneously tracking other objects in the space that could get in your way,” says Majumdar. “Most techniques typically can’t handle this sort of complexity in real-time.”

One common motion-planning approach is to sample the whole space through algorithms like the “rapidly-exploring random tree.” Although often effective, sampling-based approaches are generally less efficient and have trouble navigating small gaps between obstacles.

Landry’s team opted to use Deits’ new free-space-based technique, which he calls the “Iterative Regional Inflation by semidefinite programming” algorithm (IRIS). They then coupled IRIS with a “mixed-integer semidefinite program” (MISDP) that assigns specific flight movements to each “space-free region” and then executes the full plan.

To sense its surroundings, the drone used motion-capture optical sensors and an on-board inertial measurement unit (IMU) that help estimate the precise positioning of obstacles.

“I’m most impressed by the team’s ingenious technique of combining on- and off-board sensors to determine the drone’s location,” says Jingjin Yu, an assistant professor of computer science at Rutgers University. “This is key to the system’s ability to create unique routes for each set of obstacles.”

In its current form, MISDP has been optimized such that it can’t do real-time planning; it takes an average of 10 minutes to create a route for the obstacle course. But Landry says that making certain sacrifices would let them generate plans much more quickly.

“For example, you could define ‘free-space regions’ more broadly as links between areas where two or more free-space regions overlap,” says Landry. “That would let you solve for a general motion-plan through those links, and then fill in the details with specific paths inside of the chosen regions. Currently we solve both problems at the same time to lower energy consumption, but if we wanted to run plans faster that would be a good option.”

Majumdar’s software, meanwhile, generates more conservative plans, but can do so in real-time. He first developed a library of 40 to 50 trajectories that are each given an outer bound that the drone is guaranteed to remain within. These bounds can be visualized as ”funnels” that the planning algorithm chooses between to stitch together a sequence of steps that allow the drone to plan its flying on the fly.

A flexible approach like this comes with a high level of guarantees that the software will work, even in the face of uncertainties with both the surroundings and the hardware itself. The algorithm can easily be extended to drones of different sizes and payloads, as well as ground vehicles and walking robots.

As for the environment, imagine the drone choosing between making a forceful roll maneuver that will avoid a tree by a large margin, versus flying straight and avoiding a tree by a small amount.

“A traditional approach might prefer the first since avoiding obstacles by a significant amount seems ‘safer,’” Majumdar says. “But a move like that actually may be riskier because it’s more susceptible to wind gusts. Our method makes these decisions in real-time, which is critical if we want drones to move out of the labs and operate in real-world scenarios.”

A clear path to avoiding obstacles

In a related project, a planner searches a library of pre-computed funnels and selects one that doesn’t intersect with the obstacles.
Photo courtesy of CSAIL researchers.

CSAIL researchers have been working on this problem for many years. Professor Nick Roy has been honing algorithms for drones to develop maps and avoid objects in real-time; in November a team led by PhD student Andrew Barry published a video demonstrating algorithms that allow a drone to dart between trees at speeds of 30 miles per hour.

While these two drones cannot travel quite as fast as Barry’s, their maneuvers are generally more complex, meaning that they can navigate in smaller, denser environments.

“Enabling dynamic flight of small, off-the-shelf quadcopters is a marvelous achievement, and one that has many potential applications,” Yu says. “With additional development, I can imagine these machines being used as probes in hard-to-reach places, from exploring caves to doing search-and-rescue in collapsed buildings.”

Landry, who now works for 3D Robotics in California, is hopeful that other academics will build on and refine the researchers’ work, which is all open-source and available on github.

“A big challenge for industry is determining which technologies are actually mature enough to use in real products,” Landry says. “The best way to do that is to conduct experiments that focus on all of the corner cases and can demonstrate that algorithms like these will actually work 99.999 percent of the time.”

Landry’s work was partially supported by the Siebel Scholars Foundation, while Majumdar’s work was supported by a grant from the Office of Naval Research.

Papers

• Aggressive quadrotor flight through cluttered environments using mixed integer programming”
• Funnel Libraries for Real-Time Robust Feedback Motion Planning

If you liked this post, you may also be interested in:

• “Shrinking bull’s-eye” algorithm speeds up complex modeling from days to hours
• The Master Algorithm, with UW’s Pedro Domingos
•  Quadrocopter failsafe algorithm: Recovery after propeller loss
• ‘Tailsitter’ flying robot hovers and recovers easily thanks to new algorithm
• Robots Podcast #194: Embodied quadrotors, with Davide Scaramuzza

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

We humans take for granted our remarkable ability to predict things that happen around us. For example, consider Rube Goldberg machines: One of the reasons we enjoy them is because we can watch a chain-reaction of objects fall, roll, slide and collide, and anticipate what happens next.

But how do we do it? How do we effortlessly absorb enough information from the world to be able to react to our surroundings in real-time? And, as a computer scientist might then wonder, is this something that we can teach machines?

That last question has recently been partially answered by researchers at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL), who have developed a computational model that is just as accurate as humans at predicting how objects move.

By training itself on real-world videos and using a “3-D physics engine” to simulate human intuition, the system — dubbed “Galileo” — can infer the physical properties of objects and predict the outcome of a variety of physical events.

While the researchers’ paper focused on relatively simple experiments involving ramps and collisions, they say that the model’s ability to generalize its findings and continuously improve itself means that it could readily predict a range of actions.

“From a ramp scenario, for example, Galileo can infer the density of an object and then predict if it can float,” says postdoc Ilker Yildirim, who was lead author alongside CSAIL PhD student Jiajun Wu. “This is just the first step in imbuing computers with a deeper understanding of dynamic scenes as they unfold.”

The paper, which was presented this past month at the Conference on Neural Information Processing Systems (NIPS) in Montreal, was co-authored by postdoc Joseph Lim and professor William Freeman, as well as professor Joshua Tenenbaum from the Department of Brain and Cognitive Sciences.

How they did it
Recent research in neuroscience suggests that in order for humans to understand a scene and predict events within it, our brains rely on a mental “physics engine” consisting of detailed but noisy knowledge about the physical laws that govern objects and the larger world.

Using the human framework to develop their model, the researchers first trained Galileo on a set of 150 videos that depict physical events involving objects of 15 different materials, from cardboard and foam to metal and rubber. This training allowed the model to generate a dataset of objects and their various physical properties, including shape, volume, mass, friction, and position in space.

From there, the team fed the model information from Bullet, a 3-D physics engine often used to create special effects for movies and video games. By inputting the setup of a given scene and then physically simulating it forward in time, Bullet serves as a reality check against Galileo’s hypotheses.

Finally, the team developed deep-learning algorithms that allow the model to teach itself to further improve its predictions to the point that, by the very first frame of a video, Galileo can recognize the objects in the scene, infer the properties of the objects, and determine how these objects will interact with one another.

“Humans learn physical properties by actively interacting with the world, but for our computers this is tricky because there is no training data,” says Abhinav Gupta, an assistant professor of computer science at Carnegie Mellon University. “This paper solves this problem in a beautiful manner, by combining deep-learning convolutional networks with classical AI ideas like simulation engines.”

Heat maps of user predictions, Galileo outputs (orange crosses) and ground truths (white crosses). Source: CSAIL

Human vs. machine
To assess Galileo’s predictive powers, the team pitted it against human subjects to predict a series of simulations (including one that has an interactive online demo).

In one, users see a series of object collisions, and then another video that stops at the moment of collision. The users are then asked to label how far they think the object will move.

“The scenario seems simple, but there are many different physical forces that make it difficult for a computer model to predict, from the objects’ relative mass and elasticity to gravity and the friction between surface and object,” Yildirim says. “Where humans learn to make such judgments intuitively, we essentially had to teach the system each of these properties and how they impact each other collectively.”

In another simulation, users first see a collision involving a 20-degree inclined ramp, and then are shown the first frame of a video with a 10-degree ramp and asked to predict whether the object will slide down the surface.

“Interestingly, both the computer model and human subjects perform this task at chance and have a bias at saying that the object will move,” Yildirim says. “This suggests not only that humans and computers make similar errors, but provides further evidence that human scene understanding can be best described as probabilistic simulation.”

Researchers fed their physics prediction system videos of collisions, shown as screenshots in (a) and (b), which were then converted into simulations generated by the 3-D physics engine, shown in (c) and (d).

What’s next
The team members say that they plan to extend the research to more complex scenarios involving fluids, springs, and other materials. Continued progress in this line of work, they say, could lead to direct applications in robotics and artificial intelligence.

“Imagine a robot that can readily adapt to an extreme physical event like a tornado or an earthquake,” Lim says. “Ultimately, our goal is to create flexible models that can assist humans in settings like that, where there is significant uncertainty.”

By Adam Conner-Simons, MIT CSAIL

A researcher from MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) has developed an obstacle-detection system that allows a drone to autonomously dip, dart and dive through a tree-filled field at upwards of 30 miles per hour.

“Everyone is building drones these days, but nobody knows how to get them to stop running into things,” says CSAIL PhD student Andrew Barry, who developed the system as part of his thesis with MIT professor Russ Tedrake. “Sensors like lidar are too heavy to put on small aircraft, and creating maps of the environment in advance isn’t practical. If we want drones that can fly quickly and navigate in the real world, we need better, faster algorithms.”

Running 20 times faster than existing software, Barry’s stereo-vision algorithm allows the drone to detect objects and build a full map of its surroundings in real-time. Operating at 120 frames per second, the software – which is open-source and available online – extracts depth information at a speed of 8.3 milliseconds per frame.

The drone, which weighs just over a pound and has a 34-inch wingspan, was made from off-the-shelf components costing about $1,700, including a camera on each wing and two processors no fancier than the ones you’d find on a cellphone.

How it works

Traditional algorithms focused on this problem would use the images captured by each camera, and search through the depth-field at multiple distances – 1 meter, 2 meters, 3 meters, and so on – to determine if an object is in the drone’s path.

Such approaches, however, are computationally intensive, meaning that the drone cannot fly any faster than five or six miles per hour without specialized processing hardware.

Barry’s realization was that, at the fast speeds that his drone could travel, the world simply does not change much between frames. Because of that, he could get away with computing just a small subset of measurements – specifically, distances of 10 meters away.

“You don’t have to know about anything that’s closer or further than that,” Barry says. “As you fly, you push that 10-meter horizon forward, and, as long as your first 10 meters are clear, you can build a full map of the world around you.”

While such a method might seem limiting, the software can quickly recover the missing depth information by integrating results from the drone’s odometry and previous distances.

Barry says that he hopes to further improve the algorithms so that they can work at more than one depth, and in environments as dense as a thick forest.

“Our current approach results in occasional incorrect estimates known as ‘drift,’” he says. “As hardware advances allow for more complex computation, we will be able to search at multiple depths and therefore check and correct our estimates. This lets us make our algorithms more aggressive, even in environments with larger numbers of obstacles.”

LINKS

Paper: “Pushbroom Stereo for High-Speed Navigation in Cluttered Environments”

CSAIL’s Robot Locomotion Group

Andrew Barry

Russ Tedrake

RELATED NEWS STORIES

MIT News: “Charging solution for delivery drones: take after our feathered friends”

MIT News: “Autonomous robot flies indoors”

If you liked this article, you may also be interested in:

• What we know about the FAA’s new drone registration requirement, and what we don’t
• Robots Podcast #184: CyPhy LVL 1 Drone, with Helen Griener
• Big bucks and big audiences for drones
• Video: Qualcomm demos Snapdragon Flight for smaller, lighter camera drones

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Team’s silicone rubber gripper can pick up egg, CD & paper, and identify objects by touch alone

By Adam Conner-Simons, MIT CSAIL

Robots have many strong suits, but delicacy traditionally hasn’t been one of them. Rigid limbs and digits make it difficult for them to grasp, hold, and manipulate a range of everyday objects without dropping or crushing them.

Recently, CSAIL researchers have discovered that the solution may be to turn to a substance more commonly associated with new buildings and Silly Putty: silicone.

At a conference this month, researchers from CSAIL Director Daniela Rus’ Distributed Robotics Lab demonstrated a 3-D-printed robotic hand made out of silicone rubber that can lift and handle objects as delicate as an egg and as thin as a compact disc.

Just as impressively, its three fingers have special sensors that can estimate the size and shape of an object accurately enough to identify it from a set of multiple items.

“Robots are often limited in what they can do because of how hard it is to interact with objects of different sizes and materials,” Rus says. “Grasping is an important step in being able to do useful tasks; with this work we set out to develop both the soft hands and the supporting control and planning systems that make dynamic grasping possible.”

The paper, which was co-written by Rus and graduate student Bianca Homberg, PhD candidate Robert Katzschmann, and postdoc Mehmet Dogar, will be presented at this month’s International Conference on Intelligent Robots and Systems.

The hard science of soft robots

The gripper, which can also pick up such items as a tennis ball, a Rubik’s cube and a Beanie Baby, is part of a larger body of work out of Rus’ lab at CSAIL aimed at showing the value of so-called “soft robots” made of unconventional materials such as silicone, paper, and fiber.

Researchers say that soft robots have a number of advantages over “hard” robots, including the ability to handle irregularly-shaped objects, squeeze into tight spaces, and readily recover from collisions.

“A robot with rigid hands will have much more trouble with tasks like picking up an object,” Homberg says. “This is because it has to have a good model of the object and spend a lot of time thinking about precisely how it will perform the grasp.”

Soft robots represent an intriguing new alternative. However, one downside to their extra flexibility (or “compliance”) is that they often have difficulty accurately measuring where an object is, or even if they have successfully picked it up at all.

That’s where the CSAIL team’s “bend sensors” come in. When the gripper hones in an object, the fingers send back location data based on their curvature. Using this data, the robot can pick up an unknown object and compare it to the existing clusters of data points that represent past objects. With just three data points from a single grasp, the robot’s algorithms can distinguish between objects as similar in size as a cup and a lemonade bottle.

“As a human, if you’re blindfolded and you pick something up, you can feel it and still understand what it is,” says Katzschmann. “We want to develop a similar skill in robots — essentially, giving them ‘sight’ without them actually being able to see.”

The team is hopeful that, with further sensor advances, the system could eventually identify dozens of distinct objects, and be programmed to interact with them differently depending on their size, shape, and function.

How it works

Researchers control the gripper via a series of pistons that push pressurized air through the silicone fingers. The pistons cause little bubbles to expand in the fingers, spurring them to stretch and bend.

The hand can grip using two types of grasps: “enveloping grasps,” where the object is entirely contained within the gripper, and “pinch grasps,” where the object is held by the tips of the fingers.

Outfitted for the popular Baxter manufacturing robot, the gripper significantly outperformed Baxter’s default gripper, which was unable to pick up a CD or piece of paper and was prone to completely crushing items like a soda can.

Like Rus’ previous robotic arm, the fingers are made of silicone rubber, which was chosen because of its qualities of being both relatively stiff, but also flexible enough to expand with the pressure from the pistons. Meanwhile, the gripper’s interface and exterior finger-molds are 3-D-printed, which means the system will work on virtually any robotic platform.

In the future, Rus says the team plans to put more time into improving and adding more sensors that will allow the gripper to identify a wider variety of objects.

“If we want robots in human-centered environments, they need to be more adaptive and able to interact with objects whose shape and placement are not precisely known,” Rus says. “Our dream is to develop a robot that, like a human, can approach an unknown object, big or small, determine its approximate shape and size, and figure out how to interface with it in one seamless motion.”

This work was done in the Distributed Robotics Laboratory at MIT with support from The Boeing Company and the National Science Foundation.

RELATED

Paper: “Haptic Identification of Objects using a Modular Soft Robotic Gripper”

Bianca Homberg

Daniela Rus

Distributed Robotics Lab

[article on csail.mit.edu]