Northwestern University mechanical engineering professor Todd Murphey and his team are engineering robots that one might say could make robotic assistance as seamless as “humanly” possible. With support from the National Science Foundation (NSF), the team is using novel algorithmic tools, such as a drawing robot, to develop the algorithms, or rules of behavior, that would greatly enhance a robot’s ability to adapt to human unpredictability.

Murphey points out that in order for robots to help people, they have to have at least a basic understanding of the types of tasks people can do. Some tasks, like lifting and placing an object, are close to the types of tasks that robots already do. Other tasks, like drawing, are harder for robots, partly because there are so many ways to get the same image. As Murphey explains: “And so, drawing is a type of task that’s maybe not the same as that sort of precision manufacturing task that we’ve seen robots do historically.”

Murphey sees many possibilities for robots that work alongside humans in everyday tasks, but one application his lab is focused on currently is physical therapy. The goal is to develop robots that can help patients not only move through the paces of their physical therapy without hurting themselves but allow them to complete movement tasks by creating mechanical environments to make this feasible. “Algorithms developed here will eventually run on physical therapy robots designed to help people with tasks like balance and feeding themselves,” says Murphey.

His collaborator, Julius P. Dewald, runs the Northwestern School of Medicine’s Physical Therapy and Human Movement Sciences Department. Dewald has pioneered the use of robotics in stroke rehabilitation and sees great promise in Murphey’s new approaches.

The National Science Foundation (NSF) announced a $6.1 million, five-year award to accelerate fundamental research on wireless communication and networking technologies through the foundation’s Platforms for Advanced Wireless Research (PAWR) program.

Through the PAWR Project Office (PPO), award recipients US Ignite, Inc. and Northeastern University will collaborate with NSF and industry partners to establish and oversee multiple city-scale testing platforms across the United States. The PPO will manage nearly $100 million in public and private investments over the next seven years.

“NSF is pleased to have the combined expertise from US Ignite, Inc. and Northeastern University leading the project office for our PAWR program,” said Jim Kurose, NSF assistant director for Computer and Information Science and Engineering. “The planned research platforms will provide an unprecedented opportunity to enable research in faster, smarter, more responsive, and more robust wireless communication, and move experimental research beyond the lab — with profound implications for science and society.”

Over the last decade, the use of wireless, internet-connected devices in the United States has nearly doubled. As the momentum of this exponential growth continues, the need for increased capacity to accommodate the corresponding internet traffic also grows. This surge in devices, including smartphones, connected tablets and wearable technology, places an unprecedented burden on conventional 4G LTE and public Wi-Fi networks, which may not be able to keep pace with the growing demand.

NSF established the PAWR program to foster use-inspired, fundamental research and development that will move beyond current 4G LTE and Wi-Fi capabilities and enable future advanced wireless networks. Through experimental research platforms that are at the scale of small cities and communities and designed by the U.S. academic and industry wireless research community, PAWR will explore robust new wireless devices, communication techniques, networks, systems and services that will revolutionize the nation’s wireless systems. These platforms aim to support fundamental research that will enhance broadband connectivity and sustain U.S. leadership and economic competitiveness in the telecommunications sector for many years to come.

“Leading the PAWR Project Office is a key component of US Ignite’s mission to help build the networking foundation for smart communities,” said William Wallace, executive director of US Ignite, Inc., a public-private partnership that aims to support ultra-high-speed, next-generation applications for public benefit. “This effort will help develop the advanced wireless networks needed to enable smart and connected communities to transform city services.”

Establishing the PPO with this initial award is the first step in launching a long-term, public-private partnership to support PAWR. Over the next seven years, PAWR will take shape through two multi-stage phases:

• Design and Development. The PPO will assume responsibility for soliciting and vetting proposals to identify the platforms for advanced wireless research and work closely with sub-awardee organizations to plan the design, development, deployment and initial operations of each platform.
• Deployment and Initial Operations. The PPO will establish and manage each platform and document best practices as it progresses through the lifecycle.

“We are delighted that our team of wireless networking researchers has been selected to take the lead of the PAWR Project Office in partnership with US Ignite, Inc.,” said Dr. Nadine Aubry, dean of the college of engineering and university distinguished professor at Northeastern University. “I believe that PAWR, by bringing together academia, industry, government and communities, has the potential to make a transformative impact through advances spanning fundamental research and field platforms in actual cities.”

The PPO will work closely with NSF, industry partners and the wireless research community in all aspects of PAWR planning, implementation and management. Over the next seven years, NSF anticipates investing $50 million in PAWR, combined with approximately $50 million in cash and in-kind contributions from over 25 companies and industry associations. The PPO will disperse these investments to support the selected platforms.

Additional information can be found on the PPO webpage.

This announcement will also be highlighted this week during the panel discussion, “Wireless Network Innovation: Smart City Foundation,” at the South by Southwest conference in Austin, Texas.

Array of Things is designed as a “fitness tracker” for the city of Chicago, collecting new streams of data on Chicago’s environment, infrastructure, and activity. Credit: Rob Mitchum/Urban Center for Computation and Data

From autonomous vehicles to flash flood alert systems, technology transforms how people lead their daily lives and how local cities and communities function.

Last September, the Administration launched the National Smart Cities Initiative to help communities tackle local challenges and improve city and municipality services. Today, the National Science Foundation (NSF) — the lead federal agency in the effort — announced more than $60 million in Smart Cities-related grants for Fiscal Year (FY) 2016, with additional investments planned for FY 2017. This new funding adds to the nearly $40 million the agency awarded last year to support researchers working to design, adapt and manage the smart and connected communities of the future.

“The effective integration of technology and data into decision making and physical infrastructure has the ability to transform society, allowing local cities and communities to overcome physical, social, economic, and infrastructural challenges,” said Jim Kurose, assistant director of NSF’s Computer and Information Science and Engineering Directorate. “NSF-supported research in this area will help cultivate more livable, workable, sustainable and connected communities.”

The grants bring academic researchers from a range of disciplines together with community stakeholders and civic leaders. Through research that integrates digital tools and engineering solutions into the physical world, these partners will work to solve important challenges in health and wellness, energy efficiency, building automation, infrastructure and public safety.

“New understanding and innovations gained from NSF-funded research in Smart and Connected Communities will help measure and shape the pulse of communities’ physical and social infrastructures to improve the quality and efficiency of our lives in America’s cities, now and for our future,” said Grace Wang, acting assistant director of NSF for Engineering.

NSF has long supported the fundamental research that underlies smart and connected communities. This research has included advanced networking and connectivity, sensing, real-time data analytics, control, automation and decision-making. The agency has also been instrumental in transitioning these technologies to widespread use. For example, since 2012, the US Ignite initiative seeded the development of numerous new “gigabit application prototypes,” capable of processing large amounts of information in real-time, which has improved regional radar systems, autonomous vehicle management, and more.

Severe storms and rapidly accumulating rain recently contributed to historic flooding that caused extreme damage to the main street of Ellicott City, located not far from the University of Maryland, Baltimore County (UMBC) campus. Credit: Photo collections from the Office of the Maryland Governor, managed by the Maryland State Archives. [CC BY 2.0]

Today’s new NSF investments in support of the Smart Cities initiative include:

• $24.5 million planned investment in Smart and Connected Communities for FY 2017, including an award solicitation exclusively focused on fundamental research and research capacity-building that will transform our nation’s cities and communities for the future. This program will develop a robust, multidisciplinary, diverse research workforce capable of addressing the challenges that our cities and communities face, with a focus on collaboration and partnership between researchers and community stakeholders to improve the quality of life for all.
• $10 million in new awards to develop and scale next-generation internet applications and technologies through the US Ignite program. These awards provide citizens with access to gigabit-enabled networks and services, bringing data and analytics to decision-makers in real time. The focus of the research ranges from the development of advanced networking technology to the creation of application and service prototypes that leverage advanced technology.
• $8.5 million in new awards for high-risk, high-reward research through Smart and Connected Communities. These Early-Concept Grants for Exploratory Research (EAGER) projects, combined with supplemental funding to existing NSF-funded grants, represent NSF’s initial efforts to grow a Smart and Connected Communities research community and pilot new research approaches. This includes $300,000 to a multidisciplinary team of researchers developing a new network architecture to expand internet access and engagement, and support community-building on Native American reservations.
• $7 million in new Partnerships for Innovation: Building Innovation Capacity projects that translate breakthrough discoveries into emerging technologies through academic-industry collaborations. Such technologies can transform smart service systems, such as smart hazard notification systems, smart buildings and sensor networks to improve transportation efficiency.
• $4 million in new Cyber-Physical Systems awards focused on Smart and Connected Communities. These awards support research that integrates computing, networking and physical systems — for example, in self-driving cars and smart buildings. Collectively, these awards are helping to establish the foundation for Smart Cities and the Internet of Things.
• $2 million in new “Spokes” that extend the Big Data Regional Innovation Hubs. These awards will use data science to improve the smart electric grid, keep bridges safer, grow better crops through the use of drone technology, and allow students to conduct citizen science on air pollution.
• $1.5 million in new Smart and Connected Health research awards with a focus on Smart and Connected Communities. These awards will support the development of next-generation health-care solutions that leverage sensor technology, information and machine learning technology, decision support systems, modeling of behavioral and cognitive processes, and more. This includes $300,000 to a multidisciplinary team of researchers to integrate community and clinical data to model the evolution of influenza in communities, supplementing prior investment by the Defense Department’s Defense Advanced Research Projects Agency, or DARPA.
• $1.4 million in new Big Data research focused on Smart and Connected Communities. These awards will drive innovation in data analytics and data-driven decision making to fuel the growth and development of the communities of the future.
• $1 million for researchers to participate in the 2016 NIST Global City Teams Challenge, supporting high-risk, high-reward research on the effective integration of networked computing systems and physical systems to meet community challenges. This includes $250,000 for real-time monitoring and detection of flash floods in several Maryland cities.
• $1 million in new research and capacity-building awards that support lifelong learning, which is critical to cities and communities of the future. The awards will leverage networks and technology to foster lifelong learning in multiple formal and informal settings, enabling students to solve problems that arise within their communities.

Through the programs listed above and the projects they support, computer and information scientists, engineers, and social, behavioral, and economic scientists will collaborate with industries, non-profits, local governments and anchor institutions, such as schools, libraries and hospitals. The effort will continue to nurture and grow a research community focused on smart and connected communities, and will help train and prepare the next generation of scientists and engineers to advance solutions that improve tomorrow’s cities and communities.

Image shows the “MuddyBot” robot that uses the locomotion principles of the mudskipper to move through a trackway filled with granular materials. The robot has two limbs and a powerful tail, with motion provided by electric motors.
Credit: Rob Felt, Georgia Tech

Could a tail have allowed ancient vertebrates to make the transition from water to land?

Reporting in Science today, researchers from Georgia Institute of Technology, Carnegie Mellon University, Clemson University and National Institute for Mathematical and Biological Synthesis described results of a groundbreaking study to answer this question using amphibious fish, a custom-built robot and mathematical models of movement.

The study tested the hypothesis that coordinated tail movement played an important role in evolution of terrestrial vertebrates. The researchers found that, for the first critical step out of an aqueous environment and onto a riverbank or mudflat, stabilizing the body with a tail provided substantial benefits.

(video credit: Georgia Institute of Technology, Clemson University, Carnegie Mellon University)

The effort, supported by the National Science Foundation (NSF), the Army Research Office and the Army Research Laboratory, brought together experts in physics, biology, robotics and mathematics who explored the question through a variety of experimental methods.

Daniel Goldman, an associate professor in the Georgia Tech School of Physics and the principal investigator on the NSF award that supported the work, says he has long been obsessed with the question of how the first creatures moved onto land.

“I was interested in evolutionary biology from a natural history perspective: imagining the grand scope of life on Earth and trying to understand how animals lived hundreds of millions of years ago and in this case, trying to understand how major transitions in behavior happened.” Goldman says. “For me this has been a nice integration of my interests applied to a question which I just think is fascinating: what did the earliest animals do when they were trying to crawl around on land?”

To answer the question, he and his collaborators took a three-pronged approach.

First, they explored the behavior and body mechanics of the African mudskipper — an amphibious fish that lives in tidal areas near shore and uses its front fins and tail to move on land with a “crutching” motion. The mudskipper is one of a handful of living organisms considered to be potentially close in body structure to the first terrestrial vertebrates.

Researchers study the motion of mudskippers to understand how early terrestrial animals might have moved about on mud and sand. This animal was photographed at the Georgia Aquarium in Atlanta.
Credit: Rob Felt, Georgia Tech

In co-author Richard Blob’s lab at Clemson University, researchers recorded the movements of mudskippers in an environment akin to riverbanks where early land-dwellers may have emerged to see how they behaved.

“We found when mudskippers are using their tail on a sufficiently inclined sandy slope they do pretty well and when they don’t, they don’t do well,” Goldman concluded.

Georgia Tech Associate Professor Dan Goldman with the MuddyBot robot in the trackway where its motion was studied.
Credit: John Toon, Georgia Tech

Next, the researchers developed a simplified, mudskipper-like version of a robot, which they call “MuddyBot,” on which they could systematically vary the angle and movements of the robot’s flipper-like limbs and tail. They used MuddyBot to find out which coordinated motions of limb and tail were most effective on granular surfaces of different inclines. They call this approach “robophysics” — a novel way to understanding the behavior of long-lost species.

Finally, they used a mathematical and engineering method called “geometric mechanics” (developed in the 1980s by physicist and Nobel Prize winner Frank Wilczek and his student Alfred Shapere) to analyze all the possible ways the MuddyBot can move in space and on different surfaces. This helps determine what types of movement may or may not have allowed the creature to pull itself up onto a sandy or muddy slope. This process also involved developing a better understanding of the physics of sand and the way that a granular medium can compress or slip when something tries to move on it.

The variety of approaches provided the teams with quantitative evidence to support the idea that the first locomotors may have had tails and may have used a crutching motion to move out of the water.

“Insight from these experiments led us to hypothesize that propulsive use of the tail, an appendage that has received relatively little attention in previous studies of the invasion of land, may have been the critical adaptation that allowed these early walkers to gain ground on challenging substrates,” says Benjamin McInroe, a co-author on the paper and then an Georgia Tech undergraduate (now a Ph.D. student at the University of California, Berkeley) who analyzed the mudskipper data provided by the Clemson team.

“Professor Goldman and his collaborators are combining physics and engineering prototyping approaches to understand the physical principles that allow animals to move in different environments,” says Krastan Blagoev, program director in the NSF Division of Physics. “This novel approach to living organisms promises to bring to biological sciences higher predictive power and at the same time uncover engineering principles that we have never imagined before.”

The team’s well-documented and analytic-rich findings only chart the critical first step out of the aqueous environment and not the full progression up a riverbank and onto level ground. After that first step, things get even more complicated because sand flows in a way that scientists don’t yet have the physics to describe.

“Even this ridiculously seemingly simple little crutching motion with coordinated tail use confronts our ignorance in three or four different disciplines: biology, paleontology, robotics, and mathematics,” says Goldman. “That’s a summary of how far away we are from really understanding it.”

Instead of being discouraged by this fact, the realization motivates Goldman.

“That’s the great joy in all this,” he says. “Just looking at these simple problems, there’s a huge amount of insight to be had from applying new tools.”

The research provides a rigorous approach to studying evolutionary questions in the future by combining animal, robotic and mathematical approaches. It even may improve designs for robots that can move on sand or loose dirt and help in search and rescue missions – a focus of Howie Choset’s work at Carnegie Mellon University.

“Our computer modeling tools allow us to visualize, and therefore better understand, how the mudskipper incorporates its tail and flipper motions to locomote,” says Howie Choset, a professor in the Robotics Institute at Carnegie Mellon University. “This work also will advance robotics in those cases where a robot needs to surmount challenging terrains with various inclinations.”

It is impossible to say for sure yet if the earliest locomotors used a crutching behavior like the mudskipper, Goldman says. For that reason, the team is now making robots to test some alternative possibilities, including robots with bodies like salamanders that move with a diagonal gait.

“It really is exciting to imagine that some of these models that we’re cooking up in the lab in these relatively simple robots could have anything to say about something that happened 360 million years ago,” he says, “and to be able to demonstrate that these questions, which seem so simple on first blush, actually require deep insights into the fundamentals of all sorts of disciplines including physics, biology and robotics.”

A small 3-D printed ionic polymer-metal composite soft robotic hand. National Science Foundation researchers are working to transform this material into artificial muscles. Credit: Kam K. Leang. University of Utah

Forget steel and aluminum. The robots of tomorrow may be able to squish, stretch and squeeze.

Novel robotic devices, part of the emerging field of soft robotics, offer many advances over conventional robots. Soft robots can more easily maneuver in tough spaces. They can better interact with humans, making them excellent assistants for elderly people. And one day they may lead to high-tech artificial muscles: a life-changing innovation for millions of disabled people around the globe.

James Carrico, a Ph.D. graduate student at the University of Utah, displaying a small 3-D printed ionic polymer-metal composite soft robotic hand. Credit: Kam K. Leang

Creating artificial muscles requires not only developing a powerful, flexible material, but figuring out how to precisely control and cleverly manufacture it. That’s the mission of Kwang Kim of the University of Nevada, Las Vegas and his National Science Foundation (NSF)-funded team.

Kim is lead investigator on a NSF award pairing a diverse group of researchers — at four U.S. universities plus research institutions in Japan and South Korea — to transform a novel polymer-based material into artificial muscles. The research is supported through NSF’s Partnerships for International Research and Education (PIRE) program, which supports innovative, global research collaborations across all fields of science and engineering.

PIRE leverages U.S. funding and expertise to tackle global challenges. Kim’s U.S. team is working with researchers from the Korea Advanced Institute of Science and Technology (KAIST) and Japan’s National Institute of Advanced Industrial Science and Technology, both known for strong expertise in robotics. (KAIST, for example, won the recent Robotics Challenge, hosted by the Defense Advanced Research Projects Agency.)

One of the big challenges in soft robotics is finding the right material. “It has to be soft, but it also has to produce enough power to do lots of different things,” Kim said. His team is using a type of synthetic material called Ionic Polymer-Metal Composites, which is a kind of electroactive polymer — meaning running electricity through the material makes it change shape.

The PIRE award will work to transform electroactive polymers, a type of material that changes shape when electricity runs through it, into artificial muscles. Credit: Kwang J. Kim, University of Nevada, Las Vegas; Kam K. Leang, University of Utah (both formerly of University of Nevada, Reno)

“In robotics you’ve got to be able to move and you’ve got to be able to sense,” said Kam Leang, an associate professor at the University of Utah Robotics Center and a co-investigator on this project. Traditional robots use electric motors to do the former. “In this PIRE, we are using the electroactive polymer itself.”

Electroactive polymers can also be used to sense motion, making them a great candidate for soft robotics. Leang and his colleagues have also devised a way to 3-D print the material. His component of the PIRE research is focused on how to scale up the manufacturing, as well as devising ways to better control the motion of the polymer. Others are working to better understand — and improve — the polymer material to make it more responsive, strong and affordable.

The project, which received NSF funding last fall, is still in its early stages. Kim, who has been working in electroactive polymers for nearly two decades, said soft robotics itself is still a relatively new field.

“I’m learning every day.”

Investigators: Paul Oh, Kam Leang, Kwang Kim, Chulsung Bae, Maurizio Porfiri

Related Institutions/Organizations: New York University, The University of Utah, University of Nevada Las Vegas, Rensselaer Polytechnic Institute

If you liked this article, you may also be interested in:

• National Robotics Initiative celebrates five-year anniversary
• NSF and partners award $37M to advance nation’s co-robots
• Speedy robots may someday run to the rescue

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

We now live in a world where increasingly co-robots interact with people to improve our productivity and safety. Additional research investments that enable robots to interact with other robots and devices will help ensure U.S. economic growth increased global competitiveness and improved quality of life.
Credit: NSF

By: Nisha Cooch, NSF

This month marks the fifth anniversary of the National Robotics Initiative (NRI). The National Science Foundation (NSF) has been an active leader in the program since it was launched by President Obama in 2011 and is proud of the significant progress made so far in robotics.

Advances in fundamental technologies that enable robots to collaborate with humans, as well as with other robots and devices has led to a surge in the interest and use of robotics in our everyday lives. Moving forward, NRI will continue to support robotics research that will help spur innovation in robotics, enhance our economy and national competitiveness, and improve our quality of life.

Critical questions in robotics remain, including the following:

1. How can robots understand the world around them?

Improving sensor capabilities will support robots’ abilities to detect situational information and to integrate that information into models of robots’ surrounding environments.

2. How can robots make high-level decisions in complex situations?

Research in cognition will improve our understanding of how the human brain processes information and will make it easier to program robots so they can interpret different types of information.

3. How can robots and humans interact through language?

Advances in natural language processing and speech will help enable robots to produce and interpret language, as well as to understand and express acoustic, facial and gestural cues that accompany speech.

4. How can robots be impactful in social contexts?

Investigating fundamental aspects of emotion and expression is critical for building human-like robots that can perceive and express subtle emotional signals.

5. How can individual robots execute a range of movements?

Motion research can help provide robots with the mobility and agility required to traverse different terrains, as well as the dexterity to manipulate small objects.

6. How do we ensure robots function optimally?

Studies on quantitative measurement tools will allow for the continuous evaluation of robots, and resiliency research will empower robots to perform despite dysfunctions or failures.

As robots capable of collaboration emerge, new questions will arise. For instance, in a world of collaborative robots, how do we best incorporate collaborative robots into our existing infrastructures? Once robots are integrated, how do we ensure the safety of people, devices and information? NSF is committed to addressing these important issues so the potential of the robotics age translates to significant benefits for society.

NSF’s long-term investments in fundamental science and engineering research such as these projects have led to novel machines that safely partner with people in nearly every environment. Visit NSF.gov/robotics for more stories about the emerging robot generation.

Click to download

Credit: Aaron Bestick, UC Berkeley

The Congressional Robotics Caucus will host an expo on Thursday, June 9 in Washington, D.C. to promote the progress and promise of collaborative robotics (co-robot) research.

The event will mark the five-year anniversary of the National Robotics Initiative, a multi-agency effort among the National Science Foundation (NSF), NASA, the National Institutes for Health (NIH), the U.S. Department of Agriculture (USDA), the Department of Defense (DOD) and the Department of Energy (DOE) to accelerate the development and use of robots that work beside or cooperatively with people and solve problems in areas of national priority.

The National Robotics Initiative advanced research in soft robotics and other novel research areas. In this example, a soft inflatable robot arm from Carnegie Mellon is actuated using cables, motors and pneumatics.
Credit: Sanan/CMU

The event, hosted by Congressional Robotics Caucus Co-Chairs Congressman Rob Woodall and Congressman Mike Doyle, will feature the latest in robotic technologies, as well as a discussion among leading scientists, educators and thought-leaders, and will promote improved public understanding of development of co-robots in the modern world.

All event information can be found on their website.

The panel will feature:

• Moderator Martial Hebert, The Robotics Institute, Carnegie Mellon University
• Lynne Parker, co-chair, NITRD Interagency Working Group on Robotics and Intelligent Systems
• Robert Atkinson, president, Information Technology and Innovation Foundation
• Henrik Christensen, Institute for Robotics and Intelligent Machines, Georgia Tech University; fellow, IEEE Robotics & Automation Society

The exhibition will feature researchers leading NRI-funded projects and agencies, including:

• Aaron Becker, University of Houston
• Howie Choset, Carnegie Mellon University
• Jnaneshwar Das, University of Pennsylvania
• Aaron Ames, Georgia Tech Jessy Grizzle, University of Michigan, and Koushil Sreenath, Carnegie Mellon University
• Satyandra K. Gupta, University of Southern California
• Greg Hager, Johns Hopkins University
• Robert MacCurdy, MIT
• Marcie O’Malley, Rice University
• Josip Markus, Ekso Bionics
• Andrew Williams, Marquette University
• Tyr Wiesner-Hanks, Cornell University
• Cang Ye, University of Arkansas at Little Rock
• Department of Energy, Environmental Management
• National Institute of Standards and Technology, Robotic Systems for Smart Manufacturing Program

Rescuers in Turkey are working tirelessly to free this young boy, one of the many victims of Sunday’s devastating 7.2 magnitude earthquake. He’s not yet been freed, but some have been lucky. Rescue workers manage to carry this toddler to safety on Monday morning. Some of the survivors aren’t badly injured. Others, when rescued, are being taken straight to hospital.

More than a thousand people are feared dead following the quake, and many more are thought to be still trapped under collapsed buildings. For them, time will soon start running out.

This was the call from an ITN/ODN television news reporter following a massive 2011 earthquake in Van Province, Turkey.

University of Michigan robotics engineer Jessy Grizzle with MABEL.

For Jessy Grizzle, a robotics engineer at the University of Michigan in Ann Arbor (UM), events like this remind him of the importance of his work — developing feedback and control algorithms that one day will give bipedal robots the balance needed to conduct search and rescue missions in dangerous environments.

“Tragedies like these remind the robotics community that robots are needed,” he says. “They certainly keep me and my team focused on achieving the best walking robot control algorithms we can.”

Grizzle, along with a group of robotics engineers and students at UM, is not only working to develop algorithms — self-contained, step-by-step operations — to be performed by walking robots, he’s working to revolutionize them.

The walking problem

Existing bipedal robots have control software that relies on sensors to precisely read the various features of a walking surface. They also have large, flat feet to help maintain balance when walking. These designs, say some engineers, drastically limit the kind of locomotion that can be achieved.

For example, a robot’s flat feet fail to account for variations in the ground surface the way standard human feet do, which perform thousands of minute adjustments and rotations of the heel, foot and toe with respect to the ground. Plus, flat-footed designs do not easily allow bipedal machines to adjust their balance, pick up their legs and avoid obstacles as required in human work environments or search and rescue situations.

Grizzle says prevailing techniques can only handle the flat-footed phase of normal walking and hence yield very unnatural gaits.

“In practice, large, heavy feet are used to avoid normal foot rotation, which can be very useful for accommodating uncertain environments,” he says.

In addition, bipedal robots require scientists and engineers to reprogram them to accomplish different walking tasks. A robot programmed to walk over a flat floor would be tuned differently than another robot capable of traversing stairs or stepping over fallen debris as seen in a video from the 2015 DARPA Robotics Challenge.

The switch between tasks is difficult and time-consuming. Grizzle wants bipedal robots to be able to make the transitions with speed and to walk faster, much faster. He also wants them to step over and around obstacles, and on uneven walking surfaces, without tipping over.

https://youtu.be/2cwMrilzlYY

To accomplish this, he has two risky, but potentially transformative goals in mind. The first is radically updating the feedback control algorithms that allow robots to stand, walk and step over obstacles. The second involves developing mathematical designs for robot balance that can quickly be transferred from one walking robot to another.

The running solution

Working with a National Science Foundation program called INSPIRE that funds risky, potentially transformative, cross-disciplinary research, Grizzle and his team aim to make their goals a reality. NSF’s Directorates for Engineering, Computer and Information Science and Engineering, and Mathematical and Physical Sciences support the research.

Our overriding objective is to develop control algorithms that give bipedal robots a keen sense of balance, says Grizzle, making them capable of walking rapidly over rough terrain, without overreliance on precise measurements of the ground profile.

To understand where he’s going, it’s necessary to look at where he’s been, starting with MABEL, a novel, NSF-funded, bipedal robot in Grizzle’s Control Systems Laboratory at UM.

MABEL breaks the rules. Rather than walking on flat feet, MABEL walks on unactuated point feet. In other words, MABEL has no feet at all in the traditional sense.

Actuators are the muscles of a robot; they convert stored energy into movement. MABEL’s feet have no actuators. Instead, it walks on what look like stilts. Its feet have no supplemental pivot points that can be rolled to help keep balance such as ankles, heels or toes that rely on anatomical muscles.

As a substitute, MABEL depends on next-generation control algorithms that give it not only an ability to walk, but also to run. Tests show MABEL, like some athletes, can run 6.8 miles per hour with both feet in the air for 40 percent of each stride. In addition, without falling over, it can seamlessly step over, off and on obstacles.

MABEL uses what Grizzle calls a “nonlinear, compliant hybrid zero dynamics controller with active force control.” In simpler terms, his team developed software that gives MABEL the ability to quickly switch between several natural gaits for walking and running.

When MABEL moves, sensor data from the robot’s joints are fed back into the newly designed control algorithm that computes in real-time what MABEL’s next step should be. It works step after step, giving it human-like walking and running capabilities.

“We have shown that our mathematical models exhibit behaviors consistent with normal walking and running,” says Grizzle. “Assumptions such as flat-footed walking are no longer required.”

The future of walking and running robots

The next step may be the team’s biggest challenge: developing mathematical designs for robot balance that easily can be transferred from one robot to another to enable similar walking performance across multiple platforms. Success would remove much of the trial, error and time needed to design bipedal-robot control algorithms.

The design difficultly revolves around differences in the center of mass found in robots with varying “anatomies.” For instance, one robot may have a low center of mass, while another robot has a high center of mass, making a universal controller that maximizes balance for multiple walking robots hard to achieve.

Today’s high-end, humanoid robots have mathematical models that depend on 50 or more anatomy-related variables to produce balance. But their control algorithms use only a handful of those variables to produce a simplified version of the machine’s center of mass. This accounts for much of the awkward movement and primitive elements of balance seen in most current bipedal robots.

Grizzle takes a different tact. His control algorithms use the full complexity of a robot’s mathematical model. Thus far, the team has used this method to design balance algorithms that perform relatively well when the robot walks in a straight line, but the algorithms still struggle when the robot moves laterally.

Last summer, however, Aaron Ames, an associate professor of electrical and computer engineering at Georgia Institute of Technology, realized Grizzle’s goal in part by integrating Grizzle’s method in the core algorithm of a walking, humanoid robot called DURUS.

Ames was able to extend Grizzle’s basic method to allow more phases in DURUS’ walking gait. In addition, he developed a program that allowed him to optimize his algorithm in minutes instead of hours, bringing into view Grizzle’s goal of reducing the time needed to generate robot control algorithms.

The result: DURUS was able to walk 10 times more efficiently than previous walking robots.

“I think this marks an important milestone in transferring dynamic walking algorithms between multiple robotic platforms,” says Ames, “and realizing these concepts on advanced humanoid robots.”

As the technology continues to grow, robotics engineers believe the future for walking, assistive and rescue robots is bright.

Leaders from the National Science Foundation and the Department of Energy sign a memorandum of understanding, signifying DOE’s inclusion in the National Robotics Initiative. Credit: NSF/Sandy Schaeffer

Yesterday, the National Science Foundation (NSF) — in partnership with the Department of Defense (DOD), the Defense Advanced Research Projects Agency (DARPA), the National Aeronautics and Space Administration (NASA), the National Institutes of Health (NIH) and the United States Department of Agriculture (USDA) — announced $37 million in new awards to spur the development and use of co-robots, robots that work cooperatively with people.

From unmanned vehicles that can inspect and fix ailing infrastructure to co-robots that can collaborate with workers on manufacturing tasks, scientists and engineers are developing the next generation of robots that can handle critical tasks in close proximity to humans, providing for unprecedented safety and resilience.

The awards mark the fourth round of funding through the National Robotics Initiative (NRI), a multi-agency program launched in June 2011 as part of the Advanced Manufacturing Partnership (AMP), with NSF as the lead federal agency.

This year, the initiative funded 66 new research proposals — of one to four years in duration — to 49 distinct institutions in 27 states. Together, they advance the fundamental understanding of robotic sensing, motion, computer vision, machine learning and human-computer interaction. This year’s NSF-funded NRI awards can be viewed here.

The NRI awards address the entire development cycle of robots, from fundamental research to prototyping and testing the deployment in critical environments to help make safe, helpful and affordable co-robots a reality.

The awards include, for example, research to prototype improved brain-controlled prosthetic devices, to design teams of robots that can undertake search-and-rescue operations, and to build robots that can assist with a variety of healthcare tasks.

A close-up view of the 3-D model created by a bridge inspection robot from the ARIA project flying under a bridge to create a high-resolution 3-D model and image dataset that can aid inspection. Credit: Varun Kasireddy, Carnegie Mellon University

“Our engineers and scientists are creating a world where robotic systems serve as trusted co-workers, co-inhabitants, co-explorers and co-defenders,” said Pramod Khargonekar, NSF’s assistant director for engineering. “The National Robotics Initiative serves the national good by encouraging collaboration among academic, industry, nonprofit and other organizations — and by speeding the creation of the fundamental science and engineering knowledge base used by researchers, applications developers and industry.”

NSF’s investments in robotics explore both the technical and engineering challenges involved in developing co-robots, as well as the long-term social, behavioral and economic implications of co-robots across all areas of human activities.

As part of the NRI, NSF also supports developing new methods to establish and infuse robotics in educational curricula. A full list of the NRI investments made by NSF is available through NSF’s award search.

Looking ahead, NSF recently issued an updated call for NRI proposals, which the Department of Energy joined as a new partner.

“Basic research to develop the underlying understanding and technologies for robots and robotic systems is resulting in profound advances in the field with the potential to augment human abilities, perform dangerous tasks unsuitable for people and improve our quality of life,” said Jim Kurose, NSF’s assistant director for computer and information science and engineering . “Working together with many federal agencies has helped to develop a broad portfolio of robotics research with a greater impact than any one agency could do by itself. We are excited to be growing this effort through the addition of the Department of Energy.”

Since its inception, NSF and partnering agencies have invested more than $150 million in fundamental research, prototyping and testing, and education efforts through the NRI.

Kibo allows young tinkerers to build a robot using supplied modular components, personalize it with art designs and then program it to do their bidding using colored program blocks.
Credit: DevTech research grouo, Tufts University

Playgrounds are popular spaces for young children to play and learn. They promote exploration of the physical environment and motor and social skill development, allowing young children to be autonomous while developing core competencies.

Playpens, by contrast, corral children into safe, confined spaces. Although they are mostly riskfree, there is little opportunity for exploration and imaginative play.

From a developmental perspective, the playground promotes a sense of mastery, creativity, self-confidence, social awareness and open exploration, while the playpen hinders development of these traits.

“We’re trying to develop technologies to get us as close as we can to the metaphor of the playground,” said Marina Umaschi Bers, a professor of computer science and Child Development at Tufts University, director of DevTech research group and author of Designing Digital Experiences for Positive Youth Development: From Playpen to Playground.

Many are familiar with Bers through her work on ScratchJr – a programming language where even students who are too young to read and write can put together actions in a sequence to create interactive stories, games and animations.

Last June, with NSF support, Bers and her colleagues released ScratchJr as a free app for children ages 5 to 7. (A Kickstarter campaign in May raised $75,000 to adapt the app for Android and iPad.) As of February 2015, ScratchJr. had more than 500,000 downloads worldwide.

KIBO is a robot kit specifically designed for young children aged 4-7 years old. Children build their own robot with KIBO, program it to do what they want and decorate it. KIBO gives children the chance to make their ideas physical and tangible, and KIBO does all this without requiring screen time from PCs, tablets or smartphones.
Credit: KinderLab Robotics

Speaking at NSF last spring, Bers described her latest project, the KIWI robotic kit (subsequently renamed KIBO), which teaches programming through robotics, without screens, tablets or keyboards.

Using KIBO, students scan wooden blocks to give robots simple commands, in the process learning sequencing, one of the most important skills for early age groups. By combining a series of commands, kids make the robot move, dance, sing, sense the environment or light up.

With an NSF Small Business Innovation Research grant, Bers co-founded KinderLab Robotics and, using insights from her 15 years of research, transformed the KIWI prototype into a widely-available cyberlearning toy that could impact a large number of kids.

KIBO shipped its first several hundred units in 2014, but, according to Bers, building and selling KIBO was only a small part of the challenge.

“We don’t want to give away the technology and wash our hands of it,” Bers said. “We’re training the whole individual.”

For that reason, KIBO comes with a curriculum, lesson plans, badges, design journals and even teacher and parent training.

In March 2015, Bers and Amanda Sullivan, also of Tufts’ DevTech Group, published an article in the International Journal of Technology and Design Education describing learning outcomes from the eight-week robotics curriculum in pre-kindergarten through second grade. The results showed that beginning in pre-kindergarten, children were able to master basic robotics and programming skills, while older children were able to master increasingly complex concepts using the robotics kit in the same amount of time.

Both the KIBO robotic kit and the ScratchJr programming language have left the academic ivory tower and are now available to the wider public.

“When we teach children how to read and write, we don’t expect everyone to become a journalist or a novelist,” Bers said, speaking about KIBO in The New York Times. “But we believe they’ll be able to think in new ways because it opens the doors to thinking. We believe the same thing for the skills of programming and engineering.”

Alelo’s virtual role-play technology creates safe learning environments for students to practice real-life conversations. Customized feedback can be provided during and after interaction to analyze performance.

Cyberlearning project teaches language and cross-cultural communication with avatars and robots.

Communication skills are critical in a global economy.

Since communication often occurs across cultures, people must know more than just the vocabulary and grammar of a foreign language. They must also have cultural awareness–a recognition of the “dos and don’ts” that each culture maintains–to communicate smoothly, effortlessly and with confidence.

Virtual role-play – where learners engage in simulated encounters with artificially intelligent agents that behave and respond in a culturally accurate manner – has been shown to be effective at teaching cross-cultural communication.

“You learn by playing a role in a simulation of some real-life situation,” said W. Lewis Johnson, a former professor at the University of Southern California and co-founder of Alelo Inc., a company that makes cyberlearning tools. “You practice communication with some artificial intelligent interactive characters that will listen and respond to you, depending on what you say and do. It helps develop fluency, but it also helps to develop confidence.”

With an award from the National Science Foundation (NSF), Johnson and his team at Alelo (a Hawaiian word that means “language” or “tongue”) developed a range of products to teach cross-cultural communication using virtual worlds. Alelo’s tools have already achieved success in military training and English-as-a-second-language programs and are now being applied to a wide range of learning applications.

“We found in our military training that when soldiers use this approach, once they get into a foreign country–let’s say they’re sitting down with local leaders–they feel as if it’s a familiar situation to them, even though they’ve never experienced it in real life before,” Johnson said. “That degree of comfort is extremely powerful.”

As an example, Johnson showed a role-playing scenario that orients officers to the etiquette expected at a Chinese banquet. (The goal: Don’t get drunk!)

“How do you show proper respect for your host in participating in these toasts but not fall down drunk in the process?” Johnson asked. “That’s an effective communication problem. Role-playing teaches some of the communication skills that you could use if you find yourself in this situation.”

Importantly, Alelo’s virtual role-playing tool gives one the ability to practice conversation and etiquette until you get them right.

To develop a version to teach English as a second language (ESL) to those in the United States, the researchers interviewed immigrants to determine what cultural issues they found most problematic. The instruction they created–available in multiple languages–explains how to handle different situations that one might face as a new immigrant in the United States and provides tips on culturally appropriate behavior.

Students can practice on a range of devices, allowing them to use the program to prepare for classroom activities, as follow-up to classroom activities or as needed in the field. The ESL program even offers a virtual coach who provides help and feedback.

In 2012, Alelo partnered with the Commonwealth of Virginia to develop and test virtual role-play learning products for conversational practice in Chinese.

Tammy McGraw, the former director of digital innovations and outreach at the Virginia Department of Education, had met Johnson at a Wired conference years earlier and, although Alelo was demonstrating cyberlearning technologies for soldiers, McGraw was struck by how the virtual characters were able to impart both linguistic and cultural lessons.

Moreover, the software could address the shortage of qualified teachers able to teach Chinese, Arabic and other less popular languages, while allowing the state to increase the number of students who were able to take classes that were over-enrolled.

The RALL-E project combines Robokind’s advanced social robotics and Alelo’s virtual role-play technology. Using automated-speech recognition and customized acoustic models, RALL-E can understand and respond to Mandarin Chinese speakers of all levels.

The first Web-based course was tested last year via Virtual Virginia, a statewide virtual school program and received overwhelmingly positive ratings from the students.

“The virtual classes enabled students to be more self-directed than teachers could normally support,” McGraw said.

Recently, McGraw and Johnson have been adapting the insights learned in virtual role-playing research for a new platform: a robo-tutor.

With support from NSF, Alelo has taken RoboKind’s lifelike social robot and equipped it with the ability to converse in Chinese. The technology gives learners abundant opportunities to practice their conversational skills and lowers the entry barrier for difficult languages such as Chinese.

“It’s non-threatening,” McGraw said. “The robot is not going to be offended if you say the wrong thing. He’s not going to laugh at you if you pronounce Chinese incorrectly. But he also has the ability to engage learners through active interaction.”

Though still in the prototype phase, they imagine RoboKind could be used in the classroom, with kids coming up to engage with it at will. It could also be used outside of formal class as a personal tutor. Teachers at Thomas Jefferson High School of Science and Technology in Alexandria, Va., signed on to try out the computer-based pilot.

By combining artificial intelligence and robotics with insights from virtual role-playing research, Alelo is pioneering new avenues of personalized education.

“The question we’re trying to answer is: what are the aspects of the technology that are most important for mastering communicative skills?” McGraw said.

“We have two different realizations of virtual role-play technology. They have different strengths, different weaknesses. What are they going to be good for? Which is good for what? Doing this research, we hope to get some answers to those questions.”

Related Websites
Alelo: http://www.alelo.com/

A young doctor leans over a patient who has been in a serious car accident and invariably must be experiencing pain. The doctor’s trauma team examines the patient’s pelvis and rolls her onto her side to check her spine. They scan the patient’s abdomen with a rapid ultrasound machine, finding fluid. They insert a tube in her nose. Throughout the procedure, the patient’s face remains rigid, showing no signs of pain.

The patient’s facial demeanor isn’t a result of stoicism – it’s a robot, not a person. The trauma team is training on a “human patient simulator,” (HPS) a training tool which enables clinicians to practice their skills before treating real patients. HPS systems have evolved over the past several decades from mannequins into machines that can breathe, bleed and expel fluids. Some models have pupils that contract when hit by light. Others have entire physiologies that can change. They come in life-sized forms that resemble both children and adults.

But they could be better, said Laurel D. Riek, a computer science and engineering professor at the University of Notre Dame. As remarkable as modern patient simulators are, they have two major limitations.

“Their faces don’t actually move, and they are unable to sense or respond to the environment,” she said.

Riek, a roboticist, is designing the next generation of HPS systems. Her NSF-supported research explores new means for the robots to exhibit realistic, clinically relevant facial expressions and respond automatically to clinicians in real time.

“This work will enable hundreds of thousands of doctors, nurses, EMTs, firefighters and combat medics to practice their treatment and diagnostic skills extensively and safely on robots before treating real patients,” she said.

One novel aspect of Riek’s research is the development of new algorithms that use data from real patients to generate simulated facial characteristics. For example, Riek and her students have recently completed a pain simulation project and are the first research group to synthesize pain using patient data. This work won them best overall paper and best student paper at the International Meeting on Simulation in Healthcare, the top medical simulation conference.

Riek’s team is now working on an interactive stroke simulator that can automatically sense and respond to learners as they work through a case. Stroke is the fifth leading cause of death in the United States, yet many of these deaths could be prevented through faster diagnosis and treatment.

“With current technology, clinicians are sometimes not learning the right skills. They are not able to read diagnostic clues from the face,” she said.

Yet learning to read those clues could yield lifesaving results. Preventable medical errors in hospitals are the third-leading cause of death in the United States.

“What’s really striking about this is that these deaths are completely preventable,” Riek said.

One factor contributing to those accidents is clinicians missing clues and going down incorrect diagnostic paths, using incorrect treatments or wasting time. Reading facial expressions, Riek said, can help doctors improve those diagnoses. It is important that their training reflects this.

In addition to modeling and synthesizing patient facial expressions, Riek and her team are building a new, fully-expressive robot head. By employing 3-D printing, they are working to produce a robot that is low-cost and will be one day available to both researchers and hobbyists in addition to clinicians.

The team has engineered the robot to have interchangeable skins, so that the robot’s age, race and gender can be easily changed. This will enable researchers to explore social factors or “cultural competency” in new ways.

“Clinicians can create different patient histories and backgrounds and can look at subtle differences in how healthcare workers treat different kinds of patients,” Riek said.

Riek’s work has the potential to help address the patient safety problem, enabling clinicians to take part in simulations otherwise impossible with existing technology.

Related Website:
Robotics, Health & Communication Lab http://www3.nd.edu/~rhclab/

Humans have evolved to be incredibly efficient at walking. In fact, simulations of human locomotion show that walking on level ground and at a steady speed should theoretically require no power input at all.

But anyone who works on their feet or has taken an arduous hike knows otherwise. In fact, people expend more energy during walking than any other activity in daily life, and for the elderly and those with mobility issues, that energy can be precious.

For decades, engineers have envisioned systems that could make walking easier. In fact, so many researchers have tried to build unpowered exoskeletons and failed that it was hotly debated in the field whether it was even possible to improve the efficiency of walking without adding an external energy source.

In news reported last week in Nature, researchers from Carnegie Mellon University and North Carolina State University have demonstrated an unpowered ankle exoskeleton that reduces the metabolic cost of walking by approximately 7 percent. The results are roughly the equivalent of taking off a 10-pound backpack, and are equivalent to savings from exoskeletons that use electrically-powered devices. The research was based upon work supported by the National Science Foundation.

“It’s a real exciting milestone for the field of assistive devices,” said Thomas Roberts, a professor of ecology and evolutionary biology at Brown University and an expert in the biomechanics of locomotion, who was not involved in the research. “They’ve taken an assistive device and lowered the cost of human walking. That’s kind of a big deal because walking is already really cheap, and they did it with a very simple, but clever device.”

The device is the result of eight years of patient and incremental work, mapped out on a whiteboard by Steve Collins and Greg Sawicki when they were graduate students together at the University of Michigan in 2007.

“Walking is more complicated than you might think,” said Collins, an assistant professor of mechanical engineering at Carnegie Mellon. “Everyone knows how to walk, but you don’t actually know how you walk.”

Collins, Sawicki and co-author M. Bruce Wiggin succeeded where so many in the past had failed by performing careful analyses of the biomechanics of human walking and then designing a simple, ultra-light-weight device that relieved the calf muscle of its efforts when it wasn’t doing any productive work.

Ultrasound imaging studies had revealed that the calf muscle exerts energy not only when propelling the body forward, but also when it performs a clutch-like action, holding the Achilles tendon taut.

“Studies show that the calf muscles are primarily producing force isometrically, without doing any work, during the stance phase of walking, but still using substantial metabolic energy,” Collins explained. “This is the opposite of regenerative braking. It’s as if every time you push on the brake pedal in your car, you burn a little bit of gas.”

With this insight in mind, the team created an ankle exoskeleton that offloads some of the clutching muscle forces of the calf, reducing the overall metabolic rate.

A mechanical clutch engages when the foot is on the ground and disengages when the foot is in the air, to avoid interfering with toe clearance. This clutch takes over the effort of the calf, producing force without using consuming any energy and thereby reducing the overall metabolic rate.

In developing the device, the research team faced a challenge. When you place heavy objects on the legs, there’s an initial penalty that increases your energy costs. Previous efforts had not been able to overcome that initial penalty. For that reason, it was critical to the researchers to keep the device light.

Over several years and many iterative designs, the team developed a carbon-fiber design that is ultra-light, yet rugged and functional. The entire device weighs approximately one pound per leg, or less than a work boot.

According to experts, the device is a triumph of elegance, simplicity and bio-specific interventions over complex, over-engineered designs.

“This unexpected and unprecedented result, with the potential to improve such a familiar human activity as walking, was discovered during a fundamental scientific study of mechanically augmented ankle function,” said Jordan Berg, a program director at NSF. “It is a great example of how basic research can lead to new beneficial devices.”

One of the long-term goals of Collins and Sawicki’s project is to use lightweight, energy-efficient exoskeletons to assist individuals with mobility issues.

“You can imagine these lightweight efficient devices being worn on the affected limb to help people with the permanent aftereffects of stroke,” Collins said. “We’re hopeful that designs that use similar techniques can help people who have had a stroke walk more easily. We’re still a little ways away from doing that, but we certainly plan to try.”

In the future, the team intends to test the current device with individuals who have a variety of mobility issues to determine what designs might work best for different populations. They are also interested in developing exoskeleton components for the knee and the hip, where they believe they may be able to garner even larger benefits.

“As we understand human biomechanics better, we’ve begun to see wearable robotic devices that can restore or enhance human motor performance,” said Collins. “This bodes well for a future with devices that are lightweight, energy-efficient, and relatively inexpensive, yet enhance human mobility.”

Related Website:
Experimental Biomechatronics Lab:http://biomechatronics.cit.cmu.edu/

The UCLA Biomechatronics Lab develops a language of touch that can be “felt” by computers and humans alike.

Research engineers and students in the University of California, Los Angeles (UCLA) Biomechatronics Lab are designing artificial limbs to be more sensational, with the emphasis on sensation.

With support from the National Science Foundation (NSF), the team, led by mechanical engineer Veronica J. Santos, is constructing a language of touch that both a computer and a human can understand. The researchers are quantifying this with mechanical touch sensors that interact with objects of various shapes, sizes and textures. Using an array of instrumentation, Santos’ team is able to translate that interaction into data a computer can understand.

The data is used to create a formula or algorithm that gives the computer the ability to identify patterns among the items it has in its library of experiences and something it has never felt before. This research will help the team develop artificial haptic intelligence, which is, essentially, giving robots, as well as prostheses, the “human touch.”

The driving routes (in green) computed by a Lexicographic Value Iteration (LVI) algorithm for an attentive driver (above) and a tired driver (below) based on traffic and road conditions. Credit: Shlomo Zilberstein, University of Massachusetts Amherst

For decades, researchers in artificial intelligence, or AI, worked on specialized problems, developing theoretical concepts and workable algorithms for various aspects of the field. Computer vision, planning and reasoning experts all struggled independently in areas that many thought would be easy to solve, but which proved incredibly difficult.

However, in recent years, as the individual aspects of artificial intelligence matured, researchers began bringing the pieces together, leading to amazing displays of high-level intelligence: from IBM’s Watson to the recent poker playing champion to the ability of AI to recognize cats on the internet.

These advances were on display last week at the 29th conference of the Association for the Advancement of Artificial Intelligence (AAAI) in Austin, Texas, where interdisciplinary and applied research were prevalent, according to Shlomo Zilberstein, the conference committee chair and co-author on three papers at the conference.

Zilberstein studies the way artificial agents plan their future actions, particularly when working semi-autonomously — that is to say in conjunction with people or other devices.

Examples of semi-autonomous systems include co-robots working with humans in manufacturing, search-and-rescue robots that can be managed by humans working  remotely and “driverless” cars. It is the latter topic that has particularly piqued Zilberstein’s interest in recent years.

The marketing campaigns of leading auto manufacturers have presented a vision of the future where the passenger (formerly known as the driver) can check his or her email, chat with friends or even sleep while shuttling between home and the office. Some prototype vehicles included seats that swivel back to create an interior living room, or as in the case of Google’s driverless car, a design with no steering wheel or brakes.

Except in rare cases, it’s not clear to Zilberstein that this vision for the vehicles of the near future is a realistic one.

“In many areas, there are lots of barriers to full autonomy,” Zilberstein said. “These barriers are not only technological, but also relate to legal and ethical issues and economic concerns.”

In his talk at the “Blue Sky” session at AAAI, Zilberstein argued that in many areas, including driving, we will go through a long period where humans act as co-pilots or supervisors, passing off responsibility to the vehicle when possible and taking the wheel when the driving gets tricky, before the technology reaches full autonomy (if it ever does).

In such a scenario, the car would need to communicate with drivers to alert them when they need to take over control. In cases where the driver is non-responsive, the car must be able to autonomously make the decision to safely move to the side of the road and stop.

“People are unpredictable. What happens if the person is not doing what they’re asked or expected to do, and the car is moving at sixty miles per hour?” Zilberstein asked. “This requires ‘fault-tolerant planning.’ It’s the kind of planning that can handle a certain number of deviations or errors by the person who is asked to execute the plan.”

With support from the National Science Foundation (NSF), Zilberstein has been exploring these and other practical questions related to the possibility of artificial agents that act among us.

Zilberstein, a professor of computer science at the University of Massachusetts Amherst, works with human studies experts from academia and industry to help uncover the subtle elements of human behavior that one would need to take into account when preparing a robot to work semi-autonomously. He then translates those ideas into computer programs that let a robot or autonomous vehicle plan its actions — and create a plan B in case of an emergency.

There are a lot of subtle cues that go into safe driving. Take for example a four-way stop. Officially, the first car to the crosswalk goes first, but in actuality, people watch each other to see if and when to make their move.

“There is a slight negotiation going on without talking,” Zilberstein explained. “It’s communicating by your action such as eye contact, the wave of a hand, or the slight revving of an engine.”

In trials, autonomous vehicles often sit paralyzed at such stops, unable to safely read the cues of the other drivers on the road. This “undecidedness” is a big problem for robots. A recent paper by Alan Winfield of Bristol Robotics Laboratory in the UK showed how robots, when faced with a difficult decision, will often process for such a long period of time as to miss the opportunity to act. Zilberstein’s systems are designed to remedy this problem.

“With some careful separation of objectives, planning algorithms could address one of the key problems of maintaining ‘live state’, even when goal reachability relies on timely human interventions,” he concluded.

The ability to tailor one’s trip based on human-centered factors — like how attentive the driver can be or the driver’s desire to avoid highways — is another aspect of semi-autonomous driving that Zilberstein is exploring.

In a paper with Kyle Wray from the University of Massachusetts Amherst and Abdel-Illah Mouaddib from the University of Caen in France, Zilberstein introduced a new model and planning algorithm that allows semi-autonomous systems to make sequential decisions in situations that involve multiple objectives — for example, balancing safety and speed.

Their experiment focused on a semi-autonomous driving scenario where the decision to transfer control depended on the driver’s level of fatigue. They showed that using their new algorithm a vehicle was able to favor roads where the vehicle can drive autonomously when the driver is fatigued, thus maximizing driver safety.

“In real life, people often try to optimize several competing objectives,” Zilberstein said. “This planning algorithm can do that very quickly when the objectives are prioritized. For example, the highest priority may be to minimize driving time and a lower priority objective may be to minimize driving effort. Ultimately, we want to learn how to balance such competing objectives for each driver based on observed driving patterns.”

It’s an exciting time for artificial intelligence. The fruits of many decades of labor are finally being deployed in real systems and machine learning is being adopted widely and for different purposes than anyone had ever realized.

“We are beginning to see these kinds of remarkable successes that integrate decades-long research efforts in a variety of AI topics,” said Héctor Muñoz-Avila, program director in NSF’s Robust Intelligence cluster.

Indeed, over many decades, NSF’s Robust Intelligence program has supported foundational research in artificial intelligence that, according to Zilberstein, has given rise to the amazing smart systems that are beginning to transform our world. But the agency has also supported researchers like Zilberstein who ask tough questions about emerging technologies.

“When we talk about autonomy, there are legal issues, technological issues and a lot of open questions,” he said. “Personally, I think that NSF has been able to identify these as important questions and has been willing to put money into them. And this gives the U.S. a big advantage.”

Investigators
Shlomo Zilberstein
Donald L. Fisher
Claudia Goldman
Kyle Hollins Wray
Luis Pineda
Avinoam Borowsky
Richard G. Freedman
Abdel-Illah Mouaddib

Related Institutions/Organizations
University of Massachusetts Amherst

Years Research Conducted
1996 – 2018

Total Grants
$2,385,857

Related Websites
The Resource-Bounded Reasoning Lab at the University of Massachusetts, Amherst: http://rbr.cs.umass.edu/

Credit: Siddharth Srivastava, Shlomo Zilberstein, Abhishek Gupta, Pieter Abbeel, Stuart Russell.

People typically consider doing the laundry to be a boring chore. But laundry is far from boring for artificial intelligence (AI) researchers like Siddharth Srivastava, a scientist at the United Technologies Research Center, Berkeley.

To AI experts, programming a robot to do the laundry represents a challenging planning problem because current sensing and manipulation technology is not good enough to identify precisely the number of clothing pieces that are in a pile and the number that are picked up with each grasp. People can easily cope with this type of uncertainty and come up with a simple plan. But roboticists for decades have struggled to design an autonomous system able to do what we do so casually — clean our clothes.

In work done at the University of California, Berkeley, and presented at the Association for Advancement of Artificial Intelligence conference in Austin, Srivastava (working with Abhishek Gupta, Pieter Abbeel and Stuart Russell from UC Berkeley and Shlomo Zilberstein from University of Massachusetts, Amherst) demonstrated a robot that is capable of doing laundry without any specific knowledge of what it has to wash.

Earlier work by Abbeel’s group had demonstrated solutions for the sorting and folding of clothes. The laundry task serves as an example for a wide-range of daily tasks that we do without thinking but that have, until now, proved difficult for automated tools assisting humans.

“The widely imagined helper robots of the future are expected to ‘clear the table,’ ‘do laundry’ or perform day-to-day tasks with ease,” Srivastava said. “Currently however, computing the required behavior for such tasks is a challenging problem — particularly when there’s uncertainty in resource or object quantities.”

Humans, on the other hand, solve such problems with barely a conscious effort. In their work, the researchers showed how to compute correct solutions to problems by using some assumptions about the uncertainty.

“The main issue is how to develop what we call ‘generalized plans,'” said Zilberstein, a professor of computer science and director of the Resource Bound Reasoning Lab at UMass Amherst. “These are plans that don’t just work in a particular situation that is very well defined and gets you to a particular goal that is also well defined, but rather ones that work on a whole range of situations and you may not even know certain things about it.”

The researchers’ key insight was to use human behavior — the almost unconscious action of pulling, stuffing, folding and piling — as a template, adapting both the repetitive and thoughtful aspects of human problem-solving to handle uncertainty in their computed solutions.

By doing so, they enabled a PR2 robot to do the laundry without knowing how many and what type of clothes needed to be washed.

Out of the 13 or so tasks involved in the laundry problem, the team’s system was able to complete more than half of them autonomously and nearly completed the rest–by far the most effective demonstration of laundering AI to date.

The framework that Srivastava and his team developed combines several popular planning paradigms that have been developed in the past using complex control structures such as loops and branches and optimizes them to run efficiently on modern hardware. It also incorporates an effective approach for computing plans by learning from examples, rather than through rigid instructions or programs.

“What’s particularly exciting is that these methods provide a way forward in a problem that’s well known to be computationally unsolvable in the worst case,” Srivastava said. “We identified a simpler formulation that is solvable and also covers many useful scenarios.”

“It is exciting to see how this breakthrough builds upon NSF-funded efforts tackling a variety of basic-research problems including planning, uncertainty, and task repetition,” said Héctor Muñoz-Avila, program director at NSF’s Robust Intelligence cluster.

Though laundry robots are an impressive, and potentially time-saving, application of AI, the framework that Srivastava and his team developed can be applied to a range of problems. From manufacturing to space exploration to search-and-rescue operations, any situation where artificially intelligent systems must act, despite some degree of uncertainty, can be addressed with their method.

“Using this approach, solutions to high-level planning can be generated automatically,” Srivastava said. “There’s more work to be done in this direction, but eventually we hope such methods will replace tedious and error-prone task-specific programming for robots.”

Investigators
Siddharth Srivastava
Shlomo Zilberstein
Paper

Related Institutions/Organizations
United Technologies Research Center
University of Massachusetts Amherst

Locations
Berkeley , California
Amherst , Massachusetts

Related Programs
Robust Intelligence

Related Awards
#0915071 RI: Small: Foundations and Applications of Generalized Planning

Years Research Conducted
2009 – 2015

Total Grants
$503,519

Related Agencies
DARPA
Office of Naval Research

Andrew Williams is shown with a couple of girls from the CompuGirls project. He says, “In this project we’re communicating to the girls that we’re going to teach you how to use these technology tools with robotics to advance your community, better understand yourself and show that you do belong in technology and you have some valuable contributions you can make.”
Photo credit: John C. Williams, Humanoid Engineering & Intelligent Robotics (HEIR) Lab, Marquette University

If one of the keys to building a future technology workforce is finding new ways to engage women and minorities in computer science, giving teenage girls an understanding of what the field is, and how they might contribute, matters.

Social scientist Kimberly Scott has been doing this since 2007, when she launched a National Science Foundation (NSF)-funded project called CompuGirls. Based at Arizona State University (ASU), the project reaches girls aged 8-12 from under-resourced schools in the Greater Phoenix area and Colorado, with activities to develop technical skills and encourage computational thinking in a way that’s culturally relevant.

CompuGirls is designed to reach girls outside of their normal school day. Opportunities include a one-week program during Fall or Spring Break, an 8th-period class over a couple of semesters, and a three-week summer class. In Phoenix, Scott is able to combine all three courses over five weeks. The activities include digital storytelling; working with Scratch software to build a game or simulation related to a social or community issue the girls have identified; and exposure to coding through researching a topic and putting it together in a virtual world. Some of the girls’ work is accessible at the CompuGirls website.

In engaging the girls in computer science, Scott and her team have to challenge assumptions that the field is boring and doesn’t relate to their interests.

“CompuGirls instills a sense of discipline, and we do see gains in their technical knowledge and comfort,” said Scott. “We also see that their definition of what a technologist is needs to be expanded.”

The program offers opportunities to interact with guest speakers from various scientific fields and to go on field trips to companies such as Intel.

In addition, taking on technical challenges such as performing debugging – which students may complain about at the time – gives them a sense of accomplishment; and Scott’s research shows that it makes them more interested in ideas related to computational thinking. Where Scott has analyzed the students’ outcomes across comparison groups, the CompuGirls participants have shown a greater ability to project themselves into the future, and are more able to articulate how what they do in the present can help them meet their future goals.

Being able to imagine a future identity as a technologist is key.

“The technology workforce is made up primarily of men,” said Scott. “It’s one thing to say we want to broaden the pipeline. I tell the girls I want them to transform the system and technologies to make them more inclusive.”

CompuGirls participants learn how to program the robots as well as study social aspects of human to robot interactions. They also look at questions such as how robots should interact with different kinds of people, and whether there’s a way for them to express emotions.
Photo credit: John C. Williams, Humanoid Engineering and Intelligent Robotics (HEIR) Lab, Marquette University

There is evidence that the CompuGirls experience has a positive impact on the educational experience of its graduates.

A couple of girls from the original class of 60 CompuGirls became first-generation college students at ASU, and worked with Scott in a Research Experiences for Undergraduates project over the summer. One of them contacted their CompuGirls classmates and found that among 10 who were at ASU as first-generation college students, three had declared a STEM-related major. All attributed their success in part to CompuGirls.

“It helped them articulate their voices,” said Scott, whose work earned her recognition as a White House Champion of Change last year. “They could say, ‘I can do this. I belong in this setting.'”

Giving young women that sense of belonging is also an emphasis of Andrew Williams, who heads the Humanoid Engineering & Intelligent Robotics (HEIR) Lab at Marquette University. Indeed, this is a theme of Williams’ recent TEDx talk, Belonging in Technology: What I learned from Steve Jobs. Prior to his appointment at Marquette, among his accomplishments was leading Spelman College’s SpelBots robotic soccer team to competing successfully at several international competitions.

A recent NSF-funded STEM for All meeting co-led by Scott was a chance to talk about the National Robotics Initiative (NRI), which he saw as a great opportunity for an intersection between Scott’s work and robotics for young African-American, Hispanic, and Native American students in grades K-12.

“I already had a lot of experience with robots,” said Williams. “We decided to write a proposal to integrate humanoid robots with CompuGirls.”

NSF funded the project. With NRI’s focus on developing robots that can closely interact with people, Williams and Scott saw the opportunity to have young women study robotics in conjunction with the social sciences, computer science and engineering in a culturally sensitive way.

Humanoid robots have the ability to interact with people–to see them, talk to them and respond to them. Among their anticipated uses is helping the elderly and people with disabilities safely interact with their environment.

“A humanoid robot is like the iPhone of robots,” said Williams. “You don’t have to explain it. It’s intuitive.”

The robotics project has built on Scott’s work in teaching digital literacy in the context of social justice, identity and gender issues. CompuGirls participants will learn how to program the robots as well as study social aspects of human-to-robot interactions. They’ll look at questions such as how robots should interact with different kinds of people, and whether there’s a way for the robots to express emotions.

“In this project we’re communicating to the girls that we’re going to teach you how to use these technology tools with robotics to advance your community, better understand yourself and show that you do belong in technology and you have some valuable contributions you can make,” said Williams.

In a world where humanoid robots will be part of our future, engaging these young women now gives them a role in shaping that future.

In Williams’ research lab at Marquette, students are already using a 3-D printer to produce parts of humanoid robots, and the robots have played robo-soccer in Brazil. They are also developing a smartphone-type interface with the robots.

Williams has also recently been awarded an NSF I-Corps for Learning grant investigating the scalable sustainability of creating humanoid robot kits–using the 3-D printed robots–to teach STEM.

Williams has a message for young women on the occasion of Computer Science Education Week.

“They should know that they belong in this technology field of computer science,” said Williams. “They can belong and they can really achieve. They can have a vision of computer science or robotics and can someday apply it at a place like Google.”

This image of a robot arm, developed by the Stanford Research Institute, is similar to the one that appeared in the 1976 NSF Annual Report. The robotic system used computer vision to identify and make decisions about parts on an assembly line. This is one of several projects from that era aimed at improving the productivity of American manufacturing processes. Credit: SRI International

The fundamental research in computing and engineering that has enabled robotics to develop in the US has been supported by the National Science Foundation (NSF) since its inception. Yet despite these early investments in sensors, machine movement and computer vision, it wasn’t until 1972 that the first grant with “robot” in the title was funded in the US . [That grant was for $39,300, and went to Laurent Siklossy at the University of Texas at Austin for Explorations in Problem Solving, Theorem Proving, Theorem Discovery and Robot Guidance.]

1970s: Robots for the factory floor

In the mid-1970s, robotics began to gather steam at NSF. Among the first research projects classified as robotics were mechanical arms (seen above, right) that could pick a part out of a box and visually identify it and orient it properly for the next step on an assembly line, as well as computer-controlled welding robots. These and other NSF-funded projects were aimed at improving the productivity of American manufacturing processes — a goal for roboticists that continues to this day.

1980s: Rise of the walking machines

The 1980s brought an increased diversification in the types of robots being explored and the ways they could be used.

At Ohio State, electrical engineer Robert McGhee and mechanical engineer Kenneth Waldron, along with a 60-member team of students and technical assistants, developed the Adaptive Suspension Vehicle (ASV), nicknamed the “Walker,” with support from NSF and the Defense Advanced Projects Research Agency (DARPA).

Ohio State’s Adaptive Suspension Vehicle (AVS), nicknamed the “Walker.” Developed by electrical engineer Robert McGhee and mechanical engineer Kenneth Waldron, along with a 60-member team of students and technical assistants, the ‘Walker’ was designed to carry cargo for industrial and military applications over rough, mountainous, icy or muddy terrain. Credit: The Ohio State University Archives

The ASV was 17 feet long, 8 feet wide, and 10.5 feet high, and had six legs to support its three-ton aluminum body. It was designed to carry cargo for industrial and military applications over rough, mountainous, icy or muddy terrain, and was capable of crossing 9-foot-wide ditches or 7-foot-high walls.

The walker used a forward mounted radar system to scan the terrain ahead and feed that data, along with instructions from the operator’s joystick, into the 16 onboard computers that coordinated and controlled the ASV’s legs. Computers moved each leg individually, up and down, forward and back, and closer or farther from the ASV’s body, for a clunky but serviceable ride.

1990s: Robots explore new environments

First built in 1991, the Omni-Directional Intelligent Navigator (ODIN) was a sphere-shaped, autonomous underwater robot capable of instantaneous movement in six directions. Credit: Autonomous Systems Laboratory, University of Hawaii

Not long afterward, researchers supported by NSF were developing robots for a very different environment: underwater. First built in 1991, the Omni-Directional Intelligent Navigator (ODIN) was a sphere-shaped, autonomous underwater robot capable of instantaneous movement in all six directions.

First built as a remotely operated robot, in 1995 it was upgraded to ODIN II, an autonomous underwater robot. Sentry, a successor robot developed through a grant from NSF, plies the deep waters today locating and quantifying hydrothermal fluxes.

In the 1990s, roboticists began turning their attention to day-to-day tasks with which a robot could assist. For instance, researchers from the University of Pittsburgh, University of Michigan and Carnegie Mellon University developed a series of mobile, personal service robots, such as Nursebot, that were designed to assist elderly people in their everyday life.

An autonomous mobile robot that “lives” in the home of a chronically ill elderly person could remind its owner to take medicine, provide videoconferencing with doctors, collect patient data or watch for accidents, manipulate objects for arthritis sufferers, and provide some social interaction. New versions have evolved over the years and a General Electric developed hospital robot is expected to be tested at a Veterans Affairs hospital in 2015.

Nursebot was designed by researchers at the University of Pittsburg, the University of Michigan, and Carnegie Mellon University to assist elderly people in their everyday life. Credit: Carnegie Mellon University.

2000s: Miniaturization and mobility

Researchers have always envisioned a future where robots could serve the general good in disaster recovery and search-and-rescue operations, but it wasn’t until 9/11 that robots were broadly put to that use.

Robotics expert Robin Murphy, then an associate professor of computer science at the University of South Florida, arrived on site the morning after the collapse of the World Trade Center. Murphy’s research on experimental mixed-initiative robots for urban rescue operations was originally funded by NSF. She brought with her a response team that included three graduate students — Jenn Casper, Mark Micire and Brian Minten— and software-guided “marsupial” robot systems. These intelligent anonymous “marsupial” robots are especially useful in rubble because the “mother” robot releases smaller robots to explore tight spaces unreachable by other means.

From September 12th – 23rd, National Science Foundation-funded researchers aided World Trade Center recovery efforts. University of South Florida engineering professor Robin Murphy and three graduate students took six urban search and rescue robots to “ground zero” in New York to help find survivors. Murphy’s 11-day mission was a part of a larger team that recovered remains of six victims. Murphy’s robots are unique in that they are small and can maneuver in very tight situations. Tethered and fitted with headlights and cameras, these robots bring distinct advantages to a rescue mission the magnitude of the World Trade Center attacks where the damage is massive and recovery very dangerous. Although they cost between $10,000 and $40,000, Murphy foresees search-and-rescue robots becoming standard equipment in fire departments across the country. Credit: FEMA

Over the next 11 days, the teams made five insertions onto the rubble piles, often at the request of the Federal Emergency Management Agency (FEMA) task force teams or sector chiefs. Murphy’s mechanized prowlers had tethers with a range of 100 feet, far out-stripping the fire department’s seven-foot camera wands. In doing so, they helped find five victims and another set of remains, though Murphy expressed regret that they hadn’t been more successful.

As the 2000s progressed, efforts by engineers to miniaturize components led to robots that were significantly smaller than those that came before. One startling example of this trend is the RoboBee project, which was awarded an Expeditions in Computing award from NSF’s Directorate for Computer and Information Science and Engineering in 2009.

Researchers in this expedition are creating robotic bees that fly autonomously and coordinate activities amongst themselves and the hive, much like real bees. The research team aims to drive research in compact, high-energy power sources, ultra-low-powered computing and the design of distributed algorithms for multi-agent systems. Most recently, RoboBees were pollinating young minds at the Boston Museum of Science in an exhibition dedicated to their complex design.

Inspired by the biology of a fly, with submillimeter-scale anatomy and two wafer-thin wings that flap almost invisibly, 120 times per second, the Robobee takes its first controlled flight. The culmination of a decade’s work, RoboBees achieve vertical takeoff, hovering, and steering. Credit: Kevin Ma and Pakpong Chirarattananon

2010s: Investing in co-robots

In June 2011, the administration launched the National Robotics Initiative (NRI) to develop robots that work with or beside people to extend or augment human capabilities, taking advantage of the different strengths of humans and robots. This provided focus and funding for robotics research. The NRI is led by NSF and supported by multiple agencies including the National Aeronautics and Space Administration (NASA), the National Institutes of Health (NIH), the U.S. Department of Agriculture (USDA), and the U.S. Department of Defense (DOD).

Since 2011, NSF and its partners in the NRI have invested more than $120 million in robotics research.

Today, robots impact our lives in a myriad of ways. Robots are being used in classrooms across the nation to capture the excitement of students and help them learn STEM (and non-STEM) principles. They’re helping doctors perform surgeries, providing assistance to individuals with disabilities and inspecting bridges and roads to ensure our safety.

“Robots, which once were limited to the realm of science fiction, are now a transformative technology, a demonstration of how NSF-funded basic research can bring about changes in human life,” said Marc Rothenberg, former NSF historian.

Though robots may be an emerging area of research, many of the underpinnings of today’s robots began in fundamental research in sensing, computer vision, artificial intelligence, mechanical engineering and many other areas that some might not immediately recognize as being related to robots.

What’s next

NSF-supported researchers are making incredible advances in robotics, creating a new generation of co-robots that can handle critical tasks in close proximity to humans, safely and with greater resilience than previous intelligent machines.

Check out our Youtube Robot playlist to learn more about more about the research of NSF-funded roboticists.

The idea of SupraPeds is to increase locomotion stability by incorporating a pair of actuated smart staffs with vision and force sensing that transforms biped humanoids into multi-legged robots. Credit: Oussama Khatib, Stanford University; Shuyun Chung, Stanford University.

Anyone who has ever watched a humanoid robot move around in the real world — an “unstructured environment,” in research parlance — knows how hard it is for a machine to plan complex movements, balance on uneven surfaces or traverse levels.

Quadruped robots, like Boston Dynamic’s BigDog, seem to be more adept, but don’t have the ability to use their hands to open doors or pull people from rubble.

Humans have trouble balancing sometimes too — say when climbing a mountain or walking on ice. In many cases, people turn to a trusted and time-tested tool: the walking staff.

“Efforts to make robots emulate humans have ignored an important fact: whenever humans approach their limits, they augment their capabilities with a diverse variety of assistive tools,” said Shuyun Chung, a postdoc working with Oussama Khatib in the Stanford University Artificial Intelligence lab.

“Perhaps the most useful tool is the walking staff, which improves support, enables load redistribution and can also be used as a sensor to probe the stability of planned footsteps.”

Researchers at Stanford University have been developing a robotic platform they call SupraPed that uses “smart staffs,” inspired by walking sticks, to better balance and traverse terrain.

Working in disaster recovery situations, the SupraPed can use the staffs to explore the terrain around them, expand the range of movements that are possible and even communicate a sense of touch to a human at a remote site, who can use that data to plan the robot’s movements.

Their work is supported by a grant from the National Science Foundation through the National Robotics Initiative, an effort to develop the next generation of robots that work beside or cooperatively with people.

A diagram showing some of the technologies incorporated into SupraPeds’ actuated smart staffs. Credit: Oussama Khatib, Stanford University; Shuyun Chung, Stanford University.

The smart staff is no simple stick. It’s equipped with 3-D vision capabilities and tactile sensors that can assess the topography of the surface, the friction of the material, its ability to sustain weight and other information relevant to planning the robot’s next move.

Just as we might poke a rock to see how stable it is before we jump on it to cross a creek, SupraPed can use its smart staffs to test the ground before it takes a step.

But endowing SupraPed or other future robots with increased mobility isn’t simply a question of handing a robot a computerized stick. The Stanford team needed to create a suite of new algorithms and control mechanisms to allow the robot to incorporate and control the staff.

Among the challenges they had to overcome in the research was that of balancing a changing center of mass during an activity. This is difficult in rough, irregular terrain and more so with a system that uses input from four independent surfaces to balance and move.

The Stanford research team developed new mathematical formulations of the relationship between internal forces and movement control, allowing them to surmount existing problems and provide improved stability for the SupraPed.

Before developing a physical prototype of their system, the researchers first ran SupraPed through a series of simulated tests using the Simulation and Active Interface framework developed in their lab. The simulations placed SupraPed in series of challenging terrains–outcroppings of rubble–and tested how the robotic platform reacted when it encountered obstacles.

In each simulation, the robot plants its smart staffs, testing its firmness just as an expert hiker might, and swings its foot forward, moving from one balanced position to the next.

Using the algorithms developed by the Stanford team, it almost never falls.

The team presented their work at the IEEE International Conference on Robotics and Automation. The first prototype of smart staff has been designed and fabricated, and is being tested in the lab environment. The researchers hope that one day, the telescopic staff easily can be integrated with any human-size humanoid robot.

“Although research on rescue robots working in hazardous environments has made tremendous progress over the past decade,” Chung said, “we believe that SupraPeds will bring a whole new level of mobility and manipulation for humanoid robots.”

The SupraPeds also herald a future where robots, like humans, use tools to augment their abilities.

Investigators
Oussama Khatib
Shuyun Chung

Related Institutions/Organizations
Stanford University

Related Programs
National Robotics Initiative 

The Baxter robot hands off a cable to a human collaborator — an example of a co-robot in action. Photo credit: Aaron Bestick, UC Berkeley.

From disaster recovery to caring for the elderly in the home, scientists and engineers are developing robots that can handle critical tasks in close proximity to humans, safely and with greater resilience than previous generations of intelligent machines.

Today the National Science Foundation (NSF), in partnership with the National Institutes of Health, US Department of Agriculture and NASA announced $31.5 million in new awards to spur the development and use of co-robots — robots that work cooperatively with people.

The awards mark the third round of funding made through the National Robotics Initiative (NRI), a multi-agency program launched in September 2012 as part of the Advanced Manufacturing Partnership Initiative, with NSF as the lead federal agency.

“Robots and robotic systems have the potential to augment human abilities, improve our quality of life and perform dangerous tasks unsuitable for people,” said Suzi Iacono, acting assistant director of the Computer and Information Science and Engineering Directorate at NSF. “Working with our federal partners in NRI has spurred new research directions that weren’t previously possible without these collaborations.”

The 52 new research awards, ranging from $300,000 to $1.8 million over one to four years, advance fundamental understanding of robotic sensing, motion, computer vision, machine learning and human-computer interaction. The awards include research to develop soft robots that are safer for human interaction, determine how humans can lead teams of robots in recovery situations and design robots that can check aging infrastructure and map remote geographic areas.

A full listing of the NRI investments made by NSF is available on the NSF’s NRI Program page.

The NRI awards address the entire development cycle of robots, from fundamental research to deployments in critical environments, and will help make safe, helpful and affordable co-robots a reality.

“Our engineers and scientists are creating a world where robotic systems serve as trusted co-workers, co-inhabitants, co-explorers and co-defenders,” said Pramod Khargonekar, assistant director of NSF’s Engineering Directorate. “The National Robotics Initiative serves the national good by encouraging collaboration among academic, industry, non-profit and other organizations — and by speeding creation of fundamental science and engineering knowledge base used by researchers, applications developers and industry.”

NSF’s investments in robotics explore both the technical and engineering challenges of developing co-robots and the long-term social, behavioral and economic implications of co-robots across all areas of human activities. As part of the initiative, NSF also supports the development of new methods for the establishment and infusion of robotics in educational curricula.

Earlier this month, NRI announced its latest solicitation, which has been joined by the Defense Advanced Research Projects Agency (DARPA) and the Department of Defense as new partners. The program expects to award up to $50 million in 2015.

If you liked this article, you may also be interested in:

• Japan holds first “robotics revolution” council meeting
• £1.25M advanced drone lab gets the go ahead
• China takes long view in funding robotics, innovation
• EU launches world’s largest civilian robotics programme – 240,000 new jobs expected
• Swiss to invest almost CHF30 million in digital fabrication research over next 4 years
• Robot tourism coming soon to Korea: Masan Robot Land project finally breaks ground

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.