Towards intelligent industrial co-robots

By Changliu Liu, Masayoshi Tomizuka

Democratization of Robots in Factories

In modern factories, human workers and robots are two major workforces. For safety concerns, the two are normally separated with robots confined in metal cages, which limits the productivity as well as the flexibility of production lines. In recent years, attention has been directed to remove the cages so that human workers and robots may collaborate to create a human-robot co-existing factory.

Manufacturers are interested in combining human’s flexibility and robot’s productivity in flexible production lines. The potential benefits of industrial co-robots are huge and extensive, e.g. they may be placed in human-robot teams in flexible production lines, where robot arms and human workers cooperate in handling workpieces, and automated guided vehicles (AGV) co-inhabit with human workers to facilitate factory logistics. In the factories of the future, more and more human-robot interactions are anticipated to take place. Unlike traditional robots that work in structured and deterministic environments, co-robots need to operate in highly unstructured and stochastic environments. The fundamental problem is how to ensure that co-robots operate efficiently and safely in dynamic uncertain environments. In this post, we introduce the robot safe interaction system developed in the Mechanical System Control (MSC) lab.

Fig. 1. The factory of the future with human-robot collaborations.

Existing Solutions

Robot manufacturers including Kuka, Fanuc, Nachi, Yaskawa, Adept and ABB are providing or working on their solutions to the problem. Several safe cooperative robots or co-robots have been released, such as Collaborative Robots CR family from FANUC (Japan), UR5 from Universal Robots (Denmark), Baxter from Rethink Robotics (US), NextAge from Kawada (Japan) and WorkerBot from Pi4_Robotics GmbH (Germany). However, many of these products focus on intrinsic safety, i.e. safety in mechanical design, actuation and low level motion control. Safety during social interactions with humans, which are key to intelligence (including perception, cognition and high level motion planning and control), still needs to be explored.

Technical Challenges

Technically, it is challenging to design the behavior of industrial co-robots. In order to make the industrial co-robots human-friendly, they should be equipped with the abilities to: collect environmental data and interpret such data, adapt to different tasks and different environments, and tailor itself to the human workers’ needs. For example, during human-robot collaborative assembly shown in the figure below, the robot should be able to predict that once the human puts the two workpieces together, he will need the tool to fasten the assemble. Then the robot should be able to get the tool and hand it over to the human, while avoid colliding with the human.

Fig. 2. Human-robot collaborative assembly.

To achieve such behavior, the challenges lie in (1) the complication of human behaviors, and (2) the difficulty in assurance of real time safety without sacrificing efficiency. The stochastic nature of human motions brings huge uncertainty to the system, making it hard to ensure safety and efficiency.

The Robot Safe Interaction System and Real-time Non-convex Optimization

The robot safe interaction system (RSIS) has been developed in the Mechanical System Control lab, which establishes a methodology to design the robot behavior to achieve safety and efficiency in peer-to-peer human-robot interactions.

As robots need to interact with humans, who have long acquired interactive behaviors, it is natural to let robot mimic human behavior. Human’s interactive behavior can result from either deliberate thoughts or conditioned reflex. For example, if there is a rear-end collision in the front, the driver of a following car may instinctively hit the brake. However, after a second thought, that driver may speed up to cut into the other lane to avoid chain rear-end. The first is a short-term reactive behavior for safety, while the second needs calculation on current conditions, e.g. whether there is enough space to achieve a full stop, whether there is enough gap for a lane change, and whether it is safer to change lane or do a full stop.

A parallel planning and control architecture has been introduced mimicking these kind of behavior, which included both long term and short term motion planners. The long term planner (efficiency controller) emphasizes efficiency and solves a long-term optimal control problem in receding horizons with low sampling rate. The short term planner (safety controller) addresses real time safety by solving a short-term optimal control problem with high sampling rate based on the trajectories planned by the efficiency controller. This parallel architecture also addresses the uncertainties, where the long term planner plans according to the most-likely behavior of others, and the short term planner considers almost all possible movements of others in the short term to ensure safety.

Fig. 3. The parallel planning and control architecture in the robot safe interaction system.

However, the robot motion planning problems in clustered environment are highly nonlinear and non-convex, hence hard to solve in real time. To ensure timely responses to the change of the environment, fast algorithms are developed for real-time computation, e.g. the convex feasible set algorithm (CFS) for the long term optimization, and the safe set algorithm (SSA) for the short term optimization. These algorithms achieve faster computation by convexification of the original non-convex problem, which is assumed to have convex objective functions, but non-convex constraints. The convex feasible set algorithm (CFS) iteratively solves a sequence of sub-problems constrained in convex subsets of the feasible domain. The sequence of solutions will converge to a local optima. It converges in fewer iterations and run faster than generic non-convex optimization solvers such as sequential quadratic programming (SQP) and interior point method (ITP). On the other hand, the safe set algorithm (SSA) transforms the non convex state space constraints to convex control space constraints using the idea of invariant set.

Fig. 4. Illustration of convexification in the CFS algorithm.

With the parallel planner and the optimization algorithms, the robot can interact with the environment safely and finish the tasks efficiently.

Fig. 5. Real time motion planning and control.

Towards General Intelligence: the Safe and Efficient Robot Collaboration System (SERoCS)

We now work on an advanced version of RSIS in the Mechanical System Control lab, the safe and efficient robot collaboration system (SERoCS), which is supported by National Science Foundation (NSF) Award #1734109. In addition to safe motion planning and control algorithms for safe human-robot interactions (HRI), SERoCS also consists of robust cognition algorithms for environment monitoring, optimal task planning algorithms for safe human-robot collaboration. The SERoCS will significantly expand the skill sets of the co-robots and prevent or minimize occurrences of human-robot collision and robot-robot collision during operation, hence enables harmonic human-robot collaboration in the future.

Fig. 6. SERoCS Architecture.

This article was initially published on the BAIR blog, and appears here with the authors’ permission.

References

BAIR Blog is the official blog of the Berkeley Artificial Intelligence Research (BAIR) Lab.

BAIR Blog is the official blog of the Berkeley Artificial Intelligence Research (BAIR) Lab.