In this illustration, NASA’s Ingenuity Mars Helicopter stands on the Red Planet’s surface as NASA’s Perseverance rover (partially visible on the left) rolls away. Credits: NASA/JPL-Caltech

The difference between robotics and automation is almost nonexistent and yet has a huge difference in everything from trade shows, marketing, publications to academic conferences and journals. This week, the difference was expressed as an opportunity in the Dear Colleague Letter below from Professor Ken Goldberg, CITRIS CPAR and UC Berkeley, who suggested that students whose papers were rejected from ICRA, revise them for CASE, the Conference on Automation Science and Engineering. This opportunity was expressed beautifully in the title quote from Professor Raja Chatila, ex President of IEEE Robotics and Automation Society and current President of IEEE Global Society on Ethics of Autonomous and Intelligent Systems. “One robot on Mars is robotics, ten robots on Mars is automation.”

Dear Colleagues,

Over 2000 papers were declined by ICRA today, including many that can be
effectively revised for another conference such as IEEE CASE (deadline 15
March).

IEEE CASE, the annual Conference on Automation Science and Engineering, is
a major IEEE conference that is one of three fully-supported IEEE
conferences in our field (with ICRA and IROS).

In 2021 CASE will be held 23-27 August.  It will be hybrid, with a live
component in Lyon France and an online component:
https://case2021.sciencesconf.org/

IEEE CASE was founded in 2006 so is smaller but growing quickly.  The
acceptance rate for the last CASE was about 56%, higher than ICRA 2021
(48%), IROS, or RSS.  I consider this a feature not a bug: it is an
excellent venue for exploratory and novel projects.

IEEE CASE continues the classic conference model of featuring a 10-15 min
oral presentation of each paper in contrast to poster sessions.  This is
particularly exciting for students, who get the valuable experience of
lecturing and fielding questions in front of an audience of peers.

IEEE CASE also has a tradition of spotlighting papers nominated for awards
such as Best Paper, Best Student Paper, etc.  Each nominated paper is
presented in special single session track on Day 1, where everyone at the
conference attends and there is a lively Q&A led by judges.

IEEE CASE emphasizes Automation.  Automation is very closely related to
Robotics. There is substantial overlap, but Automation emphasizes
efficiency, robustness, durability, safety, cost effectiveness. Automation
also includes topics such as optimization and applications such as
transportation and mfg. I like how RAS President Raj Chatila summed up the
relationship 10 years ago: “One robot on Mars is robotics, ten robots on
Mars is automation.”

In China there are over 100 university departments
focused on Automation.  The impact factor for the IEEE Transactions on
Automation Science and Engineering (T-ASE) this year is on par with T-RO
and higher than IJRR. Automation is important to put robotics into
practice.

Ken Goldberg

Professor, Industrial Engineering and Operations Research

William S. Floyd Jr. Distinguished Chair in Engineering, UC Berkeley

Director, CITRIS People and Robots Lab

“Grocery workers are risking their safety, often for poverty-level wages, so the rest of us can shelter in place,” said John Logan, director of labor and employment studies at San Francisco State University. “The only way the rest of us are able to stay home is because they’re willing to go to work.” [Washington Post April 12 2020]

In our April 7th edition of “COVID-19, robots and us”, we heard from Linda Pouliot, CEO and founder of Dishcraft Robotics and Vipin Jain, CEO and founder of Blendid. Both provided us with insights into robotics in the food industry and the difficulties and joys of adapting robots to COVID-19 work.

Dishcraft Robotics is Linda Pouliot’s fourth startup, and second robotics startup. She previously cofounded Neato Robotics, the number two player in automated vacuuming. Dishcraft Robotics is a clean dish delivery service for cafeterias, or large kitchens. On a daily basis they deliver customized clean dishes to the customer, they collect the dirty dishes, and return them to a central hub where custom robots wash them.

“We initially thought we would develop the robot and install it directly into dish rooms. And then we found that it was a really high hurdle for an operation to have a big capital expense, and that we could solve their problem in a really frictionless way, by simply doing the delivery because there’s no upfront cost for them. There is no risk for them to try it. And in fact, we give a free couple week trial and we have found that once you start with dish craft, you immediately convert because the service solves all their labor problems, it also is more sustainable. And in today’s environment, it’s very, very sanitary.”

“We give them carts full of clean dishes, they are our dishes that are proprietary. The way our robots work is using magnetics. And so it requires using our specialized wares. And so we just bring in flexible, clean dishes and we give them a collection system where they can drop off. It’s very organized, it’s ADA compliant. It’s pretty space saving compared to where people normally store their dishes. So it’s worked really well for us. And that means for a customer, there’s no construction costs. There’s no months of planning. You know, it’s just been pretty delightful for both us and them.”

“Currently we service essential businesses and we have started to offer our services to help in this time of need to hospitals. Dish rooms are very small and cramped spaces and we realized that workers in them were unlikely to be able to be socially distance. Also today, the hospitals are overloaded. Some have twice the volume of what they normally expect a dish room to have to process. And so this was a great opportunity for them to use robots and use our service and keep their existing staff very safe.” [Linda Pouliot April 7 2020]

Vipin Jain is the CEO and founder of Blendid, a food robotic kiosk startup. After about 5 years of R&D Blendid opened their first kiosks just over a year ago in March 2019 at the University of San Francisco, followed by Sonoma State University, and while they were initially doubtful that people were ready to have their food or drink made by robots, they’ve seen great interest.

“We were told that automating food was too hard. And people won’t like it. People don’t want robots to make the food, people said. But people have different tastes and and standardized food doesn’t cut it anymore. So you need a better solution so that I can get what I like to eat anytime of the day wherever I am. And that can only be done using a lot of data, AI and robotics.”

“We were just getting ready to start deploying into retail, corporate and hospital environments when the COVID hit. So I wish we were a little bit ahead further ahead in terms of deployment because as you can imagine, food is an essential service. We all want food, we all want access to food, but in this environment, we will be fearful about the safety of people working on food preparation, because we don’t know how the food was handled.” [Vipin Jain April 7 2020]

Rich Stump is one of the cofounders of Studio Fathom for prototyping, product development and production services. Fathom specializes in 3d printing additive manufacturing, dealing with multiple materials and leveraging traditional manufacturing alongside, allowing them to solve some interesting engineering challenges or product development challenges. Fathom has been very active in COVID-19 response.

“There’s three main initiatives that we have going right now with COVID-19. And some of them are, are potentially revenue generating, but most of them are just just trying to help us with our expertise and capabilities. The first is obviously with PPE challenges we have. We have a facility in Asia and southern China and so we have a vast amount of supply chain resources. So, we called in our colleagues over there and we found a number of factories that had FDA approved PBE supplies that that were overrun from the flattening of the curve in Asia.”

“The news has been talking about investment into 3M and Honeywell ramping up production in the US, which is absolutely great. The problem is, it’s just not going to get done in time, right? Anytime you buy 150 thermoforming machines and you try to ramp up production to make n95 masks, it’s going to take weeks or potentially months to get any volume. So what we were trying to do is match hospitals and senior homes and folks that needed PPE with supply chain resources that we had in Asia.”

“We’ve had a bunch of roadblocks to the point where we’ve tried to reach out to the state government officials to try to remove some of the roadblocks with customs and tariffs and freight constraints. And so that’s been a an interesting challenge. But we’ve been able to connect, I think over 1.5 million and counting supplies into the US from Asia. It’s not our everyday business, but we have resources, so we tried to help there.

“The second initiative was around the test kits. Obviously there’s been a shortage of test kits. And it turns out, interestingly enough, one of the big shortage items is the nasopharyngeal swab that goes back into your nasal cavity. The main manufacturer was in Italy, and obviously with everything happened in Italy, that that shut down a lot of the supply. There’s been a community of about 50 folks that have come out of the 3d printing community to see if we can 3d print these swabs. And we’ve made a lot of progress.”

There’s now four manufacturers approved to 3d print nasopharyngeal swabs. Next week, we’ll get into production of these in at least hundreds of thousands, if not millions, in order to support the short term need. Long term, it probably doesn’t make sense to 3d print them given the cost base. But using 3d printing technology, we can design swabs that could potentially perform better than the traditional swab. So that’s been a fun and challenging project at the same time.”

“And the third initiative is around the ventilators. You have folks like Lawrence Livermore developing a ventilator with spare parts that are available today. And you have JPL, the NASA team, trying to develop their own ventilator. Then you have the large automotive manufacturers who have partnered up with various existing ventilator manufacturers and are trying to ramp up production. By today I think more than 270 people have come to us with either a ventilator design, a mask design, or some type of shield design, just over the last 10 days. There’s just a tremendous amount of activity. And it’s great because everyone wants to help, but at the same time, it presents a lot of challenges because so much effort going in so many different directions, that you you worry that, you know, all these efforts won’t end up being impactful by the time we need the supplies.” [Rich Stump April 7 2020]

Mark Martin is the Director of Industry and Workforce Development for California Community Colleges and founder of the Bay Area Manufacturers Discussion Forum. He’s very connected with manufacturers in the region and has a lot of insights into what’s going on in the manufacturing sector.

“Many of the manufacturing companies are being hurt and having to furlough people. There’s about 8000 in the Bay Area and we’re trying to help them repurpose. In some things, it’s relatively easy, maybe you have an injection mold facility, and you can injection mold these face shields, or some of these parts of the face shield. You have to make the molds but it’s still within your core business. Others are contract manufacturers that could theoretically assemble ventilators because they’re already doing medical products. Ventilators are a complicated product, but not necessarily more complicated than some of these people already doing.”

“But for others, you might have to get specialized machines, which can take a long time. And then you need to have expertise around. If you haven’t done medical products before, do you know how to handle the equipment and ensure QC? And the FDA approves factories, not just the designs. I created a list of Bay Area manufacturers and I got almost 200 responses with things they think they can manufacture and I’ve supplied that list to the state government.”

“And community colleges have makerspaces or fab labs. So where I’m located at Laney College in Oakland, we started working on face shields a week and a half ago, because that was relatively simple for us to be able to do. And we printed up 500 face shields to Highland hospital. It took us a day to do it with six people. And I asked the hospital what their daily use was for face shields and they said 600 a day. So I was like, all right, that was a day’s worth for this one hospital.”

“And then we actually took a design that was done by Stratasys for 3D printing and modified it a little bit. And in like two days, we got a molder to set up injection molding, although it’s going to take a couple of weeks to finish. And honestly, I have no idea if we’re going to need in a couple weeks. Because I have no idea what the demand is.”

“Apple’s bringing in over a million a week and others are doing this hundreds or even thousands a time in basements and little shops. But we don’t know if the demand is 5 million a week or 50 million. “

“So that that’s the thing the manufacturers all trying to figure out. I kept wondering why the government didn’t take a little bit of control. Even just financial incentives to say we will backstop you if you supply PPE, so that if the demand falls out, you don’t lose money.” [Mark Martin April 7 2020]

Ken Goldberg is the William S. Floyd Jr Distinguished Chair in Engineering at UC Berkeley, Director of CITRIS People and Robots Initiative and a regular guest on COVID-19, robots and us. Today in keeping with the theme of food robotics, Ken explained the reasoning behind one of his latest projects, AlphaGarden.

This is a farm bot that you can buy from a local company, in San Luis Obispo for about $3,000. Our garden includes 14 species of edible herbs and plants and they’re all growing in close proximity. The challenge is that they’re competing for increasingly scarce light and water. So what happens is that this starts to get quite out of control. Right now, the garden has been acting very autonomously because we haven’t been able to get into the greenhouse, and so it’s starting to decay, and certain invasive species like mint have essentially taken over.”

“We’re trying to optimize diversity, that is to maximize the number of different plant types that are growing. But the key idea here is that we’re using AI to simulate the garden, because a grow cycle takes three months. So we have a simulator that can simulate 10,000 times the speed of nature. We have 64 gardens but thousands of parameters to fine tune the system. Then we evaluate each of those gardens to figure out how to tune the parameters for the watering and pruning devices.”

“This is an ongoing project but the end goal is to be able to learn a policy for successful gardening because polyculture is much more labor intensive than monoculture. The reason that I call it an art project is because it’s extremely difficult for AI and robotics. It’s a big challenge. I’m not at all convinced we’re actually going to succeed. We’re really putting AI to the test. But just this week we did a deep learning method based on the data that we collected from our simulator. And in time we hope to learn how to be able to automate some of these very difficult tasks like organic gardening.” [Ken Goldberg April 7 2020]

The moderator of the weekly ‘COVID-19, robots and us’ discussions, Andra Keay, is the Managing Director of Silicon Valley Robotics, supporting innovation and commercialization of robotics technologies. Andra was also the Industry Co-Chair of the Human-Robot Interaction Conference 2020, which is taking place online. In the April 3 Industry Talks Session, Chris Roberts from Cambridge Consultants described the process of working out what sort of robotics or automation made sense for food production.

“Chris described a project with the food service industry to look at using robotics and AI. And of course what he wanted to build was a robot dipping chips and flipping burgers and all the rest, but on evaluation, that was not the best direction to go. In this instance AI optimization of locations was the best option. And this is something I’ve heard from a number of other people who’ve been doing robots for the actual preparation of food itself. For example, Creator, who make those fantastic hamburgers, didn’t have a robot arm, imitating the way humans cook. Instead, they completely re architected the entire process of constructing a burger, so that it could be done mechanically.” [Andra Keay April 7 2020]

There’s much more to hear in our weekly discussion on ‘COVID-19, robots and us’ from April 7 2020 with guest expert speakers: Linda Pouliot, CEO of Dishcraft Robotics Vipin Jain, CEO of Blendid Rich Stump, Principal at Studio Fathom Mark Martin, Industry & Workforce Development California Community Colleges and Bay Area Manufacturing Discussion Forum Ken Goldberg, Director of CITRIS People and Robots Initiative and William S. Lloyd Jr Distinguished Chair in Engineering at UC Berkeley; moderated by Andra Keay, Silicon Valley Robotics.

The Baxter robot hands off a cable to a human collaborator – an example of a co-robot in action. Photo credit: Aaron Bestick, UC Berkeley.

Since 2001, the University of California’s Center for Information Technology Research in the Interest of Society (CITRIS) has leveraged multi-disciplinary research to tackle large societal problems like IT for healthcare, and intelligent infrastructure for public safety, water management and environmental sustainability. Now CITRIS is launching a new initiative that will focus on robotics research that benefits humans and involves humans in the loop.

Sensitivity to human issues is a key aspect of the new robotics initiative, which has been created under the premise that for robotics to reach its potential, humans – and human needs – will have to be integrated into the picture. Ken Goldberg, one of the CITRIS faculty, explained in a phone interview that while there have been many exciting developments in robotics in recent years thanks to the confluence of a thriving research community, new software packages, open source systems, and the availability of low cost components, fearful public sentiment has also been growing in parallel.

“There has been a lot of press about the issue of jobs, loss of privacy, and the idea that as robots advance they will becoming threatening to humans,” he said. “We want to be able to focus on robotics applications that are beneficial to humans, and that have humans uppermost in mind. We firmly believe that humans are essential to the operation of robots – and that robots will be far more effective as tools that enhance humans rather than as something that replaces them.”

The program will draw from the expertise of over 60 faculty from across four UC campuses, covering the fields of control theory, optimization, computer vision, machine learning, statistics, law, philosophy and art. “It’s a very broad group,” said Goldberg. “We can be blinded when we are all thinking along similar lines and we all have similar assumptions and experiences. It’s helpful to bring people together from different backgrounds – that diversity often leads to insight.”

Adding robotics as a focus area may be an important step for CITRIS, but it is not a very large one given the background of some of CITRIS faculty and emeritus directors. Ruzena Bajscy, the founding Director of CITRIS, is known for her research in the fields of robotics, artificial intelligence and machine perception, but her credentials reach into neuroscience, applied mechanics and computer science as well. “She’s an expert in computer vision and perception, and she teaches and continues to do research on human-centered robots, which is one of the themes of this new initiative,” said Goldberg, who had Bajscy as an undergrad advisor.

CITRIS was formed in 2001, when researchers within the UC system realized that the real opportunities lay not just in developing new and innovative technologies, but in applying them. It is one of four California Institutes for Science and Innovation that were created to leverage private and government resources and shorten the path to entrepreneurial investment. The CITRIS plan involves three main pillars: sensitivity to human issues; rigorous theory evaluated on standard benchmarks; and modular systems built on shared software toolkits. Goldberg’s interview (continued below) sheds light on how these pillars fit into the CITRIS model.

Excerpts from Ken Goldberg’s interview:

How does the CITRIS initiative define what is “robotic work” and what is “human work”?

In the past there was this idea that you put the robots in the cage in the assembly line and you had to keep humans away from them because they were doing something dangerous – like spot welding or machining. But with companies like Willow Garage and Baxter and others, we are starting to think about robots differently – as something that can work side by side with humans. So we have to design them differently, we have to think about the issues of safety, privacy and security. We are also looking at the potential that humans could be involved in teaching robots by going through demonstrations and having robots learn by observation. With robots in the home, for example, one project that we’re interested in is decluttering. 

We’ve gotten used to the idea of robots doing dangerous jobs, but in that capacity they are very separate from us. Now that we want collaborative robots – robots that can work alongside humans – we seem to be OK giving them the boring jobs (the jobs that are “robotic”, to use the word as it is currently applied). But by definition those jobs are not creative, communicative or social. If we are working alongside robots, and even teaching them, will we be looking for robots that are able to be more than just “robotic”?  

I think that creativity, and to a large degree, sociability, is something that humans are going to have a great advantage in (as we should), and I hope we continue to have the edge in those areas. But I think we can develop robots that are more responsive. Maybe they won’t be able to express or innovate themselves, but they can start to observe nuances of manipulation by observing and measuring human performance, for example. Essentially, we want them to assist us.

One example would be surgery, where the surgeons are performing a complex operation. Certain aspects of the operation that are monotonous and tedious, but there are also many aspects of the surgery that require extremely specialized skills. So the key is to balance those two things out. You don’t want the robot to do the things that require skill, but you do want it to be able to perform something like suturing, or debridement – tasks that take up valuable time in surgery.

It’s the same analogy with what’s happening with driving. There’s been a lot of talk about the fully autonomous car, but I think what we are going to see is much more practical – the idea of driver assist. For example, when I’m driving to work, there’s a certain period when I’m on the freeway and it doesn’t require as much attention. Or when I’m stuck in traffic, it’s very tedious to be inching along. These things require attention, but really we could have an automated system do that.

One of the key pillars of the CITRIS initiative is modular systems built on shared software toolkits. Given the rapid advances in robotics over the past five years, where is that headed?

The major development has been with ROS – the Robot Operating System – which has been a tremendous benefit for the field. When I was a grad student, and for many years thereafter, when you wanted to build a robot system you started from scratch. Now you can go and get libraries and pull lots of components.

I think what’s exciting is the question of how to take that to the next level … how to have systems that use a software-as-a-service (SaaS) model where the entire software is maintained in the cloud. You could send out data to be processed in the cloud and you would get back results, such as motion plot plans or grasp plans. And just like with Google docs, the benefit is that all the software maintenance, installation, and updates are done in the cloud so that the end user or the robot that is operating in the field doesn’t have to worry about all that, and can essentially just access the latest software.

Consolidating that is a major benefit and would certainly fast track things for the end user. But it can be a battle to find group willing to undertake this kind of consolidation and support that. Are you seeing more interest from large companies in supporting software as a service for robotics?

A number of programs have had company and industry support for open source software at various levels of development. Because we are a public institution, Berkley has always championed is the idea of open source – of putting code out there and making it publicly available. We’d like to continue in that direction by having software that we can support and make available as packages. One thing that industry seems interested in is that, as certain piece of software builds momentum, a startup might form around it to take it to the next level. We want to be open to that and put together software and data resources as part of the new initiative. Pieter Abbeel is going to be leading that aspect.

Many more startups are leveraging ROS now, but at the ultimate commercializing level there is still hesitancy from the larger companies, and sometimes it means that ROS is being used as a development environment rather than being seen as the final endpoint because of issues around maintaining its commercial viability. Is open source more likely to be part of the commercialization process if more public institutions are supporting it?

I think an example of this is the AMP lab at UC Berkley – the Algorithm, Machines and People lab. They have been putting out [[software for systems]] in machine learning and they’ve been extremely successful. The SPARK package has millions of users now and has been incorporated into commercial products, and they’ve created a spinoff – Databricks – that is is taking that to a new level. So I think that a first step is to put out software that people can use. And then entrepreneurs and investors can see that there is a demand for it – that’s what takes it to the next step where it can be commercialized.

This example points out that robotics isn’t developing in isolation and is able to leverage so many of the things that have been able to roll out in software and mobile technologies. This brings me to another area that you’ve outlined for CITRIS: rigorous theory evaluated on standard benchmarks. I think that’s critically important. What’s leading towards this, and how will you follow through?

Let me give you an example. One of my areas of research is grasping, and I feel that this field has never had a very solid benchmark of objects to test grasping on. There are some datasets … for example, the field of machine vision has been doing this very effectively for many years, and there are beautiful datasets for testing machine vision algorithms. But grasping hasn’t had that yet, so this is something we are pursuing. For example, Pieter Abbeel is working with colleagues at CMU and Yale to develop a kit of parts that are common to home and medical environments, and he is going to make this available as a suitcase or box of parts. The goal is to get funding so that he can make it available at very low cost and send it out to research groups around the world so that they could test their algorithms on common parts and compare them. 

We are excited because now we have a critical mass of faculty and students who can use these benchmarks and begin producing their own benchmarks.

Another example is the Amazon Picking Challenge that will be at ICRA this year. Interestingly, one of Pieter’s students did a very detailed scan of each of the objects from the picking challenge, using multiple cameras with 3D sensors on a nice white background, and made them available to the community. That kind of data set is very difficult for any individual lab to produce on their own. So our idea was: we will do that scanning and share it with everybody. And our hope is that it could be used again as a nice benchmark set.

Unfortunately those objects are not particularly interesting from a grasping perspective. In fact we were just lamenting that they are very interesting from a computer vision perspective, but they are all very similar in shape so they don’t really give us a lot to work with in terms of grasp algorithms.

They are being driven by a particular commercial area – logistics and warehousing – aren’t these a set a of the most common objects in that environment?

They are common in that environment, but boxes and books and are pretty rectilinear, and from a grasping perspective, we are interested in things that are a little more challenging.

So in terms of benchmarks you’d be calling for something like the agriculture industry – where the variety of shapes and the delicacy of the objects creates a lot of interesting challenges – to sponsor a grasping challenge? 

Yes, that’s a great example. Or a company like an assembly, automotive or electronics assembly company present us with a really complex set of objects that need to be grasped and manipulated. That would be really interesting. A lot of companies are nervous about doing that because for proprietary reasons they don’t want to share their objects, and so this is where a university lab can help out because we can actually do the dirty work of collecting all that. We’d like to work with industry and we see this as filling a gap that industry hasn’t set itself to do yet.

You have mentioned that when robotics research is confined to very small labs, the resources are simply not there to develop standardized benchmarks and problem sets. How are virtualized environments extending the reach of these benchmarks to greater section of the research community?

There’s a need for standardized simulation environments and we have seen a lot of progress there in the last few years. There is indeed a bigger community of users sharing tools that can start to become more widely accepted. Again, the key advantage is that you can start comparing results across groups and labs. The key is having a diversity of ideas and perspectives brought together. I’m a firm believer that we can be blinded to insights because we are all thinking along similar lines and we all have similar assumptions and experiences. So it’s helpful to bring people together from different backgrounds and that diversity often leads to an insight. We’ve always assumed things are one way, and this encourages to try things another way. That’s the benefit. There are so many researchers working in these areas. To some degree everybody is working in their own labs, but we want people to share, interact and communicate between the labs and students. That’s where I think we have enormous possibility.

“Why We Love Robots” is a 5-minute short documentary film co-directed by UC Berkeley Prof. Ken Goldberg and his wife, award-winning filmmaker Tiffany Shlain. The film was nominated for an Emmy Award and won a “Botscar” (Robot filmOscar) at the 2014 Robot Film Festival, and explores the human fascination with robots and new research trends in the field.

The film combines cultural references and found footage to explore our human fascination with robots and emerging research in Cloud Robotics and Belief Space planning.