What was the topic of your PhD research and why was it an interesting area?

My topic of research was developing an optical tactile sensor to track head motion during radiotherapy. I worked on both the hardware and software development of this sensor, though my focus was mostly on the software side. Its importance comes from the fact that during radiotherapy, patients undergoing head and neck cancer treatment are typically immobilised. This is usually done using a thermoplastic mask, which can feel very claustrophobic, or a stereotactic frame. Frames are more common for brain cancers, but they have to be surgically inserted into the patient’s skull using pins. Either of these immobilisation tools may be used depending on the situation. When patients are uncomfortable, they are more likely to move, which affects the accuracy of treatment, especially with thermoplastic masks.

Another major issue is that current systems use ceiling-mounted cameras to record patient motion. These cameras cannot be placed too close to the patient because of the electromagnetic environment around the equipment. Their view is also frequently occluded because the patient moves into a tunnel to receive the ionising beams, which makes it difficult to capture rotational motion.

One alternative is an infrared camera with a nose marker, but this only captures translational motion. Currently, when a nose tracker detects movement beyond a certain threshold, treatment is paused, the patient is repositioned, and treatment resumes. It is difficult to adapt this system to reliably measure the rotational motion of the patient’s head in the radiotherapy environment.

This is where the Motion Capture Pillow (MCP) comes in, which contains the optical tactile sensor I developed. The goal with this system is similar to the nose tracker, but with more accurate rotational feedback for the radiographer. It can be placed beneath the patient’s head and attached to the treatment bed. It estimates how much the patient’s neck is rotating and improves patient comfort. Radiographers can receive real-time feedback on both translational and rotational movement. The advantages of this system are that there are no occlusions, because the pillow is in direct contact with the patient’s head, and it is more compatible with radiotherapy environments because the sensor is non-ferromagnetic. Its premise is to maintain patient comfort, stay compact and easy to integrate into the pre-existing systems for radiotherapy, whilst improving the accuracy of the treatment through real-time head tracking.

Labelled diagram of the Motion Capture Pillow – Optical tactile sensor for head tracking during radiotherapy. The pneumatic pillow is a deformable rubber-like sheet with embedded white markers, held in its convex shape using air pressure. The fibrescope represents a non-ferromagnetic fibre optic bundle used as a lens extension to an area scan camera. The camera is ferromagnetic and will require safe positioning and fixation.

What were the main contributions of your work?

There were four main contributions to my work. The first contribution focused on making the system more non-ferromagnetic and improving the imaging and tracking approach. Previous work used a webcam and binary image processing within the optical tactile sensor to track marker displacement. I ultimately decided to use a fibrescope, optical flow tracking algorithm, and grayscale imaging instead, which improved the sensor’s tracking ability.

The second contribution focused on optimising marker density. The optical tactile sensor consists of an array of markers on a deformable rubber-like sheet, resembling a pillow. The deformation of these markers is captured by the camera. I investigated how dense the marker array needed to be by adjusting the spacing between markers to determine what worked best for this application.

The third contribution involved sensor fusion to improve reliability and robustness. To do this, I integrated a gyroscope and used Kalman filtering to fuse data from the gyroscope and the MCP. This was important for Gamma Knife systems, which are radiosurgery platforms used for brain cancers. They tend to have higher accuracy requirements than linear accelerators, which are commonly used for head and neck cancers, and lower constraints on the use of ferromagnetic components.

The final contribution was a participatory design study conducted in collaboration with clinicians and the social sciences department. We explored how the MCP could be integrated into hospital workflows and assessed its feasibility.

How feasible is it to integrate this sensor into hospital workflows?

Clinicians did seem to be very on board with it, but the study was more qualitative than quantitative. While they felt the idea had merit, there were reservations about adopting new technology and the associated learning curve.

They were also concerned about accuracy. Improving accuracy and reliability is essential for clinicians to feel confident using the system. At present, further development is needed before it can be widely implemented.

What future work is planned in this area?

One area to investigate is the differences between the mannequin and participant data. The pillow shape is controlled by a pneumatic system with a pressure sensor and air pump. When the patient or mannequin moves, pressure changes occur. The system compensates to maintain a set pressure, but this introduces errors in the motion readings. The mannequin produced more errors than the participant data. It may not accurately simulate human motion on the pillow, and the testing setup may introduce discrepancies that do not reflect real-world behaviour.
So, future work includes stabilising and refining the pressure control system to improve reliability. If necessary, reconsidering the use of gel on the sensors could be an option. Gel had been used previously but was abandoned due to clinician concerns about attenuation of ionising beams. However, if avoiding gel significantly compromises sensor performance, revisiting this approach may be worthwhile.

In addition, more participant data collection is needed. Not all previously collected data could be used due to ground-truth measurements being partially occluded in the experimental setup. Additional participant studies would provide a clearer understanding of performance across different individuals. Another priority is improving the fibrescope’s resolution and angle to better visualise high-density marker arrays. Hardware upgrades would help ensure a clearer field of view and improve overall system performance.

About Bhoomika

Bhoomika Gandhi is a recent PhD graduate from the University of Sheffield Medical Robotics group. Her undergraduate degree was in Bioengineering – Medical Devices and Instruments, with control engineering and robotics being the key themes.

In this episode, Abate flew to Denver, Colorado, to get a behind-the-scenes look at the future of recycling with Joe Castagneri, the head of AI at Amp Robotics. With Materials Recovery Facilities (MRFs) processing a staggering 25 tons of trash per hour, robotic sorting is the clear long-term solution.

Recycling is a for-profit industry. When the margins don’t make sense, the items will not be recycled. This is why Amp’s mission to use robotics and AI to bring down the cost of recycling and increase the number of items that can be sorted for recycling is so impactful.

Joe Castagneri
Joe Castagneri graduated with his Master of Science in Applied Mathematics, with an undergrad degree in Physics. While still in university, he first joined the team at Amp Robotics in 2016 where he worked on Machine Learning models to identify recyclables in video streams of Trash in Materials Recovery Facilities (MRFs). Today, he is the Head of AI at Amp Robotics where he is changing the economics of recycling through automation.

RoboCup is an international scientific initiative with the goal of advancing the state of the art of intelligent robots, AI and automation. The annual RoboCup event returned to an in-person format for 2022, taking place from 13-17 July in Bangkok. RoboCup comprises a number of leagues, with perhaps the most well-known being the soccer leagues.

In the Humanoid League, autonomous robots with a human-inspired body plan and senses play soccer against each other. We spoke to Jasper Güldenstein, a member of the technical committee, about the competition at RoboCup 2022, and also about the Humanoid League Virtual Season. As a biweekly virtual competition taking place between the physical RoboCup events it provides teams with an opportunity to test their ideas and keep in touch throughout the year.

Could you give us an overview of the Humanoid League competition at RoboCup this year?

This year we had the first in-person event after a three year break. It was really good to have the majority of the teams coming back together and working on their robots again. Although a lot of teams came with fewer participants than they usually do, we still managed to have 12 teams in the kid size league and three teams in the adult size. Unfortunately, some teams could not participate due to travel restrictions, but we hope to see them back next year.

Humanoid league finalists with their robots

What was the format for the competition?

At the beginning we had a drop-in round, which is where we play games with one robot from each team participating in a joint team. So, we have eight robots playing from eight different teams. That is useful for finding out which teams are likely to perform the best in the competition. Next, the results from the drop-in games were used as seeding for the round robin phase of the regular competition, with the strongest teams separated into different groups. After the round robin, we have a knockout competition. The seeding method means that we can hopefully avoid the situation where very good teams get kicked out early. We saw that the most interesting games were towards the end of the competition when the teams performed really well.

Have you noticed improvements since the last physical competition?

I’d say definitely that one big thing that has improved for a lot of teams is the localisation. A lot of teams are more easily able to localise themselves on the field, and they don’t run off randomly. They are more certain that they are in the correct position.

Furthermore, I think the kicking has improved. The robots kick the ball much further than they used to. People have been tuning their kicking motions to increase the distance.

In terms of computer vision, this has definitely improved quite a bit. Something we did differently this time, which was inspired by what we did in the virtual season, is that we had a set of six different balls, all from previous FIFA competitions. For each game a ball was drawn randomly, so the teams couldn’t really prepare for all the balls. Although they were visually quite different, the teams didn’t really have any problems detecting the ball. We’ve seen, in general, that computer vision approaches have improved and these improvements have been transferred to the RoboCup competition. I think that almost all teams are using a neural network to detect the ball. This is a change from three, four, five years ago, where many teams used hand-tuned classical computer vision algorithms.

To talk a bit more about ball detection, it will be interesting to see what happens if we move to an environment with natural and/or varying light conditions. This year we were in a convention hall with uniform lighting. I believe next year, in Bordeaux, there is going to be some form of natural light coming in, and perhaps even fields that are outside. It’s still at the planning stage but we are looking forward to that. It will be a challenge and I strongly believe that the participants will find approaches to make their vision approach robust against these varying conditions.

The setup and testing for the Humanoid League competition at RoboCup 2022, Bangkok.

Thinking about the transfer from the simulation to the real world, are there any specific elements that lend themselves well to being transferred?

In terms of computer vision, we had a bit of transfer. In the virtual season we concentrated a lot on changing the lighting conditions and having varied backgrounds, to be able to emulate the real world a bit better. I think a few teams used their vision approaches from the virtual season in the real world.

However, I think the strongest part is behaviour. Teams were able to test their strategies in the virtual competition and adapt every other week. For example, CIT Brains, which won the virtual season and the physical competition, made quite a few changes to their strategy and they had robust systems running. Their strategy worked really well, and in the final they managed to score several goals win against the previous world champions (Rhoban Football Club).

How did the competition go for your team (Hamburg Bit-Bots)?

We actually had quite a few hardware problems, especially on the mechanics side. The motors are wearing out and warped due to wear, and flexed more than we expected. This meant we had difficulties walking stably. And, if you can’t walk in a stable manner that defeats the purpose of everything else. It’s a really integrated system – if one component breaks, you are out of luck as you are very restricted in what you can change during the competition as you don’t have much spare equipment with you.

However, what was good for us was that we had a lot of software up and running, and a lot of it had been tested really well during the virtual league. We had to try and find a way round the walking problem algorithmically, to try to find walking parameters that were more stable. We also switched from [robot operating system] ROS 1 to ROS 2 which brought with it many challenges. We actually did a write up on the issues we faced, with some tricks and tips.

Will there be a new virtual league season this year?

Yes, we’ve discussed this in our technical committee and we plan on doing it again. The last event was successful and the teams enjoyed it. We plan on making some changes, such as logging the games to extract some interesting metrics and doing some analysis on those.

Another thing we want to do is domain randomisation, making the environment a bit more varied. This means that the approaches have to be more robust. The hope is that, when they are more robust, they can be transferred more easily to the real world. We were thinking about making the terrain slightly uneven. Another approach could be to modify the models of the robots such that the joints emulate a bit of wear, so they simulated actuators might be a bit weaker or stronger randomly, and teams have to find robust approaches to deal with that.

We won’t do everything at the beginning. We’ll move through the season and talk to the teams and form some organising groups to develop the simulation further, to run the games and to organise the competition itself. We are always happy to have input and we always talk to the teams to see what they think. It’s a competition but it’s something we build together.

Humanoid robots at RoboCup 2022, Bangkok.

Could you tell us about the satellite event that took place at RoboCup?

This was a discussion about how to get teams more interested in participating and how to bridge the junior leagues and the major leagues.

We know that some people who participated in RoboCup Junior specifically selected a University that has a RoboCup team so that they could join that team. It would be awesome for more people to do this, and for more juniors to know what the major league is about.

To bridge the gap between the junior and major leagues we don’t want to introduce another league, but we want some form of events where the two groups can meet, and where the juniors can show off how well they are doing. It would be good to have more interaction between the leagues, although we haven’t decided on the exact format yet.

About Jasper

Jasper Güldenstein is a PhD student at University of Hamburg. His research focuses on humanoid navigation. He has been participating in RoboCup as a member of the Hamburg Bit-Bots since 2016 where his focus is developing not only the software of the team but also the electronics of the robot platform. In his recent master thesis he evaluated using reinforcement learning to perform path planning and execution for humanoid robots.

The team at Retrocausal built a computer vision platform that allows any manufacturer to rapidly set up an activity recognition pipeline to assist with manual assemblies. Using any standard camera pointed at an assembly station, the system can learn the assembly procedure and give real-time feedback.

Deci:

At Deci, they are tackling the problem of taking a Machine Learning model optimized for a single hardware platform and adapting it to work on a different hardware platform.

When developing an ML model running on the edge, a time-consuming development cycle is made to optimize the model to run at peak capacity. However, over time hardware platforms get updated and companies need to make the decision of whether or not to update their hardware and dedicate engineering resources to manually tweak a multitude of settings.

Deci allows companies to upload their models to the cloud, then automatically optimize them for running on a variety of different hardware platforms that they own in their facilities.

StegAI:

Dr. Eric Wengrowski co-founded StegAI after publishing a paper in CVPR 2019 on embedding “fingerprints” that are embedded in images and videos that are invisible to the human eye, but detectable by their software.

Their product modulates the pixels in an image in such a way that even after compressing, resizing, or printing out the images, their deep learning algorithms will still be able to detect the origin of the image.

This powerful tool is used by social media firms to protect against potential copyright infringement when users upload content to their platforms

At ICRA 2022, the researchers weren’t the only ones working with cutting-edge technology. We spoke to the exhibitors to get real-life demos of their products.

Tangram Vision

Tangram Vision is a hardware-agnostic sensor fusion platform. It streamlines the development and deployment of critical sensor infrastructure like calibration, fusion, and monitoring for any number of cameras, depth sensors, LiDAR, radar, and IMU. Their co-founder, Adam Rodnitzky, walks us through their sensor fusion platform.

FLX Solutions

Matt Bilsky, Founder and CEO of FLX Solutions gives us a live demo of their robot, the FLX BOT. Matt Bilsky applied his Ph.D. in Mechanical Engineering to create a novel, highly compact robot that is designed to reach and inspect parts of a building that a human cannot reach. The FLX BOT is one inch in diameter and is made of modular links that can be attached one after the other to extend the reach and degrees of freedom of the robot.

Exyn Technologies

Exyn Technologies specializes in aerial robotics, drone swarms, multi-modal sensor fusion, 3D mapping, obstacle avoidance, and autonomous navigation & planning.

Exyn’s focus is on developing software for aerial robots so they can operate in GPS-denied environments, without human control, prior information, or pre-existing infrastructure (e.g. no motion capture system).

Pollen Robotics

Pollen Robotics developed the robot, Reachy, an open-source humanoid robot with a quirky appearance. Reachy’s primary userbase is researchers studying fields such as teleoperation. Using mixed autonomy, Reachy is able to be teleoperated by humans, while still having onboard intelligence to autonomously infer the actions you want it to do.

Diligent Robotics, founded by Andrea Thomaz and Vivian Chu, develops socially intelligent automation solutions for hospitals. Moxi, their flagship robot, delivers items like medications and wound dressings between departments to save the clinical staff’s time. Diligent has just closed their Series B funding round with $30 million. 

We sat down with Dr. Thomaz to talk about Moxi, how to manage people’s expectations about robots, and advice for young people and women in robotics. This interview has been lightly edited for clarity. 

Andrea Thomaz with Diligent’s flagship robot Moxi.

What kinds of problems are you trying to solve with Moxi?

We are building Moxi to help hospitals with the massive workforce shortage that they’re seeing now more than ever. We actually started the company with the same intention several years ago before there was a worldwide pandemic, and it really has just gotten to be an even bigger problem for hospitals. I feel really strongly that robots have a place to play in teamwork environments, and hospitals are a great example of that. There’s no one person’s job in a hospital that you would actually want to give over to automation or robots, but there are tiny little bits of a lot of people’s jobs that are absolutely able to be automated and that we can give over to delivery robots in particular like Moxi. [The main problem we’re trying to solve with Moxi is] point to point delivery, where we’re fetching and gathering things and taking them from one area of the hospital to another. 

Hospitals have a lot of stuff that’s moving around every day. Every person in the hospital is going to have certain medications that need to be delivered to them, certain lab samples that need to be taken and delivered to the central lab, certain supplies that need to come up to them, food and nutrition every day. You have a lot of stuff that’s coming and going between patient units and all these different support departments. 

Every one of these support departments has a process in place for getting the stuff moved around, but no matter what, there’s stuff that happens every single day that requires ad-hoc [deliveries] to happen between all of these departments and different nursing units. So sometimes that’s going to be a nurse that just needs to get something for their patient and they want that to happen as soon as possible. They’re trying to discharge their patient, they need a particular wound dressing kit, they’re going to run down and get it because they want to help their patient get out. Or if there’s something that needs to be hand carried because the regular rounding of medications has already happened, a lot of times you’ll have a pharmacy technician stop what they’re doing and go and run some infusion meds for a cancer patient, for example. It sort of falls between these departments. There’s different people that would be involved but a lot of times it does fall on the nursing units themselves. A nurse explained to us one time that nurses are the last line of defense in patient care.

Moxi performing a delivery for a clinical staff member.

What is changing with this most recent round of funding?

Over the last 6-12 months, the demand has really skyrocketed such that we’re barely keeping up with the demand for people wanting to implement robots in their hospitals. That’s the reason why we’re raising this round of funding, expanding the team, and expanding our ability to capitalize on that demand. A couple of years ago, if we were working with a hospital it was because they had some special funds set aside for innovation or they had a CTO or a CIO that had a background in robotics, but it certainly wasn’t the first thing that every hospital CIO was thinking about. Now that has completely changed. We’re getting cold outreach on our website from CIOs of hospitals saying “I need to develop a robotic strategy for our hospital and I want to learn about your solution.” Through the pandemic, I think everyone has seen that the workforce shortage in hospitals is only getting worse in the near term. Everybody wants to plan for the future and do everything they can to take small tasks off of the plates of their clinical teams. It’s been really exciting to be part of that market change and see that shift to where everybody is really really open to automation. Before we had to say “No no no, this is not the future, I promise it’s not scifi, I promise these really work.” Now [the climate has] really shifted to people understanding “This is actually something that can impact my teams.”

[Two of our investors are hospitals, and] that’s been one of our most exciting parts of this round. It’s always great to have a successful funding round, but to have strategic partners like Cedars-Sinai and Shannon Healthcare coming in and saying “Yeah, we actually want to build this alongside you” — it’s pretty exciting to have customers like that. 

What kinds of technical problems did you run into when you were either building Moxi or deploying it in a hospital environment? How did you solve those problems? 

One that was almost surprising in how often it came up, and really impacted our ability [to run Moxi in the hospital environment] because we have a software-based robotic solution that is connecting at a regular basis to cloud services, [was that] we had no clue how terrible hospital WiFi was going to be. We actually spent quite a while building in backup systems to be able to use WiFi, backup to LTE if we have to, but be smart about that so we’re not spending a whole bunch of money on LTE data. That was a problem that seemed very specific to hospitals in particular.

Another one was security and compliance. We just didn’t know what some of the different requirements were for hospitals until we actually got into the environments and started interacting with customers and understanding what they wanted to use Moxi for. When we were first doing research trials in 2018 or 2019, we had a version of the robot that was a little bit different than the one we have today. It had lots of open containers so you could just put whatever you wanted to on the robot and send it over to another location. We quickly learned that that limited what the robot was allowed to carry, because so much of what [the customers] wanted was to understand who pulled something out of the robot. So now we have an RF badge reader on the robot that is connected to locking storage containers that are only going to open if you’re the kind of person that is allowed to open the robot. That was an interesting technical challenge that we didn’t know about until after we got out there. 

Moxi’s locking storage containers.

How did you work with nurses and the other healthcare professionals you were working with to figure out what would be the most helpful robot for them? 

My background, and my co-founder Vivian Chu’s background, is in human-robot interaction so we knew that we didn’t know enough about nursing or the hospital environment. We spent the first 9 months of the company in 2018 building out our research prototype. It looked a lot like what Moxi looks like today. Under the hood it was completely different than what we have today in terms of the reliability and robustness of the hardware and software, but it was enough to get that platform out and have it deployed with nursing units. We embedded ourselves with four different nursing units across Texas over a year-long period. We would spend about 6-8 weeks with a nursing department, and we were just there — engineers, product people, and everybody in the company was cycling in and out a week or two at a time. 

We would ask those nurses: “What would you actually want a robot like this to do?” Part of this that was really important was they didn’t have good ideas about what they would want the robot to do until they saw the robot. It was a very participatory design, where they had to see and get a sense for the robot before they would have good ideas of what they would want the robot to do. Then we would take those ideas [to the company] and come back and say “Yes we can do that,” or “No we can’t do that.” We came out of that whole process with a really great idea. We like to say that’s where we found our product market fit — that’s where we really understood that what was going to be most valuable for the robot to do was connecting the nursing units to these other departments. We can help a nurse with supply management and getting things from place to place within their department, or we can help them with things that are coming from really far away. [The second one] was actually impacting their time way way more.

Because the capabilities of robotic systems are usually misinterpreted, it can be really hard to manage the relationship with stakeholders and customers and set appropriate expectations. How did you manage that relationship?

We do a lot of demonstrations, but still with almost every single implementation you get questions about some robot in Hollywood, [and you have to say] “No, that’s the movies” and explain exactly what Moxi does. 

From a design perspective, we also limit the English phrases that come out of Moxi’s mouth just because we don’t want to communicate a really high level of intelligence. There are lots of canned phrases and interactions on the iPad instead of via voice, and a lot of times the robot will just make meeps and beeps and flash lights and things like that. 

Before starting the company, I had a lab, and one of the big research topics that we had for a number of years was embodied dialogue — how robots could have a real conversation with people. I had a very good appreciation for how hard that problem is, and also for just how much people want it. People come up to a robot, and they want it to be able to talk to them. How you can [set expectations] with the design and behavior of the robot has been a focus of mine since before we started the company. We purposefully don’t make the robot look very human-like because we don’t want there to be android human-level expectations, but [the robot does have a face and eyes so it can] communicate “I’m looking at that thing” and “I’m about to manipulate that thing,” which we think is important. It’s really about striking that balance. 

What would you say is one lesson that you’ve learned from your work at Diligent so far and how are you looking to apply this lesson moving forward?

The difference between research and practice. On the one hand, the motivation and reason for starting a company is that you want to see the kinds of things that you’ve done in the research lab really make it out into the world and start to impact real people and their work. That’s been one of the most fascinating, impactful, and inspiring things about starting Diligent: Being able to go and see nurses when Moxi is doing work for them. They are so thankful! If you just hang back and watch Moxi come and do a delivery, almost always people are super excited to see the robot. They get their delivery and they’re like, “Oh, thank you Moxi!” That feels like we’re really making a difference in a way that you just don’t get with just research contributions that don’t make it all the way out into the world. 

That being said though, there is a long tail of things that you have to solve from an engineering perspective beyond [developing a feature]. My VP of engineering Starr Corbin has this great way of putting it: The research team will get a certain thing on the product to be feature complete, where we’ve demonstrated that this feature works and it’s a good solution, but then there’s this whole phase that has to happen after that to get the feature to be production ready. I would say my biggest lesson is probably everything that it takes, and the entire team of people it takes, to get something from being feature complete to production ready. I have a deep appreciation for that. How fast we can move things out into the world is really dictated by some of that. 

Andrea Thomaz (left) and Vivian Chu with Moxi.

What advice would you give young women in robotics? 

If I put my professor hat on, I always had advice that I liked to give women in robotics, in academia, and just kind of pursuing things in general. Imposter syndrome is real, and everybody feels it. All you can do to combat it is not underestimate yourself. Speak up and know that you deserve a seat at the table. It’s all about hard work, but also making sure that your voice is heard. Some of the mentorship that I gave to a lot of my women grad students when I was a professor was around speaking engagements, speaking styles, and communication. It can be really uncomfortable when you’re the only anything in the room to stand up and feel like you deserve to be the one speaking, and so the more that you practice doing that, the more comfortable it can feel, the more confident you’ll feel in yourself and your voice. I think finding that confident voice is a really important skill that you have to develop early on in your career. 

What’s one piece of advice you’ve received that you always turn to when things are tough? 

There are two mentors that I’ve had who are women in AI and robotics. [In my] first year as a faculty member [the first mentor] came and gave a research seminar talk. I for some reason got to take her out to lunch by myself, so we had this amazing one-on-one. We talked a little bit about her talk, probably half of the lunch we talked about technical things, and then she just kind of turned the conversation [around] and said “Andrea, don’t forget to have a family.” Like, don’t forget to focus on that part of your life — it’s the most important thing. She got on a soapbox and said “You have to have a work life balance it’s so important. Don’t forget to focus on building a family for yourself, whatever that looks like.” That really stuck with me, especially as [when you’re] early in your career you’re worried about nothing but success. It was really powerful to have somebody strong and influential like that telling you “No, no, this is important and you need to focus on this.” 

The other person that’s always been an inspiration and mentor for me that I’ll highlight [was the professor teaching a class I TA’d for at MIT]. I had found a bug in one of her homework problems, and she was like, “Oh, fascinating.” She was so excited that I had found a question that she didn’t know the answer to. She [just said], “Oh my gosh I don’t know, let’s go find out!” I remember her being this great professor at MIT, and she was excited to find something that she didn’t know and go and learn about it together as opposed to being embarrassed that she didn’t know something. I learned a lot from that interaction: That it’s fun to not know something because then you get to go and find the answer, and no matter who you are, you’re never expected to know everything.

Kate speaks with Brad Bogolea, CEO and Co-founder of Simbe Robotics. Simbe Robotics developed a mobile robot named Tally, which is bringing advanced shelf insights to improve the retail shopping experience.

Tally provides a state-of-the-art sensing system on a robust, scalable platform that collects analytics in real-time.

Brad Bogolea

Brad Bogolea is the CEO and Co-Founder of Simbe Robotics, where he is responsible for the company’s vision and execution of its leading retail intelligence solution. In November 2015, Brad brought to market the Tally robot, the world’s first autonomous shelf auditing and analytics solution to help retailers ensure merchandise is always stocked, in the right place, and correctly priced. The National Retail Federation Foundation has named Brad to its list of “People Shaping Retail’s Future.

Prior to Simbe, Brad spent 10 years in the energy and wireless sensor industry. Most recently Brad worked at Silver Spring Networks, where he led global product management and business development efforts for their energy management and data analytics platform for energy utilities. Products under Brad’s leadership allowed the world’s largest utility companies and their energy consumers alike to gain efficiency and visibility of their energy usage through smart sensors and data.

Brad holds a B.S. in Computer Science and Engineering from Pennsylvania State University where he also returned to serve as an Entrepreneur-in-Residence.

Links

• Simbe Robotics
• Download mp3 (34.8 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

Associate professor Vivienne Sze is bringing artificial intelligence applications to smartphones and tiny robots by co-designing energy-efficient hardware and software. Image credits: Lillie Paquette, MIT School of Engineering

Not so long ago, watching a movie on a smartphone seemed impossible. Vivienne Sze was a graduate student at MIT at the time, in the mid 2000s, and she was drawn to the challenge of compressing video to keep image quality high without draining the phone’s battery. The solution she hit upon called for co-designing energy-efficient circuits with energy-efficient algorithms.

Sze would go on to be part of the team that won an Engineering Emmy Award for developing the video compression standards still in use today. Now an associate professor in MIT’s Department of Electrical Engineering and Computer Science, Sze has set her sights on a new milestone: bringing artificial intelligence applications to smartphones and tiny robots.

Her research focuses on designing more-efficient deep neural networks to process video, and more-efficient hardware to run those applications. She recently co-published a book on the topic, and will teach a professional education course on how to design efficient deep learning systems in June.

On April 29, Sze will join Assistant Professor Song Han for an MIT Quest AI Roundtable on the co-design of efficient hardware and software moderated by Aude Oliva, director of MIT Quest Corporate and the MIT director of the MIT-IBM Watson AI Lab. Here, Sze discusses her recent work.

Q: Why do we need low-power AI now?

A: AI applications are moving to smartphones, tiny robots, and internet-connected appliances and other devices with limited power and processing capabilities. The challenge is that AI has high computing requirements. Analyzing sensor and camera data from a self-driving car can consume about 2,500 watts, but the computing budget of a smartphone is just about a single watt. Closing this gap requires rethinking the entire stack, a trend that will define the next decade of AI.

Q: What’s the big deal about running AI on a smartphone?

A: It means that the data processing no longer has to take place in the “cloud,” on racks of warehouse servers. Untethering compute from the cloud allows us to broaden AI’s reach. It gives people in developing countries with limited communication infrastructure access to AI. It also speeds up response time by reducing the lag caused by communicating with distant servers. This is crucial for interactive applications like autonomous navigation and augmented reality, which need to respond instantaneously to changing conditions. Processing data on the device can also protect medical and other sensitive records. Data can be processed right where they’re collected.

Q: What makes modern AI so inefficient?

A: The cornerstone of modern AI — deep neural networks — can require hundreds of millions to billions of calculations — orders of magnitude greater than compressing video on a smartphone. But it’s not just number crunching that makes deep networks energy-intensive — it’s the cost of shuffling data to and from memory to perform these computations. The farther the data have to travel, and the more data there are, the greater the bottleneck.

Q: How are you redesigning AI hardware for greater energy efficiency?

A: We focus on reducing data movement and the amount of data needed for computation. In some deep networks, the same data are used multiple times for different computations. We design specialized hardware to reuse data locally rather than send them off-chip. Storing reused data on-chip makes the process extremely energy-efficient.  

We also optimize the order in which data are processed to maximize their reuse. That’s the key property of the Eyeriss chip that was developed in collaboration with Joel Emer. In our followup work, Eyeriss v2, we made the chip flexible enough to reuse data across a wider range of deep networks. The Eyeriss chip also uses compression to reduce data movement, a common tactic among AI chips. The low-power Navion chip that was developed in collaboration with Sertac Karaman for mapping and navigation applications in robotics uses two to three orders of magnitude less energy than a CPU, in part by using optimizations that reduce the amount of data processed and stored on-chip. 

Q: What changes have you made on the software side to boost efficiency?

A: The more that software aligns with hardware-related performance metrics like energy efficiency, the better we can do. Pruning, for example, is a popular way to remove weights from a deep network to reduce computation costs. But rather than remove weights based on their magnitude, our work on energy-aware pruning suggests you can remove the more energy-intensive weights to improve overall energy consumption. Another method we’ve developed, NetAdapt, automates the process of adapting and optimizing a deep network for a smartphone or other hardware platforms. Our recent followup work, NetAdaptv2, accelerates the optimization process to further boost efficiency.

Q: What low-power AI applications are you working on?

A: I’m exploring autonomous navigation for low-energy robots with Sertac Karaman. I’m also working with Thomas Heldt to develop a low-cost and potentially more effective way of diagnosing and monitoring people with neurodegenerative disorders like Alzheimer’s and Parkinson’s by tracking their eye movements. Eye-movement properties like reaction time could potentially serve as biomarkers for brain function. In the past, eye-movement tracking took place in clinics because of the expensive equipment required. We’ve shown that an ordinary smartphone camera can take measurements from a patient’s home, making data collection easier and less costly. This could help to monitor disease progression and track improvements in clinical drug trials.

Q: Where is low-power AI headed next?

A: Reducing AI’s energy requirements will extend AI to a wider range of embedded devices, extending its reach into tiny robots, smart homes, and medical devices. A key challenge is that efficiency often requires a tradeoff in performance. For wide adoption, it will be important to dig deeper into these different applications to establish the right balance between efficiency and accuracy.

Kate speaks with Anni Kern, Head of Communication, strategy, and teams at Cybathlon for over four years. She describes the motivation and concepts for the Cybathlon organizations to develop a common platform to remove barriers between people with disabilities, technology developers, and the public. Anni also describes the specifics of Cybathlon competitions and the organization and planning.

Anni Kern

Anni Kern, CYBATHLON Head of Communication, Strategy, Teams and CYBATHLON @school. ETH Zurich is the inventor of the CYBATHLON project. The CYBATHLON offers a platform to advance research in the field of assistive technology and to promote dialogue with the public about the inclusion of people with disabilities in everyday life. The core of CYBATHLON of ETH Zurich is a unique competition in which people with physical disabilities compete against each other to complete everyday tasks using state-of-the-art technical assistance systems. At the CYBATHLON 2016, Anni Kern was responsible for the coordination and the support of the participating international CYBATHLON teams. Since 2017 she is among others responsible for the strategy, communication, conception of the live event as well as the CYBATHLON @school programme.

Links

• Download mp3 (11.8MB)
• Youtube channel
• Twitter: @cybathlon
• Facebook: @cybathlon 
• Insta: cybathlon_eth_zurich
• Linkedin: cybathlon

Winning team pilot Sander Koomen

In continuation to this series of CYBATHLON 2020 winners, today we feature the victory of PULSE Racing from VU University Amsterdam. We also had the chance to interview them (see the end of this post).

In this race, pilots with paraplegia from nine teams competed against each other using a recumbent bicycle that they could pedal with the help of functional electrical stimulation (FES) of their leg muscles. As the organizers of CYBATHLON describe, “FES is a technique that allows paralysed muscles to move again. By placing electrodes on the skin or implanting them, currents are applied to the muscles, making them contract. Thus, a person whose nerves from the brain to the leg muscles are disconnected due to a spinal cord injury (SCI) can use an intelligent control device to initiate a movement, e.g. stepping on a bike pedal. New types of electrodes and an exact control of the currents make it possible to maximise the pedal force with each rotation while avoiding early muscle fatigue.” Pilots had 8 minutes to complete 1200m on their static recumbent bike.

Functional electrical stimulation bike race illustration

Five out of the nine teams finished the 1200m race. Therefore, the podium was decided based on finishing time, with pilot Sander Koomen from the winning team PULSE Racing (The Netherlands) completing the distance in 2 minutes and 40 seconds (that’s an average speed of 27km/h). The silver medal went to team ImperialBerkel (UK & The Netherlands) with pilot Johnny Beer, who made a time of 2 minutes and 59 seconds. Finally, team Cleveland (US) with pilot Mark Steven Muhn won the bronze medal with a finishing time of 3 minutes and 13 seconds. You can watch a summary of the top 4 races in the video below.

You can also see the results from the rest of the teams in this discipline here, or watch the recorded livestreams of both days on their website.

Interview to PULSE Racing

Team PULSE Racing

D. C. Z.: What does it mean for your team to have won in your CYBATHLON category?

P.R.: We saw this as a good reflection of where we stand as a team. The result was unexpected, because sometimes it is hard to see results during training. By winning the Cybathlon, our uncertainties about our developments vanished. The golden medal emphasizes our strength and motivation as a student team. Our dream, winning the Cybathlon on the first attempt, came true. We are thankful that the Cybathlon gave us the opportunity to participate, which helped us to gain publicity.

D. C. Z.: And what does it mean for people with disabilities?

P.R.: It means that with joining our team, people with a disability are able to keep developing their bodies and mind, according to a training scheme. This improves mental well-being and health. Besides that they can still be part of the society, and become more confident.

D. C. Z.: What are still your challenges?

P.R.: In the coming years we want to focus more on the mechanics of the bike, to see if we can make some improvements. And it would be great if this form of exercise is accessible for more people with a spinal cord injury. For that it is important that more people are aware of the possibilities of Functional Electrostimulation (FES).

You can follow team updates on Instragram (#pulse.racingnl) and Facebook/LinkedIN (#pulseracing).

Winning team: HSR Enhanced with pilot Florian Hauser

In continuation to this series of CYBATHLON 2020 winners, today we feature the victory of the HSR Enhanced team from the Eastern Switzerland University of Applied Sciences (OST). In addition, we interviewed their team leader of this year, Christian Bermes.

In this race, pilots with a severe walking disability from seven teams competed against each other in a motorized wheelchair. As the organizers of CYBATHLON describe, “motorized wheelchairs can make everyday life much easier for people with a walking disability. The important thing is that they can overcome obstacles such as ramps and yet are not too large to drive under a normal table. Motorized wheelchairs that are controlled by joystick, tongue control, touchpad or other technologies are eligible for this race and are characterized by innovative approaches to overcome obstacles such as stairs”. The challenge for the pilots was to complete the following tasks:

(1) Driving up to a table until half of the thighs were covered without displacing the table to test the size and seat height of the wheelchair.

(2) Driving through furniture without displacing it to test the size of the wheelchair and precise maneuverability.

(3) Crossing uneven terrain to test the grip of the wheels, ground clearance and power.

(4) Ascending and descending stairs, and bringing the wheelchair to a standstill while descending to test the ability to climb and descend stairs in a controlled manner, and power.

(5) Driving across tilted path with different surfaces to test the drifting and tipping stability, and power.

(6) Driving a ramp up and down, opening and closing the door in the middle using an externally powered technical support (e.g. a robotic arm) to test precise maneuverability and control of technical support in a confined space.

Powered wheelchair race tasks

This year, team HSR Enhanced with pilot Florian Hauser was unbeatable again, as it happened in the 2016 edition. The silver medal went to team Caterwil from Russia with pilot Iurii Larin. The third finalist was team Fortississimo from Japan with pilot Hiroshi Nojima. Here’s a summary of the races of the top 4 finalists:

You can see the results from the rest of the teams in this discipline here, or watch the recorded livestreams of both days on their website.

Interview to Christian Bermes – (ex) Team leader of HSR Enhanced team

We had the pleasure to interview Christian Bermes, team leader of the HSR Enhanced team in both 2016 and 2020 editions. After CYBATHLON 2020, he handed over the team leads, as he moved from OST to be Professor for Mobile Robotics at the University of Applied Sciences of the Grisons.

Christian Bermes – (ex) Team leader of HSR Enhanced team

D. C. Z.: What does it mean for your team to have won in your CYBATHLON category?

C.B.: It is a huge confirmation that our first win in 2016 was not just a coïncidence, but again the result of human-centered innovation together with our pilot Florian Hauser, meticulous engineering, proper prior planning, hard training and of course a next-level pilot performance. Needless to say that there was a certain amount of luck involved, too – and hard work puts you where luck finds you.

D. C. Z.: And what does it mean for people with disabilities?

C.B.: I find it hard to answer this in general terms. Our wheelchair as it is right now will not enter a mass market, however some of its modules could, if we found partners for industrialization. I just think it’s great that people with disabilities are the figureheads and heroes of CYBATHLON. They have prepared themselves in the most professional way, have unmatched control over their machines and are simply impressive.

D. C. Z.: What are still your challenges?

C.B.: Right now we enjoy the weightlessness of the win. Soon after, CYBATHLON will publish the new race obstaces for 2024 and I am 100% sure that there will be many technical challenges right away. Moreover, budget has to be secured, sponsors must be found, the team must be sworn in, plus many things more. The goal for 2024 is clear – win another title. I have full confidence in the new lead crew and their team, they will outperform everything we have seen from HSR enhanced until this day. And with Florian Hauser as pilot, we will see lightning speed on the race day.

The story of pilot Florian Hauser

The organizers released a series of videos telling the personal story of some of the competing pilots. The pilot of the HSR Enhanced team, Florian Hauser, “is a tetraplegic since he had a motorcycle accident in 2014. However, this does not prevent him from riding fast. Not on a bike anymore, but in his wheelchair,” as the organizers describe. Apart from being the winner of his discipline in this 2020 edition, Florian also won CYBATHLON 2016 and the CYBATHLON wheelchair Series in Japan.

Winning team: Angel Robotics with pilot Byeong-Uk Kim

The last edition of CYBATHLON took place on 13-14 November, 2020. This competition, created by ETH Zurich and run as a non-profit project, aims to advance in the research and development of assistive technology by involving developers, people with disabilities, and the general public. We had the chance to interview the winning team of the powered exoskeleton race, Angel Robotics from South Korea.

In this race, pilots with complete thoracic or lumbar spinal cord injury from nine teams competed using an exoskeleton. This wearable, powered support enables them to walk and master other everyday tasks. Indeed, the motivation behind this race is that “the use of exoskeletons is still rare, they are currently mainly used for physiotherapy in hospitals and rehabilitation centers. Exoskeletons dramatically increase the mobility of people with paraplegia, which consequently improves their overall physical and psychological health and therefore might represent a welcome addition to a wheelchair”, as the organizers of CYBATHLON state. This race involved:

(1) Sitting down & standing up from a sofa, and stacking cups while standing next to a table to test the range of motion and strength in the knee and hip joints, and stability.

(2) Slaloming around furniture without displacing it to test precision of steps and agility.

(3) Crossing uneven terrain to test precision of steps and adaptation of step lengths and widths.

(4) Climbing and descending stairs to test range of motion and strength in the knee and hip joints, and step precision.

(5) Walking across a tilted path to test the lateral range of motion in hip and foot joints, and stability.

(6) Climbing a ramp, opening and closing the door in the middle of the ramp, and descending the ramp to test the range of motion in foot, knee and hip joints, stability and maneuvering in confined spaces.

Powered exoskeleton race tasks. Credit: CYBATHLON

The top three teams were the company Angel Robotics (1) from South Korea with pilot Byeong-Uk Kim, TWIICE from EPFL research group REHAssist with pilot Silke Pan, and Angel Robotics (2) with pilot Lee Joo-Hyun. Remarkably, the three of them achieved the highest score – 100 points. With this impressive result, the podium was decided based on finishing time. If you can’t wait to watch how tight the races were, you can enjoy them in the summary video below.

You can see the results from the rest of the teams in this discipline here, or watch the recorded livestreams of both days on their website.

Interview to Kyoungchul Kong – Team leader of Angel Robotics team

We had the pleasure to interview Kyungchul Kong, team leader and CEO of Angel Robotics (1&2). He is also an Associate Professor of KAIST (Korea Advanced Institute of Science and Technology).

Kyoungchul Kong – Team leader of Angel Robotics team

D. C. Z.: What does it mean for your team to have won in your CYBATHLON category?

K.K.: In WalkON Suit, the powered exoskeleton of Angel Robotics, there have been various dramatic technical advances. Since the first Cybathlon in 2016, the walking speed has become as fast as people without disabilities. The most important feature of WalkON Suit is its balance; as the center of mass is placed on the area of feet while standing straight, the wearer can stand without much effort for a long time. These superior functionalities of WalkON Suit could be proved by winning the Gold and Bronze medals at Cybathlon 2020.

D. C. Z.: And what does it mean for people with disabilities?

K.K.: While winning the Gold medal is glorious, winning two medals is especially meaningful. The physical conditions of the two pilots (i.e., the Gold medalist and the Bronze medalist) of Team Angel Robotics were extremely different. One was a male with very strong upper body, while the other was a female with much less muscles. Such different people could be successfully assisted by WalkON Suit, which means that the powered exoskeleton is not a technology optimized for a single user, but able to be utilized by many people with different body conditions.

D. C. Z.: What are still your challenges?

K.K.: In order to bring the WalkON Suit into the real life of people who need this technology, it has to be much improved in terms of wearability, price, and weight. The user should be able to wear the robot without anyone else’s help. It should be light enough to handle while sitting on a wheelchair. The price is another critical issue considering practical conditions. With these restrictions, the functionalities and performance of the robot must not be deteriorated. These are the challenges we are much trying to get over.

In this episode, we hear from Luca Colasanto, Senior Robotic Scientist at Realtime Robotics, about real-time robot motion planning in dynamic and complex environments with human-robot collaboration. Realtime Robotics focuses on accelerating conventional motion planning through optimization of algorithms and hardware to allow safe use of robotic tools in work areas with humans. Luca spoke to our interviewer Kate about Realtime Robotic’s fast motion planning technology, including key aspects, such as perception, algorithms and custom hardware.

Luca Colasanto

Luca Colasanto is a Sr. Scientist at Realtime Robotics focusing on AI-based grasping and multi-robot optimization. Luca completed his PhD in Humanoid Robotics at Italian Institute of Technology, focusing on control systems for bipedal walking machines and compliant actuators. During the past ten years, Luca has worked with successful startups to design and prototype new robotics products, and with science and technology leaders, such as Robert Bosch LLC (CA), Institute for Human and Machine Cognition (FL), and École Polytechnique Fédérale de Lausanne (Switzerland), He is the listed inventor in two patent applications and the author of several scientific publications. Luca speaks four languages and is a keen traveler, having visited or lived in more than 20 countries.

Links

• Download mp3 (15.1 MB)
• Realtime Robotics (Videos, Twitter, LinkedIn)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Audrow Nash speaks with Dave Coleman, CEO of PickNik Robotics, about the open source robotics manipulation platform called MoveIt. Coleman talks about MoveIt’s story, from inception and the early days to development and maintenance, as well as how MoveIt relates to the Robot Operating System (ROS) and their move to support ROS-2. He also speaks about MoveIt’s implementation, including global versus local planners and what that means. Coleman concludes by talking about World MoveIt Day and how those interested can begin learning MoveIt and contributing.

Here is a 2017 montage of how MoveIt has been used:

Dave Coleman

Dave Coleman completed his PhD in Computer Science at CU Boulder focusing on motion planning and his B.S. at Geogia Tech in Mechanical Engineering. Coleman has 12 years experience working in the field of robotics automation and is a leader in the open source MoveIt and ROS communities. His insights into robot-agnostic platforms for different morphologies, theoretical approaches, and different end-user requirements give him a well-rounded understanding for powerful robotic software. He has worked and consulted for all types of robotics companies including Google Robotics, Open Robotics, and Willow Garage.

Links

• Download mp3 (17.8 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Ron Vanderkley interviews Mark Pivac, Chief Technical Officer and co-founder of FBR (formerly Fastbrick Robotics) about the world’s first end-to-end autonomous bricklaying robot, ‘Hadrian X’. Three years after his first interview, we catch up with Pivac to see how FBR has expanded its operation and chat about their latest commercial prototype, ‘Hadrian X’, as well as the future of the robotic construction industry.

Mark Pivac is the primary inventor of FBR’s automated bricklaying technology. He is an aeronautical and mechanical engineer with over 25 years’ experience working on the development of high technology equipment ranging from lightweight aircraft to heavy off-road equipment. He has 20 years’ experience working with pro/engineer 3D CAD software as well as high-level mathematics, including matrix mathematics, robot transformations and vector mathematics for machine motion. Mark has also worked extensively with design, commissioning and fault finding on servo controlled motion systems achieving very high dynamic performance.

Links

• FBR Website
• Marc Pivac Profile Page
• Hadrian X in action (Video)
• Download mp3 (13.3 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this interview, Audrow Nash interviews Marco Hutter, Assistant Professor for Robotic Systems at ETH Zürich, about a quadrupedal robot designed for autonomous operation in challenging environments, called ANYmal. Hutter discusses ANYmal’s design, the ARGOS oil and gas rig inspection challenge, and the advantages and complexities of quadrupedal locomotion. 

Here is a video showing some of the highlights of ANYmal at the ARGOS Challenge.

Here is a video that shows some of the motions ANYmal is capable of.

Marco Hutter

Marco Hutter is assistant professor for Robotic Systems at ETH Zürich since 2015 and Branco Weiss Fellow since 2014. Before this, he was deputy director and group leader in the field of legged robotics at the Autonomous Systems Lab at ETH Zürich. After studying mechanical engineering, he conducted his doctoral degree in robotics at ETH with focus on design, actuation, and control of dynamic legged robotic systems. Beside his commitment within the National Centre of Competence in Research (NCCR) Digital Fabrication since October 2015 Hutter is part of the NCCR robotics and coordinator of several research projects, industrial collaborations, and international competitions (e.g. ARGOS challenge) that target the application of high-mobile autonomous vehicles in challenging environments such as for search and rescue, industrial inspection, or construction operation. His research interests lie in the development of novel machines and actuation concepts together with the underlying control, planning, and optimization algorithms for locomotion and manipulation.

Links

• Download mp3 (11.1 MB)
• Robotic Systems Lab
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Credit: nutonomy

Over the past few weeks, we’ve blogged about how machine learning is transforming research, nuclear decommissioning, and astronomy. Building on our “Ask the Experts” panel discussion on driverless cars last July, how might machine learning shape the way we move through and interact with the immediate environment around us?

As the last in our series of blog posts on machine learning in research, we spoke to Dr Nathan Griffiths to find out more about machine learning in transport. Nathan is a Reader in the Department of Computer Science at the University of Warwick, whose research into the application of machine learning for autonomous vehicles (or “driverless cars”) has been supported by a Royal Society University Research Fellowship.

What inspired you to study science? And why Computer Science particularly?

I have always been interested in how things work and understanding the patterns we see in the world around us. Science helps us to understand patterns, provides insights into how things work, and ultimately enables us to understand how complex structures can arise from simple rules. Computer Science appeals because it can be both a creative and an analytical science – we can use computers to create patterns, or use computers as tools to understand our world, including the patterns that exist within it, and predict patterns which may arise.

How is machine learning used in your research?

My research uses machine learning to give insights into how objects or people interact and how patterns emerge and evolve. This involves the use of computational models, known as agent-based modelling, to simulate the actions and interactions of entities (such as drivers or autonomous vehicles) to see how their decisions affect the wider systems in which they operate.

More specifically, machine learning can be used to analyse how cooperation emerges through interactions between objects. In the context of autonomous vehicles, there are many situations in which it is desirable to establish a cooperative environment, without the need for centralised control. For example, who should give way at a road junction where the rules are ambiguous or typically misinterpreted, such as flashing headlights to let another driver proceed? In this case, machine learning can be trained to predict what decisions a driver or autonomous vehicle will make. Machine learning algorithms will examine previous behaviours and learn from these behaviours, to then predict what will happen in the future. Many of the initial predictions will be incorrect, however the algorithm will modify its next predictions based on what it has learnt and continue to reduce its mistakes until it can make accurate predictions. An accurate algorithm could then be used to inform the decisions vehicles make and predict vehicle journeys and routes.

Modern vehicles are equipped with numerous sensors to collect vast amounts of data during vehicle journeys. For example cameras provide live images to the vehicle which are fed into an object-recognition machine learning algorithm which learns to identify and distinguish between entities such as pedestrians, cyclists or other vehicles on the road. The images can tell the vehicle how close the objects are and change the vehicle’s behaviour accordingly, such as by reducing speed to stay a safe distance from the car in front. The use of these data also enables transport to be made more efficient, such as through automatic traffic light signal detection and live notifications of parking opportunities which could help to reduce congestion.

What are the challenges in using machine learning in your field of research?

The techniques for extracting information from data and building predictive models are advanced, but perhaps the greatest challenge is to support the interface between machine learning and human users, which is likely to lead to greater public confidence in something like autonomous vehicles.

I also expect to see advances in techniques for dealing with “concept drift”, where changes in the world   need to be accompanied by changes in a machine learnt model. Concept drift is the phenomena whereby predictions may become less accurate over time if the models do not adapt to changes in the environment. This could be an issue if learning models are not able to adapt to changes sufficiently quickly to react.

Currently, autonomous vehicles are good at detecting the world around them and reacting as necessary to changes on the road. However the next challenge is to enable machine learning systems to not only react to changes and adapt to previously unseen environments (such as a swerving car or the vehicle being driven in adverse conditions) but be able to understand the probabilities of what it thinks might happen. For example, a car which is driving alongside a busy footpath will not just react to a pedestrian stepping out into the road, but will understand that there is a chance that this might happen and actively take measures to reduce risk in case it does, perhaps by positioning itself further away from the curb.

So what are the future prospects for machine learning?

The future is very bright for machine learning. Machine learning is fundamental to the development of driverless vehicles and is rapidly transforming the automotive world; vehicle manufacturers are very keen to get computer scientists on board for the betterment of their projects.

Machine learning is already central to how our vehicles work, underpinning systems such as lane-keeping assistance, reactive cruise control and automated vehicle personalisation and recognition. There is a wide and increasing range of applications that machine learning will allow us to tackle and it is enabling us to develop fully autonomous vehicles, which will be ubiquitous in the future. I also envisage that autonomous vehicles will be connected, such that the information and insights that one vehicle has learnt can be transferred to another; which may prove particularly useful for managing concept drift.

The Royal Society is undertaking a project about machine learning and its impact for the UK economy and society. To find out more please visit our project page.

This post was originally published on the Royal Society. Click here to view the original post.

Heavy Water Components Test Reactor decommissioning.

How can we create robots that can carry out important tasks in dangerous environments? Machine learning is supporting advances in the field of robotics. To find out more, we talked to Dr Rustam Stolkin, Royal Society Industry Fellow for Nuclear Robotics, Professor of Robotics at the University of Birmingham, and Director at A.R.M Robotics Ltd, about his work combining machine learning and robotics to create practical solutions to nuclear problems.

What drew you to engineering and robotics?

There are many definitions of engineering, but the one I like is “the creation of artefacts for the benefit of mankind”. Engineering is a way of using science to be creative and to create novel technologies which can bring major societal and economic benefit.

Dr Rustam Stolkin. Source: University of Birmingham/YouTube

Robotics, in particular, attracted me because robots are inherently inter-disciplinary: they use mechanics and electronics, as well as computational elements, such as machine learning and artificial intelligence. This has allowed me to become an “expert generalist”; I specialise in being extremely cross-disciplinary. At the University of Birmingham, I have research collaborations spanning eight different schools across three colleges on the campus.

Tell us about your research?

My work focuses on developing advanced robotics technologies for nuclear decommissioning, demolishing legacy nuclear sites, and safely packaging, storing and monitoring any radiological or contaminated materials. The UK contains 4.9 million tonnes of legacy nuclear waste, mainly by-products of weapons production during the Cold War, alongside a smaller contribution from nuclear power use for civilian electricity generation. Cleaning up the UK’s legacy waste is the biggest environmental remediation project across Europe; it could take over 100 years and cost up to £220billion.

Factories are highly structured environments, where robots can be pre-programmed to make repetitive motions. This has supported the revolution in robotics that we’ve seen since the 1980s. In contrast, there has been surprisingly little use of robots in the nuclear industry, aside from a very small number of robotic vehicles deployed after the Fukushima disaster, and a very few cases of deploying bespoke robotic manipulators, controlled in rudimentary ways by joystick teleoperation.

This is in part because legacy nuclear sites, some of them over 60 years old, are highly complex and unknown environments, such as contaminated buildings that have been sealed off for decades. They have an enormous diversity of scenes, materials, and objects that nuclear decommissioning robots must interact with in complex ways, such as by grasping and cutting objects.

We have only recently discovered how to use state-of-the-art machine learning methods to help robots learn adaptive and transferable skills which they can apply in new environments. This allows them to perceive and handle new kinds of objects which they have not seen before. For example, using machine learning, a robot can be trained how to grasp and pick up one object, and then use this knowledge to adapt the grasp for a new object with a different shape.

Recent “deep learning” approaches might offer part of the solution in navigating these complex environments. However such learning approaches can rely on large amounts of “training data” – labelled examples of images of objects for example – which can be used to train the robot so that it can recognise similar objects in future. It is still very unclear how to generate sufficient training data in order to enable robots to learn how to behave in useful ways in complex real environments.

Why do we need robots to carry out nuclear decommissioning?

Contaminated materials are currently decommissioned by human workers in air-fed plastic suits, which is not only unsustainable but can be dangerous. Using robots means humans do not need to enter highly hazardous work environments, and means that zones that humans cannot enter (even in protective suits) due to high gamma radiation can still be decommissioned. Using robots also reduces “secondary nuclear waste”; for every one container filled with actual primary nuclear waste, more than ten containers become filled with contaminated plastic suits, respirators, rubber gloves, and other “secondary waste” from human entries.

How have you used machine learning to develop robots capable of nuclear decommissioning?

My lab has used robots to perform fully autonomous grasping and manipulation of nuclear waste-like objects and grasping of moving objects.

We also use state-of-the-art deep-learning neural networks to do real-time 3D reconstruction of a room, and simultaneously recognise and label materials such as concrete, metal, wood and fabric. This helps us develop ways for robots to characterise nuclear environments and nuclear waste objects in real-time. We are extending these methods to also combine visual camera information with other kinds of data from radiation, thermal and other sensors.

Where do you see machine learning going within your field of research?

Robotics engineers often argue that machine learning has only been used on relatively simple problems so far. Achieving machine learning of real robotic actions, with real objects in real scenes, is a significantly more complicated challenge that is only just starting to be attempted.

Innovative new approaches will emerge in the next few years for overcoming these challenges in enabling useful robotic behaviours in real environments. The next few years will see an interesting fusion of conventional approaches, such as hand-crafted robot control algorithms, with new approaches, like machine learning of robot behaviours.

Human-robot interaction, and human-AI collaborative working will also become increasingly important. What we are working towards now is “shared control” in which artificial intelligence and humans collaborate together to control a remote robot. For example, recent research from the French institute CNRS shows human operators controlling the motion of a robot arm and gripper in relation to an object, while the AI automatically orients the gripper with the object. Simultaneously, the AI automatically controls a second robot arm to move a camera so that it always offers the best viewpoint. Thus, human and AI collaborate to control a pair of arms for a remote handling task. In the future, this kind of human-machine collaboration could transform how we interact with nuclear and hazardous environments.

The Royal Society is currently carrying out a major policy project on machine learning, and its impact for the UK economy and society. To find out more please visit our project page.

This post was originally published by the Royal Society. Click here to view the original post.

In this episode, Audrow Nash speaks with Torsten Oelkes, Executive Chairman of CUBE, about the CUBE Tech Fair. The CUBE Tech Fair is a conference that takes place in Berlin in May that seeks to facilitate connections between companies. The Tech Fair also features a start up competition where the winner will be awarded €1,000,000 without exchange of equity or debt. In this interview, Toresten talks about CUBE’s business model, the robotic companies they’re affiliated with, the CUBE Tech Fair, and the €1,000,000 prize.

Torsten Oelke

For the past 15 years Dr. Torsten Oelke has been an active player in the German digital economy. As a businessman, he built up companies and incubators. Besides his business activities, he serves as advisor to the German Minister of Economic Affairs.

Torsten is the Executive Chairman of CUBE , a global ecosystem to connect startups with leading industry players. He has assembled a team of international experts with extensive experience with industry and startups. CUBE is backed by several board members of global corporates across industries.

Links

• CUBE’s website
• Tech Fair website
• Download mp3 (28.1 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Level 3 automation, where the car handles all aspects of driving with the driver on standby, is being tested in Sweden. Image courtesy of Volvo cars

Today’s self-driving car isn’t exactly autonomous – the driver has to be able to take over in a pinch, and therein lies the roadblock researchers are trying to overcome. Automated cars are hurtling towards us at breakneck speed, with all-electric Teslas already running limited autopilot systems on roads worldwide and Google trialling its own autonomous pod cars.

However, before we can reply to emails while being driven to work, we have to have a foolproof way to determine when drivers can safely take control and when it should be left to the car.

‘Even in a limited number of tests, we have found that humans are not always performing as required,’ explained Dr Riender Happee, from Delft University of Technology in the Netherlands, who is coordinating the EU-funded HFAuto project to examine the problem and potential solutions.

‘We are close to concluding that the technology always has to be ready to resolve the situation if the driver doesn’t take back control.’

But in these car-to-human transitions, how can a computer decide whether it should hand back control?

‘Eye tracking can indicate driver state and attention,’ said Dr Happee. ‘We’re still to prove the practical usability, but if the car detects the driver is not in an adequate state, the car can stop in the safety lane instead of giving back control.’

Next level

It’s all a question of the level of automation. According to the scale of US-based standards organisation SAE International, Level 1 automation already exists in the form of automated braking and self-parking.

Level 4 & 5 automation, where you punch in the destination and sit back for a nap, is still on the horizon.

But we’ll soon reach Level 3 automation, where drivers can hand over control in situations like motorway driving and let their attention wander, as long as they can safely intervene when the car asks them to.

HFAuto’s 13 PhD students have been researching this human-machine transition challenge since 2013.

Backed with Marie Skłodowska-Curie action funding, the students have travelled Europe for secondments, to examine carmakers’ latest prototypes, and to carry out simulator and on-road tests of transition takeovers.

Alongside further trials of their transition interface, HFAuto partner Volvo has already started testing 100 highly automated Level 3 cars on Swedish public roads.

Another European research group is approaching the problem with a self-driving system that uses external sensors together with cameras inside the cab to monitor the driver’s attentiveness and actions.

Blink

‘Looking at what’s happening in the scene outside of the cars is nothing without the perspective of what’s happening inside the car,’ explained Dr Oihana Otaegui, head of the Vicomtech-IK4 applied research centre in San Sebastián, Spain.

She coordinates the work as part of the EU-funded VI-DAS project. The idea is to avoid high-risk transitions by monitoring factors like a driver’s gaze, blinking frequency and head pose — and combining this with real-time on-road factors to calculate how much time a driver needs to take the wheel.

Its self-driving system uses external cameras as affordable sensors, collecting data for the underlying artificial intelligence system, which tries to understand road situations like a human would.

VI-DAS is also studying real accidents to discern challenging situations where humans fail and using this to help train the system to detect and avoid such situations.

The group aims to have its first interface prototype working by September, with iterated prototypes appearing at the end of 2018 and 2019.

Dr Otaegui says the system could have potential security sector uses given its focus on creating artificial intelligence perception in any given environment, and hopes it could lead to fully automated driving.

‘It could even go down the path of Levels 4 and 5, depending on how well we can teach our system to react — and it will indeed be improving all the time we are working on this automation.’

The question of transitions is so important because it has an impact on liability – who is responsible in the case of an accident.

It’s clear that Level 2 drivers can be held liable if they cause a fender bender, while carmakers will take the rap once Level 4 is deployed. However, with Level 3 transitions, liability remains a burning question.

HFAuto’s Dr Happee believes the solution lies in specialist insurance options that will emerge.

‘Insurance solutions are expected (to emerge) where a car can be bought with risk insurance covering your own errors, and those which can be blamed on carmakers,’ he said.

Yet it goes further than that. Should a car choose to hit pedestrians in the road, or swerve into the path of an oncoming lorry, killing its occupants?

‘One thing coming out of our discussions is that no one would buy a car which will sacrifice its owner for the lives of others,’ said Dr Happee. ‘So it comes down to making these as safe as possible.’

The five levels of automation:

1. Driver Assistance: the car can either steer or regulate speed on its own.
2. Partial Automation: the vehicle can handle both steering and speed selection on its own in specific controlled situations, such as on a motorway.
3. Conditional Automation: the vehicle can be instructed to handle all aspects of driving, but the driver needs to be on standby to intervene if needed.
4. High Automation: the vehicle can be instructed to handle all aspects of driving, even if the driver is not available to intervene.
5. Level 5 – Full Automation: the vehicle handles all aspects of driving, all the time.

Robotics undoubtedly has the potential to improve lives in the developing world. However, with limited budgets and expertise on the ground, putting this technology in place is no small task. Step forwards WeRobotics, a new Swiss/American NGO dedicated to meeting this goal through the creation of in-country ‘flying labs’. Co-founder Adam Klaptocz explains all.

Hey Adam. Let’s start with this: what is WeRobotics?

WeRobotics is a non-governmental organisation (NGO) dedicated to bringing developing countries access to robotics technology. Such countries don’t have access because of economics, supply chains and many other reasons. Our role is to help local people learn and use this technology, and then to deploy it for social good projects within their own communities.

We do this by establishing ‘flying labs’ in different countries. The goal of each is to serve as a hub of robotics technology, where staff host training sessions, webinars and teach people how to use technology. Each lab will have drones and software, on hand and in the cloud, thanks to our technology company partners. Labs will also function as an incubator for new, local, drone-based businesses.

WeRobotics co-founder Adam Klaptocz (right) and Nepal Flying Labs Coordinator Uttam Pudasaini, running a drone mapping training session in Kathmandu, Nepal.

And who is behind WeRobotics?

We have four co-founders; myself and Sonja Betschart are based in Switzerland, and Patrick Meier and Andrew Schroeder, who are based in the U.S. We are a 501c3 non-profit organisation in the United States and have just founded a sister organisation in Switzerland too. Besides that, all of our flying labs will be legal entities as well, to go along with the concept of building local capacities and driving local engagement.

The goal is to engage local lab coordinators, local organisations, and ideally make the local flying lab its own organisation that has its own activities and is sustainable

The goal is to engage local lab coordinators, local organisations, and ideally make the local flying lab its own organisation that has its own activities and is sustainable
The goal is to engage local lab coordinators, local organisations, and ideally make the local flying lab its own organisation that has its own activities and is sustainable

And you’re talking about unmanned robots in their most general sense, not just as a synonym for flying drones?

That’s right. At WeRobotics, we’re looking at robotics in the largest sense—it’s not just eBee drones for mapping, it’s also things like drones for delivery, marine robots, underwater robots, and terrestrial robots. Anything to do with robotics and the technical ecosystem that goes around it. That includes the software and the platforms people use to share a drone’s data because, in the end, drones are just a tool for collecting and turning the data into actionable information.

We have tech partners, such as senseFly and other drone and software companies, with whom we are spreading this technology throughout our network of labs. The labs will then run training programmes, serve as incubators for locals who want to get into the business, and host conferences where these adopters can share their experiences.

So do you already have your first lab set-up? Where is that?

Absolutely, we’re a young NGO, but we’re trying to expand quite quickly.

Our first lab was officially launched recently in Nepal. It’s called Nepal Flying Labs. We have a local lab coordinator there and we’ve partnered with Kathmandu University, ICIMOD, which is a mountain and glacier study organisation, NAXA and Medair. We’re working with government agencies as well, including the survey department of the Nepalese government. So we have a certain number of partners that we’re working with to roll out projects and run trainings.

We have projects that will be running throughout the next year in Kathmandu and the surrounding areas; a lot of which is post-earthquake reconstruction work after last year’s earthquake. Also, a lot of climate-based research work.

That must be ground zero, that region, for such scientific study?

Exactly. Nepal is basically a giant mountain so the effects of climate change are really affecting communities there. There are a lot of landslides – the Himalayas are melting, the glaciers are melting – so there is a lot of work to be done in terms of disaster prevention and also in post-disaster reconstruction.

And in Tanzania, where we are setting up more flying labs. I know there’s already been a lot of work there by senseFly, Drone Adventures [case study], The World Bank and other organisations. We’re hoping to work with these organisations and others to consolidate a lot of the work that’s been done; to make it sustainable for years to come within Tanzania.

Our third major lab that we are setting up before the end of the year is in Peru. In Peru, we will start in the Amazon. Our first pilot project there is about drone delivery, but we’ll also be spinning out drone mapping as a business and other types of robotics as well.

In the countries where you operate, is one of the groups you’re engaging with the government itself? Do you see your role to also help governments regulate the use of drones?

The mandate of each flying lab, as an NGO, isn’t to replace government, it’s to help.

So the first thing to do is to work with the government and try to help boost, for example, its capacity for mapping. That’s something that we believe in strongly. We’re already working with the survey department in Kathmandu and we’ll be working with the Lands Department in Tanzania as well to try to reinforce their capacity to serve their own people. On top of that, of course, we want to be able to work within countries – within the rules of those countries – and so we’re trying to help with regulations, by working with the regulatory authorities in all of these countries.

A lot of developing countries are looking for guidance from other countries – where robotic technology might be a little more established – to see what is happening. That’s another thing that we hope, as an international NGO with locations all around the world, we’ll be able to facilitate—helping governments to learn what works in other countries, such as those our network covers, and helping them to work together, often with the civil aviation authority and other government agencies. It’s about implementing rules that make drone use responsible, but also leave the market open so that we can incubate businesses. Like that, the economy can develop around this technology instead of being blocked.

You mentioned before we started recording that the key, underpinning what WeRobotics is about, is building capacity on the ground.

Yes, that’s the key, local capacity. It shouldn’t have to be that a bunch of foreigners come in to do this work whichever way they want, often with their own desires, with their own culture and their own needs. Basically, making themselves feel good about helping instead of actually helping.

What would be best, if this ever happens again in Nepal, is that there are enough drone companies locally that know what they’re doing, and understand the legal framework within the country, so that they can do the job instead of having to bring in foreigners. And that goes for every country. I think that, in development, the more local capacity you build, the more you can really bring the country out of poverty so that it can have its economy self-sustain itself.

That’s our goal with the flying labs: to build self-sustainable local entities that build local capacity but have global contact with other labs and with our global technology partners.

What specific UAV platforms do you operate at present and how are these being used?

I truly believe that every job has a tool that’s best adapted to it. In Nepal, right now, our flying lab has aneBee and a Parrot Bebop drone. The eBee is used because Nepal is obviously very mountainous and having the capacity to deal with 3D flight mapping is very important.

Meanwhile, in Tanzania there are larger, mainly flat zones to be mapped. For that we have a Sky-Watch Cumulus drone. This is a longer-range drone that can fly for two hours and cover very large areas. So there that’s the best adapted drone. We’re also bringing a Bebop to Tanzania, because there is a big interest in drone journalism there and for that it’s important to have something that is small, cheap and easily deployed.

Then in Peru, we’re working with hybrid drones that are made more for cargo delivery. One option isQuantum Systems, which is a German company that makes even bigger, VTOL drones with an even bigger payload capacity.

So, different drones for different jobs.

100% platform agnostic.

Yes. One of our goals as an NGO is to remain more or less independent of technology. What’s important is that the job can be done with the best fit technology, in an economically viable way. We need UAVs that are easy to use, easy to train on, and relatively mature as well.

One of our goals as an NGO is to remain more or less independent of technology. What’s important is that the job can be done with the best fit technology, in an economically viable way.

We do work with some startups, but in general what you really need, if you want to make something work in the jungles of Papua New Guinea or in the middle of the desert somewhere, is a machine that’s durable and that has existed for a little while. A platform that has some sort of customer support is needed too, so it’s important that our technology company partners are happy to be involved as well.

If you imagine a few years down the road, having achieved your short and medium term goals, how does the world of WeRobotics look then?

We see ourselves, a few years down the road, having many flying labs around the world. We’re not sure if we’ll have five, or ten, or if there’ll be 50, but the goal is that we have setup a network of many flying labs in different countries.

And you’re planning on developing some in-house engineering capacity too?

Yes, because there are many problems, like conservation in developing countries, which can be solved with existing technology. But there are also many problems that can be solved with technology, but the technology doesn’t exist yet, because the people developing the technology—whether it’s Silicon Valley, China, or Europe—aren’t necessarily aware of the problems that exist in developing countries.

Whatever works, whatever’s needed to meet your goals on the ground then?

Yes. Unfortunately, there is no lack of work in development. Poverty is not disappearing next year. It’s not a market that you have to grasp today otherwise tomorrow it has been taken over. There is a huge amount of work left to do, and there’s definitely enough to go around.

If you liked this article, you may also be interested in:

• Cargo drones deliver in the Amazon rainforest
• The Drone Center’s Weekly Roundup: 2/13/17
• The art of drone flight: Creating drone costumes
• The Monospinner: world’s mechanically simplest controllable flying machine
• Flying cars are coming, what will they mean?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

An Air Force airman at the control module of an MQ-9 Reaper. Credit: John Bainter/USAF

Machine ethics offers an alternative solution for artificial intelligence (AI) safety governance. In order to mitigate risks in human-robot interactions, robots will have to comply with humanity’s ethical and legal norms, once they’ve merged into our daily life with highly autonomous capability. In terms of technical challenges, there are still many open questions in machine ethics. For example, what is deontic logic and how can it be used for improving AI safety? How do we fashion the knowledge representation for ethical robots? What about the adaptiveness of the ethical governance? These are all significant questions for us to investigate.

In this interview, we invite Prof. Ronald C. Arkin to share his insights on robot ethics, with a focus on its technical aspects.

Date: February 1st 2017

Interviewee: Prof. Ronald C. Arkin, Regents’ Professor and Associate Dean in the College of Computing at the Georgia Institute of Technology and Executive Committee Member of IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous Systems

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan).

Ron Arkin

Q1 WENG: Thank you for agreeing to an interview with us. Can you tell us a little about your background?

ARKIN: I have been a roboticist for 35 years, so it’s hard to tell [only] a little, but I will do my best. Currently I am a Regents’ Professor and Associate Dean in the College of Computing at the Georgia Institute of Technology. I got my Ph.D. from the University of Massachusetts in 1987, and have been working on robotics since that time. I actually have degrees in chemistry as well—a bachelor’s and a master’s in organic chemistry. So, I have a rather diverse background and followed a non-traditional path compared to most computer scientists.

Q2 WENG: What is your definition of “Ethical Robots”?

ARKIN: That’s an interesting question! I have been asked that one before. I would say an ethical robot is a robotic system that is able to comply with the ethical and social norms of human beings—in some cases in accordance with international law, in other cases in accordance with national or local law. In addition, also being able to comport with humans in a natural manner, obey, and adhere to the same principles as an ethical person.

Q3 WENG: Could you please tell us why you are particularly interested in “Embedded Ethics”? Why is this concept so important?

ARKIN: That’s why you need to read the preface of my book [Governing Lethal Behaviour in Autonomous Robots]. It’s a rather long story. But let me try to give you a short version. Again, it dates back to a meeting—the first international symposium in Roboethics in 2004—where I was exposed to international conventions, e.g., the Geneva convention. It also led me to understand the Vatican’s position on how they viewed appropriate ethical interactions. Perhaps even more important than that one experience was the fact that we roboticists that have been in the field for some time finally started to have success, and we were seeing our technology move out of the laboratory community and into the real world. As such, it wasn’t hard to see how this could go in a variety of different directions.

Some robotics technology could benefit humanity, some not so much. At that point, one has to ask, “who is responsible for this?” So I assumed that, in part, it was myself. I believe that I have to take the responsibility for the technology that I helped to create. The issue is that we, as roboticists, had actually succeeded in many ways in terms of creating technology that we only envisioned when we first started that is going to have potentially profound impact on our society and the world as a whole. As such, we should be proactive in the ways in which we move forward [in order] to get a better understanding of the consequences of the work, and the social implications of the technology we are designing.

https://www.youtube.com/watch?v=jOJwkvihYtc

Q4 WENG: The IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous Systems has published their report “Ethically Aligned Design” in December 2016. As a member of the IEEE Global Initiative, could you please tell us about your role?

ARKIN: I serve a variety of roles. First, I am on the executive committee of that particular initiative, so I work at a high-level with about 30 other people or so. We have committees that large. I also co-chair the committee in Affective Computing. Joanna Bryson is my co-chair on that.

Affective Computing is concerned with the issues surrounding the illusion of emotions we can potentially create in these artifacts, and the ways human beings can bond to them. We are exploring a variety of different things, including robot intimacy, or the notion of applying robotic nudges and moving people in certain directions subtly, and in so doing to change the way people act and behave using these technologies, while trying to understand what may or may not be appropriate guidelines based on ethical considerations.

It’s about 15-20 people in that committee. We are one of the newer committees and we only have a couple of pages in the initial design report. But actually, in the June meeting in Austin, for the next release of the document 2.0, we will have as much as the other committees do. Finally, I am also a member of the humanitarian committee. I think it is called economic and humanitarian. I don’t know the exact title, but I am interested in the humanitarian side, and potential uses of robotics for humanitarian applications.

Q5 WENG: What is “Deontic Logic”? Why does it matter to machine ethics?

ARKIN: It may or may not matter. It’s just one approach to a particular problem. Deontic logic is a logic of prohibitions, obligations and permissions. It is one type of framework that is more transparent in order to get a better understanding of the way in which systems operate in the real world. Then it explains its reasoning to a human being, as compared to say deep learning or other forms of neural networks which may indeed provide good results but provide no explanatory capability to humans after the decision has been made.

The notion of prohibitions basically tells the robot some things it cannot do. The notion of permissions describe the things it is potentially allowed to do. And the notion of obligations says things that the robot must do under specific circumstances. It provides a nice framework of being able to explore morality. Others have already explored this, like Bringsjord at RPI who has applied deontic logic in this framework. We are certainly not the first in that particular space. Much of the work we have done has dealt with first-order predicate logic as a basis. It doesn’t have to be deep because it is so explicit in many cases.

Q6 WENG: Your hybrid embedded ethics framework aims for governing lethal behavior of military robots. Can we apply the same technical framework to service robots? For example, personal care robots might have some non-lethal, but still risky autonomous behaviors that need to be regulated by machine ethics.

ARKIN: I believe so. Indeed the whole point of the architecture is trying to design the systems to make them generalizable and not ad hoc for particular sets of circumstances. We have yet to look at service robotics in a traditional sense, but we have already developed something we refer to as the intervening Ethical Governor, which we are using for health care applications—particularly for assisting the management of two human beings in a particular relationship, like a patient and their care giver.

A robot hands a medication bottle to a person. Photo credit: Keith Bujak. Source: Georgia Tech News Center

We are doing work for the National Science Foundation, joint with Tufts University, looking at how in early stage Parkinson’s there is a condition called the facial masking. The loss of muscle movements in the face of the patient can produce stigmatization, and the patient-care giver relationship deteriorates due to the loss of non-verbal communication. We just had a paper accepted at ICRA describing the architecture that we have and how it has been modified from the military version. We generalized it from the military constraint-based version to a rule-based version.

But also, instead of using the international humanitarian law, we are using occupational therapy guides as a basis for generating the responses when two human beings in the relationship step outside of acceptable bounds. So the acceptable bounds now are not defined by international humanitarian law, but they are defined by occupational therapy—such as when a patient or a care giver start to yell at each other, or one starts to leave the room. The hard part is recognizing when dignity is being abridged, particularly in the patient. So we model the moral emotions of empathy, shame and embarrassment. We tried to ensure that the empathy level in the care giver remains within acceptable bounds and that the patient shame and embarrassment doesn’t rise to unacceptable levels. So the intervening Ethical Governor intervenes when a really inappropriate event occurs. It then provides strong corrective action by the robot to encourage people to get along better. The moral adaptor component which we used in the ethical architecture—initially with the guilt model—has now been extended to other moral emotions like shame, embarrassment, guilt and empathy [in order to] try and provide more subtle non-verbal queues [to help the] robot steer the relationship in the correct way before it get out of hand.

Q7 WENG: Regarding your research on machine ethics, what kind of indicators you use to evaluate the final outputs of your project?

ARKIN: In the first one, we were designing a prototype to prove the concept of the architecture. We never make the claim that this is the best way to do it. It is just one way to do it. Then we demonstrate that particular capability through the use of simulations and other aspects. The system was not intended to be fielded, but hopefully others could take those core ideas and potentially put them to use in the real world if they so choose. For the intervening governor, we’re running human subject tests right now—not with Parkinson’s patients yet, but with the elderly population, 50 or older. We’re trying to get a better understanding of the correct sets of rules, and to correct the interventions in a normative fashion, so we can ultimately move on in the final year of the grant to work with actual Parkinson’s patients. Again, using standard HRI technics, such as recording the kinds of reactions they have and parsing the data from the sessions and using pre- and post- evaluation questionnaires.

Q8 WENG: Let’s talk about your proposed machine ethics framework. You had mentioned that it includes four main parts: (1) Ethical Governor (2) Ethical Behavior Control (3) Ethical Adaptor (4) Responsibility Advisor. First of all, what is the major difference between Ethical Governor and Ethical Behavior Control? Is it the difference between deliberative and reactive control, or something else?

ARKIN: The Ethical Governor is basically a standalone component to the architecture which is a bolt on component at the end of the existing system. In the case of military scenario: asking for permission—should it fire or not. This is an intervening stage which either grants or denies permission through the validation of permission-to-fire variable. So it is straightforward thing to be able to do. The Behavior Ethical Control is a different kind of thing as it works purely with the reactive component. What it tries to do is, for each of the individual behaviors themselves, to respond whether or not to engage a target. This is prior to dealing with the Ethical Governor making the decision; trying to make the decision correctly before it gets to the Ethical Governor, where individual behaviors are given ethical guidelines to try and ensure that the behavior is correct itself. Each of the behaviors is provided with appropriate ethical constraints. So the difference is: one is located within the reactive component in each of the individual behaviors that makes the activity of the architecture, and the Ethical Governor is at the end of the architectural chain.

Q9 WENG: What is the importance of Ethical Adaptor? What kinds of machine learning skill that you chose to improve its adaptiveness?

ARKIN: Machine learning is something we haven’t generally explored in this particular context. Especially the military context, we kind of engineer out the adaptiveness and machine learning for fear that the robot would be granted the authority to start to generalize the targets in the field, which we feel is inappropriate. Learning after action is okay. but that’s a different story. The importance of the Ethical Adaptor, and its cognitive models of guilt that we use, is the fact that even the actions the agent takes may inadvertently end up unethical. This is referred to as a “moral luck”. It could be good luck or bad luck. But if it is bad luck, you want to be able to ensure proactively that the bad action doesn’t occur again. That’s actually the functions of cognitive guilt for people as well to reduce the likelihood of reoccurrence of a behavior that resulted in bad actions. By giving this to a robot we hope that if it makes a mistake it will not repeat it. The probability of that behavior reoccurring would be reduced.

The best example is the case of battle damage assessment for a military scenario. For example, if a robot does its assessment and it looks like a released weapon wouldn’t create too much collateral damage, within acceptable limits, then it will release the weapon. But after it does it, the battle damage action is actually more severe then, it initially appeared to be. Under those circumstances, we want the robot in the field to reduce the probability of releasing the weapon again under similar circumstances. That’s, in essence, the analogue of guilt we use, and eventually, it will get to the point that the robot will not engage targets at all. Again, the software is updated to make sure the use is consistent with Geneva convention.

Q10 WENG: The Laws of War (LOW) and Rules of Engagement (ROE) are written in the form of natural language. How did you transfer these norms into machine readable formats?

ARKIN: It’s important to keep it in mind that we do not try and capture all the Rules of Engagement and all the Laws of War. We always advocate that the system should be used in very narrow circumstances and very narrow situations, such as building clearing operations or counter-sniper operations. It’s impossible to capture all human morality and all the Laws of War at this particular point of time. So, the question is what are the relevant Laws of War and Rules of Engagement for these particular narrow situations that the robot will find itself in.

Let’s take a room clearing operation. We try and do this using these constraint-based approaches. The Laws of War basically say that if you see a civilian, you should not shoot them; if you see a surrendered person, you should not shoot them. For aerial vehicles, if you are outside the kill zone (this is the simulations we did) which is designated by the Rules of Engagement, you cannot fire your weapon—explain simply why you cannot do that. If you release the weapon a toward a legitimate target but it would result in damage of cultural property, like a mosque or church or some similar circumstance, that is illegal as well. You are forbidden from doing those sort of things.

So, the initial work we did was more for aerial circumstances. We also computed the collateral damage using a battle damage assessment strategy to determine whether if it is within acceptable limits for a particular target and to restrain the weapon for proportionality. To be able to do this you have to be able to understand the military necessity of the target, which is assigned by a human being. You have to be able to understand the proportionality of the weapon. In other words: Which weapon is sufficient to take out the target but to do no more damage of the surroundings than is absolutely necessary? You have to ensure that you don’t destroy cultural property as well. Basically, adhere to all necessary principles, such as discrimination which is to be able to tell apart combatants and non-combatants. We also state we should restrict the use of these particular systems for high-intensity combat operations, or inter-state warfare, and not the kind of circumstances you find in modern counter- insurgency operations. They should be used with great care and great caution in very narrow circumstances where the predictability of their performance is far more likely to be good.

Q11 WENG: Why did you choose case-based reasoning (CBR) but not rule-based reasoning (RBR) when you designed Responsibility Advisor?
ARKIN: Historically we used case-based reasoning a lot for a variety of different reasons. Part of the reason is to gain information from previous experts. One of the examples in the past was to work with snipers in the laboratory. We can work with experts. It can be legal experts, it can be medical experts, it can be military experts. Again, the primary reason is to be able to readily use the experience of warfighters to guide us in operations.

Q12 WENG: Finally, your proposal “Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture” was published in 2007. There have been great technological improvements in artificial intelligence in the past decade. Are there any recent updates from your robot ethics research, or anything that you want to share with us?

ARKIN: The project ended around 2010, so there is additional information. As I say, I would encourage those who are really interested to take a look at the book entitled “Governing Lethal Behavior in Autonomous Robots” . That represents the final statement and outcome for that particular project and includes the significant technical details describing that work. Recent advances include the application in other domains, such as the caregiver-patient relationship [discussed above] using the same underlying principles, including both the Moral Adaptor and Ethical Governor in a slightly different. If we were successful, it may also be useful for assisting other human-human relationships, such as a parent and a teenager who may be at odds with one another or even in marriage consulting. We hope to use robots for the benefit of mankind, whether it’s to make sure that warfare is conducted within internationally agreed upon laws, or whether it’s to ensure that people behave appropriately toward each other

If you enjoyed this article, you may also enjoy:

• How do we regulate robo-morality?
• One being for two origins: A new perspective on roboethics
• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• On the ethics of research in robotics, with Raja Chatila
• A European perspective on robot law: Interview with Mady Delvaux-Stehres
• Open Roboethics initiative delivers statement to United Nations CCW

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Jussi Aaltonen explaining some robotics-specific topics.

The UNEXMIN (Underwater Explorer for Flooded Mines) project is almost one year old. After a busy first year of work, UNEXMIN is on-schedule to deliver the first mechanical UX-1 prototype. Jussi Aaltonen, from TUT (Tampere University of Technology), leaders of WP1 – Robotic Functions Validations, talks about what has been done over the past year concerning his team’s work in UNEXMIN’s development scene.

Luís Lopes, LPRC: What is WP1 (Robotic Functions Validations) and what has been developed from it?

Jussi Aaltonen: The purpose of WP1 is to develop and validate a robotic platform, i.e. all mechanical and mechatronic structures, systems and components of UX-1 (the name of the robot). Also, low level (directional) control principles are developed and validated in this work package. WP1 has successfully established a robotic platform which fulfils all necessary requirements.

LL: How important is the work developed in WP1 for the future UX-1 robots

JA: Work in WP1 is fundamental for the successful completion of the project. All later development stages rely on this work.

LL: The systems/subsystems that the robot will employ were defined in this WP. What can you tell about them? How crucial are they for the robot’s overall functionality?

JA: All systems are needed for the robot to achieve full functionality. However, survivability, i.e. the robot’s capability to return from its missions, is more heavily affected by some systems than others. It is, for example, possible to return with a broken multi-spectral camera, but losing a sonar may cause the robot to be lost.

LL: What kinds of tests were developed to test the hardware/software systems? How about the results obtained, were they positive?

JA: Tests are done as so-called hardware-in-the-loop (HWIL) tests in laboratory. In HWIL testing, computer simulations and real hardware components interact, making it possible to do laboratory testing more closely to real operating conditions. Components and systems have also been tested in the pool by using a plastic mock-up of UX-1. Tests have been successful, which, unfortunately, does not always mean positive results. But weaknesses were found and lessons learned.

LL: In what conditions was the prototype tested?

JA: Tests have been done in shallow depths (below 5 m) and moderate temperatures. Pressure testing at 60 bars (equivalent of 600 m depth) will start soon.

LL: The robot is designed considering stakeholders’ wants. How hard was it to define the requirements and specifications for the UX-1 robots?

JA: I have to say that requirements and specifications are the easy part. The hardest is to design a robot complying them all.

LL: Being leader of such an important WP of the UNEXMIN project must be challenging. Has it worked out as you imagined? Do you feel that everything has been accomplished?

JA: Over this past year of cooperation, UNEXMIN teams from the various partners have gotten to know each other and each other’s ways of working. The possibilities to meet the others in person and to really collaborate in real life conditions, such as in workshops, have been a great benefit and a very important thing, bonding people from all around Europe together and working towards  a common goal. Of course there are always some challenges. The biggest challenge has been finding a common and efficient enough platform for information exchange.

LL: TUT will organise a UNEXMIN workshop in the beginning of February (2017) in Tampere, Finland. What can be expected to come out of this?

JA: This event will conclude the work of the first year in the UNEXMIN project. Most of the work packages have already started or about to start (exception goes to WP7: Demonstration/Pilots), and thus we will hear a lot about results and developments in this meeting. We will also hear about the plans for a couple of essential work packages starting right after the meeting. I expect that we will have a good and fruitful meeting with the same good spirit and warm atmosphere that we have always had. There will probably be some heated discussions, but that is only a sign of people being passionate about their work.

LL: You are a specialist in the robotics field. What possible impact do you think the UNEXMIN project will have on the robotics field?

JA: UX-1 [UNEXMIN robot design-prototypes] will have a very different structure and layout than what autonomous underwater vehicles usually have. This will most probably have some effects on the field of robotics and will open new doors to further miniaturised underwater robots – the future starts here. Also, the environment where UX-1 will operate is probably the most challenging after Mars and asteroid landers and that will also give tiger leaps in the field of robotics, developing new and better robotic systems.

LL: After one year, how do you feel the UNEXMIN project has progressed?

JA: We are roughly in the place where we should be according to our plans. Work has been harder than what was expected. On the other hand, we learned a lot more, while still managing to keep pace.

This post was originally published on UNEXMIN.

If you enjoyed this article, you might also be interested in:

• Surveying flooded mines with UNEXMIN: Interview with Norbert Zajzon
• When robots become art: Interview with Yi-Wei Keng
• A European perspective on robot law: Interview with Mady Delavaux
• Art meets robotics in Patrick Tresset’s OPLINE Prize entry
• Robots Podcast #171: Robotics in theatre, film and television, with Grant Imahara and Richard McKenna

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In April 2016, British Standards Institution (BSI) published the world’s first ethical standard for the design, production, sale, and usage of social robots. “BS 8611: 2016 Robots and robotic devices” gives guidelines to identify ethical hazards and reduce associated risks to acceptable levels. BS 8611 could be crucial in determining the nature of emerging human-robot relationships. However, the publication also raises many new questions. Are there any conflicts with ISO 13482 when incorporating BS 8611 into the design of safer robots? Should we consider regulating robot ethics? If so, how?

In this interview, we invite Prof. Joanna Bryson to share her insights on how to regulate future human-robot relationships.

Date: December 9^th 2016.

Interviewee: Prof. Joanna J. Bryson, Reader at Department of Computer Science, University of Bath, and affiliate of the Center for Information Technology and Policy, Princeton University.

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan).

Prof. Joanna Bryson. Image: University of Bath

Q1 WENG: Thank you for agreeing to an interview with us. Can you tell us a little about your background?

BRYSON: My primary academic interest is comparative intelligence.  I was particularly interested in animal behavior as a student, and took a liberal arts degree from Chicago with a major in the Behavioural Sciences.  I now have two degrees in Artificial Intelligence (including my PhD at MIT) and another masters degree in Psychology (both masters are from Edinburgh.)  I’m currently an Associate Professor at Bath where I founded the Intelligent Systems group, and organized Artificial Models of Natural Intelligence, which is basically my own laboratory.

During my PhD I was working on a humanoid robot project and was astonished that people thought they had moral obligations to the robot, even though it did not function at all.  It was just a bunch of motors welded together in the shape of a human.  I began publishing about this problem, thinking that people didn’t know enough about AI, but in recent years I’ve realised they don’t understand enough about ethics.  They are confused about what is really the root of ethical obligation, and they do not understand the nature of artefacts.  They do not understand that we can construct both AI and ethics so that we either are or aren’t obliged to AI.  I have become well-known for arguing that we are ethically obliged to construct AI we are not ethically obliged to.

 Q2 WENG: What is the meaning of “ethical hazards”? Except deception and anthropomorphism, what are other ethical hazards in BS 8611: 2016?

BRYSON: I have to admit I haven’t memorised that document and don’t have time to go back to read it now.  Maybe you are talking about a moral hazard?  A moral hazard is something likely to make you do the wrong thing.  In particular, I have argued that attributing ethical status to AI is a moral hazard, because it allows us to escape the responsibilities we ourselves have as the manufacturers, builders and operators of AI.  This is particularly obvious in the case of military weapons – we have enough trouble with distributing blame between soldiers, their commanders, and the government that commissions them.  Making weapons themselves “sinks” for guilt is not honest or beneficial.  But there are also moral hazards associated with owning “friends” and “lovers” – if children relate mostly with things they own, how will they grow to be able to be equal partners with their spouse or coworkers, or members of a democracy?

Q3 WENG: Some members from Bristol Robotics Laboratory are particularly interested in “embedded intelligence”? Why is this concept important? Does it have any impact on BS 8611: 2016?

BRYSON:  I’m sorry, I don’t think it matters particularly whether AI is embedded in an artefact or “channelled” through the cloud, except obviously there are privacy issues when we connect AI onto the Internet or into any shared digital memory.  In fact, one of the ways I recommend ensuring that we have no ethical obligation to robots or other AI is to ensure there is never a unique copy: that it should be backed up to remote sites at all times.

Q4 WENG: Prof. John Holland said, “..an internal model allows a system to look ahead to the future consequences of current actions, without actually committing itself to those actions”. Can you tell us about the concept of “internal model”? Does a passive dynamic robot have its own internal model as well?

BRYSON:  Ha, that’s a great question!  The idea of an internal model is just a representation that allows, as Prof Holland says, a means for planning around and anticipating the world.  Rodney Brooks famously recommended that AI should not have a model, should use the world as it’s own best model, but in fact that is not practical. There’s not enough information in a sensory signal to disambiguate a world sufficiently to create animal-like intelligence, we need prior expectations to interpret the sensory input.  Still, Brooks was a genius of AI and the field moved forward much faster when we realised we could use minimalist representations; just those necessary for action.  We realised that humans “flush” our memory quickly if sensory information contradicts it, and operate on partial knowledge all the time.  Previously we though AI had to have perfect knowledge so it could make perfect plans, but now we understand that that isn’t tractable.

So does a passive dynamic have an internal model? No.  It is a model of an animal walking system, but to some extent it embodies expectations about for example gravity, but by and large it can do no planning or control. It arrives only where it goes and never knows that it has gotten there or been anywhere else, so certainly cannot anticipate anything.  But its design anticipates particular regularities about the world, so its designers had models that are partly embodied in the walker.

Q5 WENG: I conducted an empirical case study on humanoid robots and regulation with researchers from Waseda University. We found that inside “Tokku” Special Zones there were strong demands for regulating robotics in road traffic, privacy, tele-communication, safety, tax, highway traffic, etc… However, it is not clear that whether robot ethics should be regulated. Do you have any comments on this?

BRYSON: Unfortunately I also haven’t had time to read your study yet, but I can talk a bit about regulation.  There are two important things to recognise.  First, government should not be thought of as an imposition, particularly in a democracy, but even in an autocracy government only functions to the extent it is tolerated.  Government and regulation is the way we citizens coordinate our desires and vision.  Many people in technology seem to think of regulation only as limits on their freedom, and indeed even the word compromise can be used in both a negative and a positive way.  But technologists need to understand that when people come together and work together, they often create something of mutual benefit, public goods that are greater than the summed contribution of those who are cooperating.  Regulations can include those that mandate investment of government resources (our taxes) in innovation.  They can be liberating as well as constraining. Coming to agreements with fellow citizens about essential matters like privacy and the distribution of wealth that AI has huge impact on is essential, and yes, agreements require compromise, but it is through these that we create peace, stability, and affluence.

Having said all that, what I get from surveys of citizens is not information about what regulations would really work.  Governance and regulation are extremely complex and have been studied for centuries; it is as crazy to suggest that ordinary citizens construct regulations as that they construct nuclear weapons or that random people be drafted to represent a country in the World Cup or the Olympics.  We need to respect expertise as well as the desires of ordinary citizens.  What the fears and goals of citizens tell us is how easy or hard it would be to implement particular regulations, how likely they are to be accepted and respected.  Most people on the planet believe that their ancestors still have concerns with their own behaviour, but I wouldn’t want to build policy relying on dead ancestors as actors.  But it does make sense to know people think this if you are governing.

 Q6 WENG: What’s your definition of “ethical robots”? Should we teach them right from wrong?

BRYSON:  I think it makes more sense to program than to teach. We have enough trouble teaching citizens and governments to do good.  Most actions have ethical consequences and there is no problem in building AI to recognise and reason about these, but I would never recommend holding the AI as morally responsible, even though I think AI will necessarily take actions with moral consequences.  As the British EPSRC Principles of Robotics make clear, the responsibility for commercial products lies either in their manufacturer (if they fail to meet an advertised specification) or in their owner / operator (if they are improperly used.)  There are ethical implementations of AI, for example protecting the privacy of the individual, or limiting the damage an individual can do with an autonomous vehicle, but the AI itself should not be designated the responsible party in any circumstance.

 Q7 WENG: Do you believe that Asimov’s Three Laws of Robotics will be guiding principles to ensure the safety and security of human-robot interactions?

BRYSON:  Asimov’s laws certainly impact how people think about AI ethics; the first three (of five) Principles of Robotics reflect (and correct) Asimov’s laws.  But we are working hard to make people understand that these laws are by no means necessary or sufficient for AI ethics.  First of all, Asimov’s laws are intractable. It’s computationally impossible to implement them as described in the books, we can never know all the consequences of our actions.  Second, the books themselves show repeatedly that the laws are insufficient to create truly moral behaviour, that’s the fun of the short stories.  And third, Asimov like most science fiction writers uses AI as a stand-in for humans, for examining what it is to be human. The laws describe the robot as the responsible party, and that’s as I said earlier a mistake.

 Q8 WENG: Do you agree that robots should not be legal persons? Why?

BRYSON:  I don’t know for sure whether people shall make them legal persons, but I am certain that they should not, and in the long run will not be.  A legal person is something that is subject to human justice, and I do not believe it is possible in a well-designed system to create the kind of suffering that is intrinsic to human punishment.  There is no necessity for robots or AI to know shame, pain, regret for the passage of time.  These things are completely intrinsic to being human, we do not consider people with deficits in these concerns to be mentally healthy.  This is because we have evolved to be social organisms and there is no way to extract these properties from our intelligence.  But well-designed AI is modular, anything that is deliberately added by a designer can be removed.  So that’s one reason it would be incoherent for AI to be a legal person.  Another reason is that there is no discrete unit of AI or robots like there is for humans.  We know what it is to be a human being, though we are sometimes a little challenged by people who are brain dead or conjoined twins.  But AI cannot be neatly counted like humans, and therefore the idea of taxing a robot like a person is ludicrous.  And, again, it is a moral hazard.  The companies who decide to employ robotics should be the ones who have legal and economic liability for whatever their robots do.  We should not let them wiggle out of their responsibility because of our love for science fiction or our deep desire to construct perfect, obedient, cyber “children”.

Q9 WENG: What is the definition of “well-designed robots” in your opinion?

BRYSON: A robot is well-designed if the company that produces it is willing to take full responsibility for its actions, and further if governments are willing to license it as safe for human society.  That second requirement is necessary because some corporations have more resources than many countries, so may be willing to accept risks that individual citizens would prefer not to be subject to.

Q10 WENG: Finally, what will be the biggest challenge to developing a human-robot co-existence Society in the following decade?

BRYSON:  We still have many challenges.  I think we will solve dexterity and physical safety before we solve cyber security.  I don’t think we have a full grasp yet about what impact we are already having on human society and human autonomy.  It’s difficult to measure this because so many things are changing at once; we also have the problems of climate change, of massive increases in human numbers and access to communication.  We are currently in a period of grotesque income inequality and political polarisation – some people blame these on AI, but in fact this happened only a century ago, and probably contributed to the two world wars and the great depression. So getting on top of understanding how we are influencing each other and compromising our health with our planets’ is essential to answering this question.

If you enjoyed this article, you may also also enjoy:

• One being for two origins: A new perspective on roboethics
• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• On the ethics of research in robotics, with Raja Chatila
• A European perspective on robot law: Interview with Mady Delvaux-Stehres
• Open Roboethics initiative delivers statement to United Nations CCW

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The concept One Being for Two Origins, from the teachings of Buddha, considers the unification of technology, nature, and human beings. It emerges from the theory of the three aspects: The values of the “Good”, the “Bad”, and “Muki” (Avyakata). The theory of One Being for Two Origins follows the theory of the three aspects and explains a mechanism for how values, such as the good and the bad, emerge in reality using the three aspects. The theory has a contradictory structure, hence it can be difficult for those with a scientific background to fully comprehend it. In this interview, Nagoya University’s Associate Professor Hiroko Kamide shares with us her insights on robot ethics, with a focus on One Being for Two Origins.

Date: December 5^th 2016

Interviewee: Prof. Hiroko Kamide, Designated Associate Professor, Nagoya University, Japan

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan)

Q1 WENG: Thank you for agreeing to an interview with us. Can you tell us a little about yourself and your background?

KAMIDE: My background is social psychology. During my doctoral course, I studied close relationships and well-being. After receiving my PhD, I started researching human-robot interaction and collaborated with engineers to help realize so-called “human-friendly robots”. We established psychological methods to evaluate humanoids, and subsequently developed human-friendly behaviors for robots. I still collaborate with engineers to consider the adaptive way of introducing advanced technology, such as robots and self-driving cars, from an interdisciplinary approach.

 Q2 WENG: Currently you are also in charge of a special interest group on robots and philosophy at The Robotics of Japan (RSJ). Can you tell us something about your RSJ Special Interest Group on Philosophy in Robotics?

KAMIDE: In the RSJ group, we discuss the approach to  technology from an interdisciplinary standpoint. The members are from psychology, philosophy, law, sociology, and engineering. Psychological evaluation, like technological advancement, is important to realize robots that your average person can accept in their daily-life. This collaborative work between psychology and engineering is crucial to developing technologies that can satisfy a human’s needs and expectations. But sometimes we experience terrible events that we cannot control: nuclear accidents, wrongful use of robots, etc. Therefore we started to investigate what the appropriate approach is when implementing and dealing with the cutting-edge technology. It’s a the responsibility of researchers to convey information to ordinary users about how we should deal with technology in order to maintain harmony between technology, nature, and humans.

Q3 WENG: What is the theory One Being for Two Origins? Why is it important to the unification of humanity and nature?

KAMIDE: The theory of One Being for Two Origins has a contradictory structure, and it’s difficult for those with a scientific background to fully understand. At the first step, we begin with a theory of values—the theory of the three aspects—as an introduction to the theory of One Being for Two Origins. The theory of three aspects explains the mechanism of how values, such as good and bad, emerge from the three concepts of the good, the bad, and the “Muki”. Then the theory of One Being for Two Origins follows the theory of three aspects as the fundamental mechanism of order. Through the theory of One Being for Two Origins, we become able to recognize the truth and biases behind language, and as such can avoid conflicts between technology, nature and human-beings.

Q4 WENG: Your paper “One Being for Two Origins – A necessary awakening for the future of robotics” has been presented at the 2016 IEEE ARSO Workshop in Shanghai, China. The co-author is Professor Masahiro Mori, who is a world famous roboticist because of his “uncanny valley” theory. How come you decided to choose this topic? Does the concept One Being for Two Origins have any relationship with “uncanny valley”?

KAMIDE: His theory of uncanny valley has a very thoughtful and groundbreaking perspective, therefore many researchers in the world still refer to his theory. As for my impression, he developed a very clear sense of perspectives that leads us to understand the theory of uncanny valley. His method of observing the truth of the world is in line with the teachings of the Buddha.

Q5 WENG: Regarding regulation for robotics, you pointed out that there is widespread misunderstanding from regulators. What is the misunderstanding? What kind of legal crisis will materialize if they don’t change their way of thinking? How can your proposed theory help solve this legal crisis?

KAMIDE: As the theory of three aspects explains, the value of good and bad are just derived from Muki, that transcends such values and is a true quality of the technology. Useful technology is positive. On the other hand, it would be bad if technology harms people. Technology is fundamentally neither good nor bad. It’s the manner in which technology is applied and operated by humans that is either good or bad. It is humans themselves that attribute these aspects to the technology.

Regulations are attempts to eliminate the bad aspects of technology. However, it is not correct to assume that only the good remains when the bad is eliminated, because technologies themselves are the Muki. And if bad did not exist, good would not exist either, as one is parasitic on the other. To eliminate the bad is to interfere in the working of the Muki in producing these values. The same occurs robots.

Our mindset determines how we approaches the Muki. If we are not mentally prepared, Muki will transform into bad. If we are mentally prepared, we can treat the Muki as good. Bad can transform into good then, of course, good can also transform into bad. And the more good qualities a robot possesses, the stronger its bad qualities will be, following the transformation. Sharp scalpels save lives, but sharp daggers kill people. This is because the working of the Muki is powerful.

In all cases, the mindset of humans is crucial. So, the responsibility of us in robotics is to ensure that technology is not used in bad ways. We must both establish methods of using robots, and work to refine users’ mindsets.

Q6 WENG: Robotics is like a double-edged sword, it can either help or kill people. Robotic lethal weapons present a dilemma. Should they be banned or not? Do you have any comment on this?

KAMIDE: The problem is not in robotics but in ourselves—we who raise such dualistic questions. As I mentioned, the technology of robotics doesn’t have any values. Robotics is not double edged sword at all; our minds are double edged. People easily attribute these values to the physical entity but it is not true. We have to try and observe the fact that we see not what we see, but what we want to see.

Q7 WENG: Do you agree with using dualism as the meta-principle to teach robots right from wrong? If not, do you think it is possible to adopt your theory into creating ethical robots’?

KAMIDE: The “right” and “wrong” is always in the eye of the beholder, and there are no absolute standards. As long as humans accept that they are subject to subjective mechanisms of perceiving the world, it won’t be troublesome to develop ethical robots. But if humans see the “right” and “wrong” as existing independently, objective, it will be definitely problematic.

Q8 WENG: What are major challenges for implementing One Being for Two Origins into robot ethics?

KAMIDE: Clarifying the importance of applying wisdom, not just knowledge. In this theory, knowledge refers to the ability to divide the phenomena into detailed classifications in order to understand the difference between things. On the other hand, wisdom is necessary to discover the meaning in the different things. Knowledge is developed through education, usually, but wisdom is not taught explicitly. Therefore, we are accustomed to dualistic thinking based on knowledge, that divides the world into its constituent parts, but we have difficulty uniting the parts using wisdom, in a way of monism. This is one of reason that we often raise dualistic questions, and eventually develop conflicts.

Q9 WENG: Professor Shigeo Hirose gave an insight called “Morality Engineering” in his short review “A Dialog on Robots” Vol. 7, No. 4, JRSJ. Do you think that One Being for Two Origins might be helpful in implementing his proposal of “Morality Engineering”?

KAMIDE: His article is very interesting. Opinions from both sides who are optimistic or pessimistic to a robotized future society are quite realistic and thoughtful. Recently many philosophers have started to think about morality in robotics. His “Morality Engineering” suggests implementing “rational egoism”, and I agree with the importance of this approach. One Being for Two Origins is the methodology to transcendent and unify opposite aspects of the phenomena to see the truth. So it is helpful to consider how “rational egoism” truly functions in the interactions between humans and robots.

Click here to read Hiroko Kamide’s paper “One being for two origins — A necessary awakening for the future of robotics”.

If you enjoyed this article, you may also also enjoy:

• Considering robot care, ethics & future tech at #ERW2016 European Robotics Week
• On the ethics of research in robotics, with Raja Chatila
• A European perspective on robot law: Interview with Mady Delvaux-Stehres
• Open Roboethics initiative delivers statement to United Nations CCW

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Hiroshi Ota and Minako Inoue with 2 Robovie R3 robots in Oriza Hirata’s “I, Worker”

How can robotics help to enhance the development of the modern arts? Japan’s famous playwright, stage director Oriza Hirata and leading roboticist Hiroshi Ishiguro launched the “Robot Theater Project” at Osaka University to explore the boundary between human-robot interactions through robot theater. Their work includes renditions of Anton Chekhov’s “Three Sisters”, Franz Kafka’s “The Metamorphosis”, and their own play “I, Worker”. Their work has spread internationally to Paris, New York, Toronto and Taipei.

For this interview, we would like to invite their collaboration partner Yi-Wei Keng, director of Taipei Arts Festival, to share his insights on the intersection of robotics and the arts.

Interviewee: Mr. Yi-Wei Keng, Artistic Director of Taipei Arts Festival (TAF)

Interviewer: Dr. Yueh-Hsuan Weng, Co-founder of ROBOLAW.ASIA Initiative and Fellow at CLAST AI & Law Research Committee (Beijing) and Tech and Law Center (Milan)

Interviewer: Ms. Diane Chiang, Guest Editor at ROBOLAW.ASIA Initiative

WENG: Thank you for agreeing to an interview. Can you tell us a little about yourself and your background?

KENG: I have diverse backgrounds. I studied civic engineering in my first year at university and changed my focus to philosophy in my second year. After graduation and military service, I went to Prague to study non-verbal theater. I dedicated myself to disabled theater when I came back from Europe at the beginning of 2000s’. This multi-discipline experience helped me to think of arts in broader sense.

WENG: Can you introduce Taipei Arts Festival to our readers? What are the collaborations between Taipei Arts Festival and Robot Theatre project in recent years?

KENG: Taipei Arts Festival is the city’s festival, and was established in 1998. TAF gets its entire budget from the Taipei city government. I became the artistic director in 2012 and invited “Three Sisters, Android version” by Oriza Hirata and Hiroshi Ishiguro in 2013. Professor Hiroshi Ishiguro also visited TAF and gave public talks about his robotics work. Then in 2015 we invited their “La Metamorphose version Androide” to TAF.

WENG: What caused you to collaborate with Mr. Oriza Hirata? Did you face any challenges in implementing the piece into Taipei Arts Festival?

KENG: Well, it was really an accident. My wife read an article about the Robot Theatre Project in Monthly Art Magazine Bijutsu Techo. She told me about this and I did some research. I thought it was an interesting project, then I found Seinendan through my connections in Japan. Oriza Hirata gave a very positive response about our invitation. Everything went well during the festival, that is why we decided to invite Seinendan again in 2015.

WENG: Taiwan’s choreographer Yi Huang had used his KUKA robot companion to design his “Huang Yi & KUKA” robot choreography. Can you introduce more examples of robot choreography to us?

KENG: For example, French company Cie 111 made the “Sans Objet” in 2009, it was also a dance show with 2 dancers and an industrial robot, like Yi Huang’s KUKA project in 2013. French choreographer Blanca Li made the “Robot” in 2013 where the NAO Robot by Aldebaran Robotics to dance with real dancer.

WENG: In Japan, an art theater in Tokyo performed a theatrical rendition of Osamu Tezuka’s “Black Jack”. The character “Pinoko” was played by a robot created by Waseda University’s Takanishi Laboratory. The arms, legs, and body of the Takanishi Laboratory developed robot are detachable and equipped with 25 motors, so the robot can vividly resemble the Black Jack character. With your experience and expertise in performance art, what are your thoughts on this?

KENG: What surprises people most about robots or puppets is that we know they are objects but they act like they have soul inside. But if there were many robots in our life, then robots on stage would not be such a surprise. What matters is when the robot looks like a human being, then robot theater loses its meaning too because people can’t tell the difference from appearance. There are more and more animation films, but there is a psychological need to see real humans in films. That is what happens in animation film, we don’t want to see the animation, which shoots like the real movie, we want to experience the pleasure of difference and surprise.

CIANG: Is there a correlation between robotic theater and anthropology? If so, what is it? Can you share your insights from a puppetry aesthetics perspective?

KENG: As I say, if a robot looks like the real people on stage, then it loses its attraction for the audience. It is the same for puppet theater: the movement and design should remain dumpy and eccentric. In the future, androids in everyday life may confuse people if we can’t tell the difference between robots and human beings. We might kill someone which we declare as a robot, but in fact is a human being. Why is robot theater important? Because we could use theater to show the fictional situation which may happen in the future and discuss its meaning and possibility through story.

CIANG: In the 2016 Taipei Film Festival, there is a movie featuring a robot called “Sayonara”. This movie is an extension of Hirata and Ishiguro’s Robot Theater Project. They use an android as an actress in the movie. To human actors, what is the difference between 3D robotic theater and a 2D movie when they play with the android?

KENG: When you see the robot in a theater you confront the reality, and the statement from story becomes stronger than in the movie. There is a life event in theater but film is an illusion made by projecting lights on a screen.

CIANG: Can you tell us something about ethics debates in robot theater? Why do ethics matter to robot theater?

KENG: The main issue is the role of actors in theater. In 1908, theorist Gordon Craig declared that the Über-Marionette should replace the real actor in the future. But acting is not only imitation but also a creative process. Ethics about robot theater can help us to understand more about the essence of theater, robots and humanity.

CIANG: Do you think that it is possible to improve the performance of robots in society by borrowing experiences from performance theory?

KENG: If a robot wants to become human, they must look like a human and act like a human. So “act” has something to do with acting. According to the theory of mirror neuron, the audience can react with the acting from the stage, because their body acts the same way when similar situations happen, so the audience reacts. Acting technique divides from inside-out like Stanislavky system and outside-in like the Delsarte method. The performance theory of François Delsatre might help better than the inside-out technique since robots don’t act by feeling.

WENG: Researchers at the University of Oxford found that 47% of U.S. jobs are at risk of AI and automation in the next two decades. Do you agree that in the field of arts and drama, many future job opportunities will be replaced by robots? Or do you rather believe that new technology will create more opportunities for humans?

KENG: There are many puppet theaters in different cultures, and there exist theaters with real actors in these cultures too. There is no contradiction between having a robot actor and real actor. The emergence of robot actors could create a revolution in acting theory. The problem is not technology, but how we treat them.

If you enjoyed this article, you might also be interested in:

• A European perspective on robot law: Interview with Mady Delavaux
• Art meets robotics in Patrick Tresset’s OPLINE Prize entry
• Robots Podcast #171: Robotics in theatre, film and television, with Grant Imahara and Richard McKenna
• Flying robots perform 100th show on Broadway, using new localization technology and algorithms
• RoboThespian gets its own YouTube channel

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Raheeb Muzaffar, an information technology specialist, has developed an application-layer framework that improves the transmission of videos between moving drones and mobile devices located at ground level. His work within the Interactive and Cognitive Environments (ICE) doctoral programme will be completed soon. Raheeb explains what makes this technology innovative and talks about his plans for the future in a conversation with Romy Mueller.

Raheeb Muzaffar was already aware of what to expect when he arrived in Klagenfurt as a PhD student, travelling from Islamabad, the capital of Pakistan, December 2012: He’s travelled well before, had experience working with various universities and professional organizations, and was acquainted both with the European culture and the research environment. The doctoral programme “Interactive and Cognitive Environments (ICE)” suited him perfectly, allowing him to conduct research in an international environment. Doctoral students accepted into the programme are funded for a period of three years. During this time, they work at two universities. The research teams at the Alpen-Adria-Universität and the Queen Mary University in London offered Muzaffar, who wanted to work on the communication between drones, an environment that allowed him insight into various laboratories and research approaches. By combining the key research areas he experienced in Austria and Great Britain, he ultimately arrived at his very own topic: video transmission in drone networks.

He spotted a problematic issue in this context, which he has studied very closely over the past few years. “In a situation where several drones are airborne and are transmitting videos to several units located at ground level, we cannot currently guarantee that the footage will be reliably transmitted.” The problem is that the receiving devices do not have a feedback mechanism for the video packets being transmitted from the drones. The missing feedback mechanism causes unreliability rendering a distorted video reception. Also, when the transmitting devices and the receiving devices are in motion, the wireless transmission conditions are subject to continuous change, introducing an additional factor of uncertainty.

To address this problem, Raheeb Muzaffar has developed a so-called “video multicast streaming framework”, which allows feedback from multiple ground devices to the drones to add reliability allowing a smooth video reception. However, the framework is even more capable: “The drone does not only learn whether the video has arrived, but thanks to the feedback mechanism, the drone can adjust the transmission rate and the video quality to match the prevailing wireless conditions. Thus, the transmitting drones can re-send lost video packets following the transmission and video rate adaption.” Video communication between drones and receiving devices is useful for various applications such as search and rescue, surveillance, and disaster management.

When we asked him, if he can make profitable use of his technology, Raheeb Muzaffar laughed. At various conferences, he talked to the developers from the industry, who confirmed to him that modules endowed with similar functionality are expensive and that the WLAN protocol requires modification. However, his application is freely available. Anyone wishing to commercialise the technology as a product, is able to do so. In any case, over the course of numerous simulations and experiments, he has demonstrated that the streaming framework works and uses the conventional WLAN protocol without modifications.

Muzaffar’s research in Klagenfurt is coming to an end. By November, he will complete his doctoral thesis, and will pull up stakes in Klagenfurt, following his hopes and dreams to work in the research-based industry. The researcher was accompanied by his wife when he came to Klagenfurt. In the meantime, they celebrated the birth of their daughter. The Muzaffar family intends to travel more: “I am not tied to one location. I would enjoy working anywhere in Europe, but we also like the idea of moving to New Zealand.”

A few words with Raheeb Muzaffar

What career would you have chosen, if you had not become a scientist?

If I would not have become a scientist, I would have become a social worker. Before turning to research, I worked with Non-governmental organizations including iMMAP and UNICEF thinking that my technical expertise could help expedite the social work that the organizations are involved in. Besides the work I do plan to involve myself in some sort of social work may it be technical, educational, financial or emotional.

Do your parents understand what it is you are working on?

Yes, especially my father, he is quite interested in technology. He may not have the technical details but he is aware of research and the work I am involved in.

What is the first thing you do when you arrive at the office in the morning?

First thing in the morning is to check and answer my emails. Thereafter, I work by the plan. I always have a plan for the week/month regarding the tasks I need to do. Although sometimes things don’t go by the plan and I have to readjust the plan to meet deadlines.

Do you have proper holidays? Without thinking about your work?

Yes, at least once a year.

Why are so many people afraid of technical sciences?

In my opinion, every individual has a different aptitude to his/her work domain and one should do what one thinks he/she is interested in and can perform best. Having said that, working in the domain of technical science is quite challenging. The field has advanced and it is getting increasingly complex to even gain the basic understanding of different domains. To work in this field, one has to stay up to date. Developing something new involves a minimal margin of error. I think, people are afraid due to the complex nature of the studies that involve an extra bit of hard work, understanding, and patience.

Written by: Romy Müller, University of Klagenfurt

Originally published in German language in AAU News of the University of Klagenfurt, Austria. Also published at Medium.com.

In this episode, Audrow Nash interviews Fredrik Gustafsson, Professor in Sensor Informatics at Department of Electrical Engineering in Linköping University, about an initiative to reduce poaching in a rhino sanctuary in Ngulia, Kenya. Gustafsson discusses how he first became involved in this project, how he has worked with the rangers to develop solutions, and the future of this work.

Fredrik Gustafsson

Fredrik Gustafsson is professor in Sensor Informatics at the Department of Electrical Engineering, Linköping University, since 2005. He received the M.Sc. degree in electrical engineering 1988 and the Ph.D. degree in Automatic Control, 1992, both from Linköping University. His research interests are in stochastic signal processing, adaptive filtering and change detection, with applications to communication, vehicular, airborne, and audio systems. His work in the sensor fusion area involves design and implementation of nonlinear filtering algorithms for localization, navigation and tracking of all kind of platforms, including cars, aircraft, spacecraft, UAV’s, surface and underwater vessels, cell phones and film cameras for augmented reality. He is a co-founder of the companies NIRA Dynamics (automotive safety, including tire pressure monitoring systems found in more than 20 million, here), Softube (plug-ins for music studios and software solutions for Marshall and Fender), and Senionlab (leading manufacturer of indoor navigation solutions for sm, here).

Links:

• Download mp3 (16.9 MB)
• Project Ngulia website
• A PDF overview of Project Ngulia
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

We get some insight into Linear Dynamical Systems, which the Datta Lab at Harvard Medical School is doing some interesting work with. Plus, we take a listener question about using video games to generate labeled data (spoiler alert, it’s an awesome idea!)

We’re in the final hours of our Fundraising Campaign and we need your help! 

In this episode, Audrow Nash speaks to several robotics companies at the company showcase at RoboBusiness 2014, which took place in Boston, Massachusetts.

The following is an ordered list of the people interviewed.

1. Norm Domholt (1m 40s), CEO of NPC Robotics,  talks about their modular remote operate vehicle Black Dog.
2. Annemarie Laudanski (6m 20s), Lead Field Applications Engineer at Xsens, demos their inertial motion capture system.
3. Carl Vause (9m 14s), CEO of Soft Robotics, speaks about the capability of the soft robotic manipulators, which can be used to pick fruit, among other applications.
4. Konrad Fagertun (13m 56s), COO of NLinks, tells about their new robot which drills holes in ceilings – a laborious and difficult task for humans.
5. Asim Ikram (15m 51s), CEO of Neo Robotics, discusses the market for robots in the Gulf Cooperation Council region.
6. Harri Valpola (21m 40s), Head of Technology at Zen Robotics, talks about the companies waste sorting robotic system.
7. Dick Zhang (26m 15s), CEO of Identified Technology, speaks about their self-piloting aerial drone systems for automated energy, mining, and construction site monitoring, surveying, and mapping.

Links:

• Download mp3 (18.8 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Transcript

Norm Domholt (1m 40s), CEO of NPC Robotics:

Audrow Nash: Hi, can you introduce yourself and your company?

Norm Domholt: Yes, thank you. My name is Norm Domholt and I’m president of NPC Robotics. Today, at the show, we’re featuring and debuting our latest invention. We’ve been working on it for three years and it’s called the Black Dog.

Audrow Nash: Now, you have the Black Dog here. Can you describe it?

Norm Domholt: Black Dog is a remote control mobile platform. It’s designed modularly, so that it can be repaired in the field without probing with any sort of instruments and with the use of one particular tool – a quarter inch hex wrench. All the parts can be taken apart and replaced in minutes.

Audrow Nash: Can you tell me a bit more about how it’s modular?

Norm Domholt: We realized, through our experience with Battle Box, that the teams that seemed to win were the teams that designed their robots to be repaired quickly and without the need to take apart wiring.

It’s something that can be plugged in and replaced quickly. So, we decided that there should be around eight components in here that can be replaced with the use of this one tool and they should all be modular. There are no internal wires that you have to worry about, so everything unplugs and plugs together very quickly.

Audrow Nash:  So it’s a trunk-treaded robot that has two motors driving everything and it’s three feet long, a foot tall and two feet wide?

Norm Domholt: Yes, 42 inches long, 22 inches wide and 12 inches tall. It weighs 200 pounds and it goes four miles an hour. It’s got a lot of torque. Each motor is two-and-a-half horsepower – that’s a 24volt lithium battery.

Audrow Nash: Can you give me some examples of how it could be fitted with different components in the field?

Norm Domholt: Yes. Let’s say a SWAT team buys our robot power base; there are many different applications and mission-specific needs that they could have throughout a day. For example, they go to a house with something suspicious going on inside. They can put a telescoping camera, called RibbonLift, on the robot, within – say – ten seconds. They can then maneuver it to the building from a safe distance. If they hear shots fired and see there’s no need for the camera, they can pull the robot back within seconds, take their camera off and put a 300 pound bullet-proof shield on instead. Then, a SWAT person could operate this with a thumb throttle, while crouching behind the shield, and approach the house from a safe position. He can direct it to come back and hurl a teargas canister through the window. It could be adapted to any particular use very quickly.

Audrow Nash: Very good, and can you tell me at what stage of production your company is?

Norm Domholt: We’re debuting this today. We were hoping it would be a little further along and we could operate it. Some of the electronics are not completed yet; the mechanics are almost done and the modularity is complete. The technology that’s going in is already in place and, in another two months, we’ll be testing at facilities – breaking down doors, going through hallways, up and down stairs. We’ll be testing to see how far – wirelessly – this thing can go into a building.

Audrow Nash: Who would be interested in this product?

Norm Domholt: The same people who would be interested in a multi-purpose tool. Police can use it, the home-owner… Who would use a power-base? SWAT, the oil industry, hazardous waste…

Audrow Nash: So, lots of applications?

Norm Domholt: A lot of applications, yes.

Audrow Nash: Thank you.

Norm Domholt: You’re welcome, thank you.

Annemarie Laudanski (6m 20s), Lead Field Applications Engineer at Xsens:

Audrow Nash: Hi, can you introduce yourself and tell me about your company?

Annemarie Laudanski: Hi, my name is Annemarie Laudanski and I work for Xsens, we do animation promotion capture.

Audrow Nash: Can you tell me about what you have set up at this booth and describe it?

Annemarie Laudanski: Today we’re trying out our inertial motion capture system. It’s a full body- measure-capture system built of 17 inertial sensors. In each of these, there are 3-D accelerometers, gyroscopes and magnetometers. They’re used to measure the movement of arms and legs and send information wirelessly to the computer, where it’s translated into a bio-chemical model. So, we can measure exactly what someone is doing and portray that on our software.

Audrow Nash: At the booth you have a dancer?

Annemarie Laudanski: We do, yes. We hired Ashley. She’s the cheerleader for the Boston Celtics and she’s with us during the conference to showcase the use of our system.

Audrow Nash: You mentioned you were using exonorometers, gyroscopes and Megnatometers. So, it doesn’t require any external visual system to track them? You’re just taking the raw data from all of these sensors, running that through, and then feeding it to a model, to develop a person?

Annemarie Laudanski: That’s right. The exonorometers measure how fast each limb segment is moving, the gyroscopes measure the rate of turn of the limb and then we use the megnatometers to correct the drift in the other sensors. A fusion of these three sets of sensor data is sent wirelessly to the computer where it’s combined with our calibration procedure to build a human model. The calibration takes an average reading from the sensors, while the subject is standing still for five-seconds. This is a known position for taking sensor data, that then informs each of the trackers, so that the model is built correctly.

Audrow Nash: I see. As the dancer moves, it’s happening in real time.

Annemarie Laudanski: Yes, all of it is done in real time so all of the sensors are sending data at 30 bites per second to the computer and everything is shown with a latency of, at most, ten milliseconds, which is mostly due to the Microsoft operating system we are working with.

Audrow Nash: What are some applications?

Annemarie Laudanski: The applications are really wide for this type of technology. They go from entertainment purposes, where people are using it for movie and game design, to movement science, where it’s used for sport, rehabilitation or ergonomics and also human-robot interaction. So, you can even have someone in the system and driving a robot, or building up a database of human movements that can be used to program one and allow it to move like a human would.

Audrow Nash: Thank you.

Annemarie Laudanski: Thank you.

Carl Vause (9m 14s), CEO of Soft Robotics:

Audrow Nash: Hi, can you introduce yourself and the company?

Carl Vause: I’m Carl Vause and this is Soft Robotics Incorporated. We’re a Boston-based company that got spun out as a research group at Harvard about a year ago. We’re taking soft robotics technology and making it real, producing commercial products to solve new problems across a variety of markets.

Audrow Nash: Can you tell me about what you’ve displayed at the booth?

Carl Vause: At the booth today we have two soft hands: A very large hand for handling pumpkins, shoe bags and objects of very weighty geometry, and a small-handed, or short, robotic arm. What we wanted to show here is that, currently, there are a lot of problems with robotic manipulation. Manipulators don’t do well with soft, curvy, easily damaged objects and they don’t do well in unstructured gripping scenarios.

What we have solved using soft actuators is the ability to pick up anything with this arm, from a prickly cactus, which you would never pick up with your bare hand, to a racket ball or a cupcake. We want to show that, with almost no control system, building a very clever, soft actuator can solve a problem that has been vexing robotics: How to develop a gripper that is as adaptable and conformal as a human hand. People are spending millions of dollars to solve this problem using traditional hard robotics.

Audrow Nash: Can you describe the hand and the smaller robotic arm?

Carl Vause: One is a four-finger hand. It’s based on proprietary actuator geometry. We use vacuum to curl the fingers out of the way and they pressurize to create a grasp. We have a three-finger gripper as well. It actually has fingernails, so it can access a low profile object like a circuit board.

Audrow Nash: Can you tell me some potential applications of these soft hands? I know there are a lot.

Carl Vause: A lot, but how do you interface a robot with something that has an unknown shape? Or when the position is unknown? Or something easily damaged? Agricultural processing is interesting. Picking lettuces heads, for example. There’s a danger of introducing E. coli into the food chain, so washing and cleaning lettuce. Robotics companies and the US Food and Drug Administration want to automate that task. But robots today can’t handle lettuce, because their heads vary in size, shape and weight, and they are delicate objects. So, one of the things that we really wanted to do with our soft robotic technology is unlock that market for automation, and automate data handling.

They can even work in wet conditions; there are no electrical connections here. You can sterilize them and wash them down with bleach at the end of the day. It really does bring automation into the agricultural process. It makes it a reality.

There’s also a light-weight version for working side-by-side with a human collaborator. In a lab, for example where you need to be careful what you touch. A manipulator can pick up a range of lab instruments, or tools, and hand them safely to you. It has the credentials of collaboration.

Audrow Nash: The ones that you have on display are on open feedback loops. As in, they don’t sense what they’re picking up, they just actuate. You’ve talked a little bit about embedding sensors into the fingers in the hands. Can you tell me more about that?

Carl Vause: We can easily embed sensors to give an object grasp detection. We can embed it in our proprietary manufacturing process, in simple off-the-shelf sensors and feed them back into something very basic. If you want it very simple, take an Arduino and provide passive grasp detection. That’s very simple.

Audrow Nash: Thank you.

Carl Vause: Alright.

Konrad Fagertun (13m 56s), COO of NLinks:

Audrow Nash: Hi, can you introduce yourself and tell me about your company?

Konrad Fagertun: Yes, my name is Konrad Fagertun. I’m a co-founder of NLink Robotics. We’ve come all the way from Norway to Boston, to showcase the world’s first mobile drilling robot. We’re excited to be here at Robobusiness, showing what we’ve been working on for the last two years.

Audrow Nash: Can you describe the robot to me?

Konrad Fagertun: Yes. The challenge in the construction industry is hard labor – manual work – and the hardest of all is drilling holes in concrete ceilings. So, we made a mobile drilling robot. It comprises a mobile lift – that can drive around – and, on top of that, is a robotic arm holding a rotary hammer drill. It thrusts that up to the ceiling and can drill holes wherever you want. It uses lasers and cameras to position itself, so it can drill to a millimeter’s precision. This can be done hundreds of thousands of times, say for a large shopping mall or an office building. It’s repetitive work.

The robot is a little bit bigger than a man, and can reach several feet high, so you don’t have to get up on scaffolding or stairs to do the work; the operator can stand on the ground, safely at a distance. But, if he wants to get close, no problem, the robot is certified to work with humans, so it doesn’t matter if it crashes into your arm, it won’t hurt you and it’ll stop. It’s useful for electrical contractors, plumbers, mechanical engineers…

Audrow Nash: Thank you.

Konrad Fagertun: Thanks.

Asim Ikram (15m 51s), CEO of Neo Robotics:

Audrow Nash: Can you introduce yourself and your company?

Asim Ikram: Yes, my name is Asim Ikram and I’m the CEO of Neo Robotics in Dubai.

Audrow Nash: What do you guys do?

Asim Ikram: We’re a new start up in Dubai, not limited to robotics. In that region, they are not exposed to robots at the moment. We are starting at the lower level, to introduce smart apps and ICT, Information Communication Technologies. Then, we want to switch to robots, because the customers are the same.

Audrow Nash: So, basically you’re acting as a liaison between companies in the US and outside for bringing robots into Dubai?

Asim Ikram: Into the GCC region – the Gulf Cooperating Countries. There are six of them in the Middle East:  Saudi Arabia, Bahrain, Oman, Kuwait, Emirates and Qatar. They have a strong focus on technology, they’re teaching robotics and megatronics and have labs at almost every university. The industry will develop but, at the moment, there’s no pull from for robots.

Audrow Nash: What are some of the more popular technologies that are being brought to Dubai?

Asim Ikram: Not so much industrial robotics, but there are new production lines coming up that have industrial robots in the blueprint. Those lines should start operating in six months. The financing and planning is already done. Healthcare and education, that’s much further ahead. On the education side, there’s big competition between the private schools that are teaching things like systems programming. Now they’re also trying to get robots into the classrooms and teach with them. There’s a lot of movement there.

Audrow Nash: What robotics opportunities do you think there are in Dubai for engineers and entrepreneurs from the rest of the world?

Asim Ikram: There’s a lot of work in real-world drone applications. Already, highway police are using drones. They’ve also started making some national identity card deliveries with drones. Real applications of technology are moving fast. On the security side, I’ve seen a lot of civil defense robots for bomb disposal.

Audrow Nash: How did you decide to take this opportunity and form a company?

Asim Ikram: I’ve been working in this market for some time and I’ve been seeing a lot of pull in different areas. Healthcare and education are moving very fast and the industrial side will come up in the next one or two years. There are a few robotics service companies, but not many locally so I found a gap. It’s a good time to start something.

Harri Valpola(21m 40s), Head of Technology at Zen Robotics:

Audrow Nash: Can you introduce yourself and your company?

Harri Valpola: Yes, hello. I’m Harri Holopainen. I’m one of the founders of Zen Robotics Company in Helsinki, Finland. Currently, our main application is robots that have learnt to pick up waste. A little bit like WALL-E…

Audrow Nash: It looks a little different than WALL-E. Can you describe the robotic system you have?

Harri Valpola: Whereas WALL-E is cute, a small Hollywood robot, our robot is more like an large industrial scale model. It doesn’t look like a human; it’s not cute, but it works very efficiently. It’s powerful. It can lift and throw objects that weigh up to 20 kilos – 40 pounds – like they were nothing.

Audrow Nash: It looks like a conveyor belt and has robotic arms that move around and pick objects off of the conveyor belt and throw them to sort them. Am I correct?

Harri Valpola: Yes. It works in a similar way to human hands. All around the waist is a conveyor belt. The robot recognizes the objects and sorts them, exactly as a human would do, but faster. It’s also stronger and it doesn’t break.

Audrow Nash: What kind of sensors are you using to identify the material? What waste are you sorting now?

Harri Valpola: We are currently focused on construction and demolition waste. The sensors we are using are tailored for that particular waste. The most important material we’re sorting is  wood, followed by mineral-based parts and metal. We’re using sensors that can discriminate between these. They include metal detectors and spectrometers.

Audrow Nash: Where have you implemented the system and what sort of feedback are you getting from customers?

Harri Valpola: Currently, we have three customers in Europe; two in Finland and one in Holland. We’ve been developing this system for two years and now it’s working and customers are happy. We’re ready to scale up, now we have the product, so that other customers can start making use of this solution.

Audrow Nash: What’s the benefit of sorting the waste?

Harri Valpola: In Europe, at least, there are many companies that are floating in waste. Population densities are so high, there’s no place to put it. The only real option is to sort the waste. There are many valuable raw materials there. Sorting stones is not important but there are valuable metals, like copper and aluminum, and also wood. That’s particularly useful in Holland because they don’t have forests, and it can be recycled and used as a construction material.

Audrow Nash: Thank you.

Harri Valpola: Thank you.

Dick Zhang (26m 15s), CEO of Identified Technology:

Audrow Nash: Interviewer:  Can you introduce yourself and tell me about your company?

Dick Zhang: My name is Dick Zhang from Identified Technologies. We build and scale drones for commercial data capture. We work with construction and energy industry clients and help deliver data and insights. We get this information either by deploying robots on the ground, or by utilizing commercial planes and helicopters. It’s very expensive, it’s very dangerous and it’s very slow.

We deliver insights by collecting massive scale amounts of data on what has happened in the past, what’s going on now and what’s going to happen in the future, through predictive capabilities.

We collect the data via a three-part system: an autonomous drone, a dock station, that swaps batteries and uploads data, and a client dashboard that can be used to interact with and control the system from the cloud, from anywhere in real time.

We’ve been working with really exciting partners in energy and construction for the last six months. I run the business, look for exposure and develop partner and client relationships.

Audrow Nash: Can you tell me a bit about the demonstration – the drone, the quadcopter – you have on your booth?

Dick Zhang: The drone itself, everything inside, in terms of hardware, is really off-the-shelf components. Our dock station, that swaps batteries and outputs data, that’s all proprietary mechanical design and hardware. The software system is also proprietary. So a lot goes into flight control – safety checks, process checks, maintenance, things like that.

Audrow Nash: Swapping the batteries, is that automated?

Dick Zhang: Yes, that is automated. On the docking station, there’s a carrousel with six batteries that are always charged. It inserts another battery when the drone needs to fly.

To tell you a story, what inspired all of this was a talk with one of our clients. They run 5,000 job sites around the world. They bought an Internet tool but had no idea how they were going to train 1,500 people to use it. So I automated a system and made it self-operating and self-sufficient. We were able to employ 1,500 really rapidly across the country and across the world.

Audrow Nash: Is it totally self-sustaining?

Dick Zhang: Yes, right now all you need is a standard power plug. Future generations will evolve generators, solar panels and wind fans. But, apart from the source of power, everything happens on its own. It flies itself, captures data and swaps batteries.

Audrow Nash: What kind of navigation do you use on the drones?

Dick Zhang: Navigation in drone technology has become really sophisticated over the last couple of years. We have a combination of GPS and inertial sensors on board, so that, three or four hundred times every second, the drone is saying to itself: “I need to be in position XYZ. Right now I’m not there, but I’m close. Let me get there.” So, four hundred times every second, it’s making these corrections and adjustments so that it can fly along the path that the computer programmed.

Audrow Nash: Other than the IMU and the GPS, what other sensors do you use? Do you have a vision system for landing on the platform that swaps the batteries, and some other sensors?

Dick Zhang: Yes. Now we are working with cameras. We’ve done some early tests with our partners on infrared and gas effect. We do a lot of work in energy. As part of this start up growth, we’re going through a crafty process of crawl, walk and run. It’s the vision that enables the most capabilities, to help our clients right now, with the resources that we have. Now, to touch on how we land, that’s a huge problem. So, we built a proprietary vision system based on ultrasound. There’s a beacon and a dock and a transceiver, where you listen and triangulate.

Audrow Nash: Okay, so you use GPS to get it close enough…

Dick Zhang: Exactly.

Audrow Nash: Then the ultrasound takes over by localizing the sound somehow?

Dick Zhang: Yes, exactly.

Audrow Nash: Okay, thank you.

Dick Zhang: That was awesome. Thank you.

All audio interviews are transcribed and edited for clarity with great care, however, we cannot assume responsibility for their accuracy.

By: Vanessa Bleich
Next year’s Cybathlon will host people with physical disabilities equipped with advanced assistive devices including robotic technologies. The goal of this unique competition is to remove barriers between people with physical disabilities, researchers and the general public and to promote the development of assistive technologies that are useful for daily life. We talked to Robert Riener, the Cybathlon’s main organizer and Professor of Sensory-Motor Systems at ETH.

Robert Riener is the initiator and organizer of the Cybathlon. (Photo: ETH Zurich / Giulia Marthaler)

You are the initiator and main organizer of the Cybathlon. How did the idea for this event come about?
As a researcher, I’m always on the lookout for opportunities to present my research to the general public. A few years ago, I read an article in the newspaper about a man who climbed the stairs at the Willis Tower in Chicago using a motorised prosthetic leg. It inspired me to think about a similar event that could be held here in Switzerland – an event that would extend beyond a single race to include many other disciplines. Through talks with colleagues and employees, I gradually developed the idea further. I also thought increasingly about what exactly we wanted to communicate and achieve with this event.

What do you hope to achieve through the Cybathlon?
Our original idea, which was to present our research findings to a wider public, of course remains a goal that we continue to pursue. But we would also like to raise people’s awareness of the needs of people with physical disabilities and the obstacles they face in everyday life. Unfortunately, many people without disabilities know very little about the challenges faced today by people with physical disabilities. Alongside technical assistance, for example, accessibility is also extremely important. We believe this knowledge, in turn, can break down more barriers.

Besides overcoming reservations about the field, we want to motivate developers to create technologies that provide real support to people in daily life. That is why the races feature everyday obstacles, such as stairs, ramps and doors. We hope Cybathlon’s competitive nature will boost the development of powered assistive technology in the long term. It’s already having a positive impact: new student groups have been formed and research projects have emerged. Both students and researchers are working at full speed on their prototypes.

What are the main difficulties when developing electrically powered devices?
The weight-performance relationship is one of the greatest challenges. For example, if you want to build a wheelchair that climbs stairs, it must be wider for stability reasons. When combined with the additional mechanics and propulsion system, it’s automatically heavier. The same goes for the development of the exoskeleton. The integrated batteries must be very efficient to allow the wearer to move freely and independently throughout the day, but these batteries weigh a lot.

Cybathlon

The Cybathlon will take place on 8 October 2016 at the Swiss Arena in Kloten. People with varying degrees of disability, aided by cutting-edge assistive technology, will participate in the unique competition. A team is comprised by a technology provider and at least one pilot who directs the technological device. As a result, two medals await each winning team at the Cybathlon: one for the parathletes and another for the provider. The technological device that is deployed can either be a product that is already available on the market or a prototype from a research lab.

The competition features six different disciplines:

• Powered leg prosthesis race
• Powered arm prosthesis skills course
• Powered exoskeleton obstacle course
• Powered wheelchair race
• Functional electrical stimulation bike race
• Brain-computer interface virtual race

Robert Riener, head of ETH Zurich’s Sensory-Motor Systems Lab, is the initiator and main organizer of the Cybathlon.

Are there other challenges besides the power supply?
While the performance data for the powered leg prosthesis is already very good, the control is more problematic. The wearer’s intended movements must be correctly conveyed to the leg prosthesis. This means that the prosthesis should automatically detect what the person wants to do, be it standing up, sitting down or climbing stairs. This is less of a problem when it comes to conventional, passive prostheses because the wearer intuitively propels them by moving or swaying.

What is the current state of research in the area of powered assistive technology?
The technology is already quite advanced today in some areas. For example, it is now possible to run quicker with certain leg prostheses than a human with natural legs could. But such prostheses are only helpful for specific movements. You may be able to walk quickly, but not be able to sit down, walk up stairs, or drive a car. Most of the available aids have significant deficits with regards to everyday functions and potential users have therefore failed to embrace them. This is where the idea behind the Cybathlon comes in: systems need to be developed that provide help to people with disabilities where they need it the most – in daily life.

How does the Cybathlon differ from other competitions such as the Paralympics?
People with disabilities can participate and compete in both events. The Paralympics, however, is primarily a sporting competition. Technology remains in the background and the use of motorised wheelchairs or powered prostheses is not permitted. As a result, many people are unable to compete. At the Cybathlon, on the other hand, «pilots» with greater restrictions who primarily direct the technical aids – are also eligible to participate. Although people are at the centre of our event, technology also plays a relatively large role.

The Paralympics are focused on sporting disciplines. In addition to races honouring the athlete who reaches the finish line first, the Cybathlon competition also includes skills and obstacle courses featuring everyday tasks and barriers. It’s not only the time that counts; pilots also receives points for their skills. Just like in classic sporting events, competition drives both the pilots and the technology developers at the Cybathlon. They inspire each other to deliver their best performance.

What has the response been to the planned event up until now?
There’s been a great deal of interest in the event. We have received very positive feedback from the scientific community, the media and well-known Swiss organisations for the disabled. The disability associations share our view that the Cybathlon could contribute to the development of current devices. Some critics, however, argue that these sorts of efforts should be focused exclusively on making buildings and the environment barrier-free. But the world does have barriers and you can’t just make them disappear – there are hills, forest paths and so forth. Researchers are developing technologies so they no longer pose obstacles in the future.

• Novel robots for gait and arm rehabilitation: Interview with Robert Riener
• Introducing the Cybathlon
• Registration now open for Cybathlon 2016

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In this episode, Audrow Nash interviews Christina Brester, from the Siberian State Aerospace University, about her research on a method to identify emotional state from speech. This method performs speech analysis with a self-adaptive, multi-objective, genetic algorithm for feature selection and uses a neural network to classify those features. In this interview, we’ll discuss exactly what that means, as well as the implications and future of this research.

Christina Brester

Christina Brester completed her bachelor’s (2012) and master’s degree (2014) at the Siberian State Aerospace University (SibSAU) in Krasnoyarsk, Russia. Her master’s thesis was on Speech-based Emotion Recognition.

Currently, Brester is a PhD student in Computer Science and Engineering at SibSAU. Her research interests include evolutionary computation, neuro-evolutionary algorithms, machine learning, and speech analysis.

Links:

• Download mp3 (10.5 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Link to One of Brester’s Related Publications

We explore how to use Determinantal Point Processes. Alex Kulesza  and Ben Taskar (who passed away recently) have done some really exciting work in this area, for more on DPPs check out their paper on the topic.

Also, we take a listener question about machine learning and function approximation (spoiler alert: it is, and then again, it isn’t).

If you liked this article, you may also be interested in:

• Talking Machines: History of machine learning, w. Geoffrey Hinton, Yoshua Bengio, Yann LeCun (part 1)
• Artificial General Intelligence that plays Atari video games: How did DeepMind do it?
• Inside DeepMind
• Google’s robot and artificial intelligence acquisitions are anything but scary
• Google’s DeepMind acquisition in reinforcement learning
• Why robots will not be smarter than humans by 2029

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Raffaello D’Andrea’s robots can do amazing things: play soccer and ping-pong, perform complex choreographed acrobatics in the air, build towers brick by brick, and move pallets around warehouses. (See the video at his Web site, and check out his TED Talk.)

He also cofounded Kiva Systems, which was acquired by Amazon.com in May 2012 for $775 million with the hope of using the company’s orange robots to move items around warehouses, get products to consumers faster, and lower costs. Robots have reportedly been deployed in Amazon warehouses around the country.

Massimo Russo, a senior partner and managing director in the Boston office of The Boston Consulting Group and the regional leader of the Technology Advantage practice in North America, recently spoke with D’Andrea about the future of robotics. 

Edited excerpts from their conversation follow.

You’ve developed some amazing robots that play soccer and ping-pong. What novel potential uses for business and society do you see for robots in the future, and where are we liable to see robots next?

In the short term, the biggest impact is going to be from how we make things—how we extract things from the ground (mining, agriculture), how we make things (manufacturing), and how we distribute them. We’re going to see a lot of that innovation find its way into these sorts of realms in the next three to five years.

I still think we’re a ways away from the general-purpose robot in your home that’s going to do your bed, do the dishes, and so on. But I think we will also see in the medium-to-long term an infiltration of automation and robotics in the home.

A lot of your research has been around learning systems and learning robots. Do you see applications of learning algorithms for products such as the driverless car?

One of the biggest advantages of learning and adaptation is that it allows you to create powerful systems that are inexpensive to make. If you try to make a very precise robot, it costs a lot of money. If, however, you relax that requirement to make a machine that right off the bat is high performance, and let it learn and adapt tobecome high performance, it becomes economically viable to deploy such a machine. I think learning and adaptation will have a huge impact on how we make machines.

What has changed over the past few decades, either in terms of the underlying technology or in the economics of robotics, and how do you see robotics changing going forward?

A lot of it is probably best thought of as reaching tipping points. Let’s say you make a robot that could do something and that it costs $1.25 for it to do that, and a person could do the same thing for $1. You’re not going to use that robot. But next year, let’s say it takes 5 cents less to do it, so it becomes $1.20, and then $1.15, and then $1.10. From an outside perspective, you may say there’s no progress being made in this technology. Yet when that tipping point comes, when this robot can do the job for $1 or less, all of a sudden it becomes economically viable and everybody jumps in to adopt it. I think that there has been steady progress in the development of robotics and automation, but I think we’ve just reached various tipping points that will make them, all of a sudden, economically viable to deploy.

What kinds of systems innovations are you seeing? And if you think about this from a company perspective, are there interesting startups or maybe lessons even for the more established robotics players?

One of the neat things that big companies are doing is using robots in a cooperative fashion. A lot of people think that’s been going on for a long time, but actually that’s not true. For example, having multiple robot arms work together to manipulate objects for manufacturing, for installations, and so on—that’s a pretty new development. A lot of the big robotics companies are exploring these ideas—also integrating different types of automation technologies together. I think that’s an exciting development that big companies are mainly undertaking.

With startups, what really gets me excited are the folks who are looking at new ways to think about location and navigation. If we really want robots or autonomous systems to be pervasive, they need to know where they are—not just outdoors but also indoors. A lot of startups are trying to solve this problem by using advances in computer vision—and advances in inexpensive sensors, accelerometers, gyros, magnetometers, you name it—and in the algorithms to fuse all that information together and provide a local awareness for autonomous systems that just doesn’t exist right now.

Your startup, Kiva, was acquired by Amazon. What’s the long-term potential you see for warehouse logistics and management, distribution, and operations going forward?

What’s interesting about Kiva was that in the original business model, the key value that we thought Kiva was bringing to the space was cost. We thought, “Here’s a system that will allow us to reduce the cost of filling orders.” But it turns out that it was many of our secondary value-adds that were really exciting to customers: flexibility, adaptability, and a reduction in human errors. There was the fact that you could set up a warehouse in six weeks instead of two years, and the fact that if you wanted to reconfigure the warehouse, it was easy to do—which you couldn’t do with existing automation. The lesson was that businesses really want flexibility and adaptability.

Do you see applications of Kiva’s technology outside of warehouses, perhaps in more of a manufacturing environment?

Absolutely. We always thought about manufacturing when we were growing the company, but we were really focused on distribution. There was just so much opportunity in distribution that we said, well, manufacturing will come later—maybe we will spin off another company, maybe we will start another division. That still may happen. The underlying ideas can certainly be brought to manufacturing.

There’s been a lot of discussion about big data and companies hiring decision scientists. Are the underlying skill sets very similar, or should companies be thinking about hiring control engineers as well?

The right answer is somewhere in between. You can try to model absolutely everything and never look at data, and try to predict. This is what the Greek philosophers used to do—sit around and try to think of how the world works and forget about observations: “There should be a way to think things through.” That’s one extreme. The other extreme is, “Forget about trying to have any understanding of the physical world, we’re just going to look at data, and the data is going to tell us what causal relationship should take place and how the world works.”

We like to do things in between. We like to have physical models. We like to sit on the shoulders of the giants who have developed all of this understanding about how the world works. But we also like to use real physical data to augment our models, to refine our models, to tell us where our models have deficiencies. The more data you have, the better you can do that. I think there’s a lot to be gained by combining those two approaches. Folks who are only focused on big data—I think that’s limiting. Folks who can only do modeling—I think that’s limiting. You need people who can do modeling and who can work with big data.

Systems are inherently unpredictable and complex when you start connecting them. Do you have any cautionary tales for business leaders about how behaviors could be unpredictable around business systems?

There are many examples of when feedback or interactions with the environments can cause things to go wrong. One example that comes to mind is the Tacoma Narrows Bridge. When this bridge was designed, people didn’t really take into account the interaction of the bridge with a constant flow across the bridge. That caused the bridge to become unstable, and it collapsed. There was Chernobyl, which was basically a system that ran wild and became unstable because of various feedbacks that caused a positive runaway cycle.

Feedback is very powerful, but it’s also very dangerous. You have to be very careful when you start feeding things back without some sort of supervision, because you get unpredictability. It’s a two-edged sword: you want to harness the power, but at the same time you don’t want things to go wrong.

How do you get around that? One, try to understand things before you connect them together and try to predict what could go wrong. You can do that by simply trying to understand the system and trying to reason through it, but also you can make simulation models, agent-based models, you name it. I think that’s the first point. Second, just be cautious. Try things out, and see how it works. For really complicated things, it’s very difficult to prove that something is going to work when interconnected. A lot of times you just have to try it out.

Last week I dropped by Aldebaran’s studio to get a glimpse of Pepper in action, and was pretty excited about this robot. But then I talked with Bruno Maisonnier, the CEO of Aldebaran. And then I got really excited: what Pepper represents is another iteration in the realization of the roboticists’ dream.

Aldebaran built Pepper for Japanese telecom company SoftBank, which will start selling the robot to consumers in Japan next February for about US $1,900. The robot will be produced by Taiwanese electronics manufacturer Foxconn (who manufactures the iPhone and other Apple products). So with the marketing presence, technical abilities, and hardware systems now at a consumer-grade quality, we’re one step closer to Rosie the Robot. Or Her. Or the Cylon uprising. Remains to be seen.

Emotional interaction, according to Bruno, is “everything.” We spoke briefly on June 13, 2014 and I was able to ask him about some of this technology, its socio-cultural impacts, and the future of this work for Aldebaran. Several years ago I had been hanging around the Paris studio of Aldebaran and had seen Romeo (and wrote about it and NAO in my book on robots), so I had some sense of what was coming, but I was not expecting the emphasis on affective computing methods that Pepper embodies.

MAISONNIER: Pepper is a fantastic dream I’ve had for years – an interactive robot you can have at home that’s still large enough to be a companion and not just a toy or a pet. At Aldebaran we have a specific concept. We want to help people with robots. We want them to be companion. To help people grow, have fun, and connect them to others.

So first you need to have a cute, nice robot – not just something to say, ‘Oh, that it is nice,’ but the essence of the robot is something that lives in your home. This is not for geeks, so design is our first and most essential part. We want a natural way to interact with the robot and not have a 200-page manual that people need to read. So we need people to be comfortable and interact easily with the robot. We humans have evolved to interact with other humanoid shapes. And sometimes, even without words you understand what the other person means and yes or no can be clearly indicated with body language. We can see if someone is saying yes but meaning no, and if you are saying yes when you are sad, or no when you are happy the message is totally different. So if I want the robot interaction to be natural the robot has to understand that.

MEADOWS: How do you do that? That’s a tall order.

MAISONNIER: These drivers for both understanding and expressing emotions are key to our design. NAO was the first emotional robot, with many possibilities, but many limitations because of its size. Pepper is a larger robot, and one of the consequences of this design decision is that it can move faster. You can have the robot follow you, or if it has to go into your bedroom to look for something it won’t take 20 minutes, and it will be able to open doors. If you want to interact with emotional context, then size is important; [Pepper’s size offers] a more human interaction, rather than talking with something the size of a mouse.

MEADOWS: Ok, but an android is a bad design for a robot because the function follows the form.

MAISSONNIER: But for Pepper the function is to interact with people. The majority of people in robotics are coming from industrial robotics; they’re building left-brain robots. I’m not interested in that. There are huge applications, but this is not what I’m interested in. We want right-brain robots for friends and companions – robots that are more like Jiminy Cricket.

MEADOWS: Here at Geppetto we feel much the same way. Now Romeo and NAO have legs. Pepper’s got wheels. Why?

MAISSONNIER: The main driver is that Pepper is an emotional robot, and we rarely use our legs for social interaction. While there are some detriments, like going up stairs, there are also benefits to the wheels, such as giving the robot a longer autonomy because it can carry a heavier battery. The goal of the robot is to help by teaching, being a health coach, monitoring, growing, etc. So it is more right-brain. Yes, the form follows function because the interaction with humans is the function. So you don’t need legs.

MEADOWS: So how are emotions used to guide the robot?

MAISSONNIER: Communication is emotion. You are an artist and know this. When you are speaking, only about 20-30% of what you say is in the words. But 70-80% is not words – all the other parts are emotions. So communication is emotion.

MEADOWS: What are the methods you’ve developed to convey this?

MAISSONNIER: We’ve invented an emotional engine that collects many different inputs and combines them in multi-modal approaches that use facial analysis, tone of voice, body language, gesture, small noises like laughter, and understanding the micro-signals that your body sends when you talk with someone. All that is feeding a higher level message, which might be that “she said yes but she means no” for example. This serves as the input to our dialogue engine, and we have an expressive engine that uses automatic body language in response. It has all the possibliities – hand, movements, sound, voice, etc – to express the correct message.

MEADOWS: Beautiful. What do you do with this platform?

MAISSONNIER: We’ve been speaking about the robot itself, and all that is part of our expertise platform, but on top of that are the applications so users or developers can build apps via our API. All that goes to the developers, who are then able to use it as they want. There are many ways to use this. For example, if you are designing an application to play chess, you can use the API to deliver messages on your chess program. The system can appear frustrated, or sense frustration at key moments in the game.

The developers are core to our strategy. Aldebaran has an app store and we confirm the quality of the app so you can download this to users, free of charge, who can download the application from the store. We have an atelier [a french term between workshop, gallery, and studio] where everyone can enter, meet with robots, experiment with them, and play with our applications. This will be the home of the developers.

MEADOWS: I love this! Perfect. Are they only online or can I visit them?

MAISSONNIER: We will have an Aldebaran Atelier in Paris, on 26 June, 2014 and two more will open in Tokyo in August. By 2015 we will have two in the US (New York and California).

MEADOWS: Changing the subject a little, you have a robot that has some incredible surveillant abilities. I think there’s a risk here in that systems may not respect privacy, or may be used for ends that don’t align with the end-user. As I’ve written about on RoboHub, where I ask if surveillance is the new business model for consumer robotics, Andy Rubin is taking the robotics effort in the same direction he took the Android effort. This not only changes the landscape for Aldebaran, but it means that you have to address questions of privacy. What Is Google doing?

MAISSONNIER: I don’t know. It is opening up the [robotics] market. They will need some time before having ready robots because the culture of software is not the culture of hardware. Google is a software company, but though they have bought these companies this is not an immediate solution. Building a $1m robot is not the same as building a $1k robot .. they will need time. Their business model, as you said, is to sell advertisements. Do you want to have a spying robot at home?

MEADOWS: Well, most of us don’t seem to mind. Many people still use Google. Why is spying on me with my email different from spying on me with my robot?

MAISSONNIER: One is autonomous. If you send your mail you don’t see it flying around your house checking your books, checking your clothes. It has to be friendly, but as important is that it needs to be your friend that can protect you. You have to be sure it is on your side.

MEADOWS: Autonomy and invisibility are quite different. And a robot can do things that a user may not perceive. After all, email could be called a spybot as easily as a bots.txt file.

MAISSONNIER: This kind of thing will exist. If other robots like Aldebaran exist then between a spying robot and a non-spying robot, people will choose the right one. People will chose and we will settle the rules that we don’t want spying robots. I don’t want them, you dont’ want them. We would put a physical bridge, a switch, that will block communication and ensure privacy. That way people can say, “I want to authorize my robot to communicate with the outside world or not.”

MSM: I hope so. Where will people be able to find Pepper?

MAISSONNIER: Thousands of Peppers will be in SoftBank stores in Japan.

MEADOWS: I understand it will be about $2000, correct?

MAISSONNIER: Yes, for Pepper standalone, after that you can get insurance, monitor modules, repair modules, content, etc. These added components can be important. For me, the next step is to add content, applications in our application store – so a huge community of people will become interested in joining our adventure.

And what an adventure it will be. We look forward to seeing what great inventions Aldebaran turns out next.

If you liked this article, you may also be interested in:

• SoftBank + Aldebaran = Pepper
• NAO Next Gen now available for a wider audience
• My robot is smelling my dirty laundry (or, Notes on privacy, PRISM, and home service robotics)
• Robot love: Spike Jonze’s new sci-fi film ‘Her’ may be closer to reality than you think
• Is surveillance the new business model for consumer robotics?
• Deep Learning creating jobs in Apps, wearable tech and robotics

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

TEDGlobal 2013 in Edinburgh, Scotland. June 12-15, 2013. Photo: James Duncan Davidson

In this 4^th interview of our four-part ECHORD series, conducted last June, Sascha Griffiths from TUM talks to Raffaello D’Andrea, Professor of Dynamic Systems and Control at ETH Zurich and technical co-founder of Kiva Systems. The series explores success stories and common obstacles in industry-academia collaborations in the field of robotics, and examines the differences  between these collaborations in the US, Europe and Asia.

[Sascha Griffiths]
Can you describe your background a little? I am especially interested in your transitions between industry and academia. 

[Raffaello D’Andrea]
After getting my PhD at Caltech I became an assistant professor at Cornell University, where I was for ten years. I started the RoboCup program there and we had a RoboCup team that competed there for five years. We won the world championship four times. My area of research is dynamics and control systems, so, for me, robotics was kind of an application area. It’s a good system design problem.

While on sabbatical at MIT in 2003, I met Mick Mountz, an entrepreneur, and he told me about the problem he was trying to solve: distribution. That is when I decided to quit my sabbatical at MIT and join forces with him and Pete Wurman to start Kiva Systems.

In 2008, I officially moved to ETH Zurich but remained on as an advisor at Kiva, and I’ve been here since.

Would you say that Kiva Systems stemmed directly from your research?

No, I wouldn’t say directly. We did fundamental research on controlling robots and controlling dynamical systems. Kiva Systems has a connection to that, but it is not as though my research directly led to Kiva Systems. It was more that the knowledge, experience, and architectures that I learned during RoboCup transitioned into Kiva Systems.

Prof. Asada also mentioned in our interview with him that he thinks the practical character of RoboCup is really important. Do you think that is a good starting point?

Yes, absolutely.  A lot of our first hires at Kiva Systems were former RoboCup students from Cornell University. They learned great skills while they were doing these types of competitions:  how to work in teams and how to work at mechanical, electrical and computer science level issues. There are tight deadlines. You want to get the job done. There are a lot of parallels between doing things in competitions and being successful in the marketplace, and I think that is a great preparation.

Have you been in involved in academia-industry collaborations?

No, in fact I probably have a different view than most people on this topic, especially from most people in Europe. My thinking has been shaped more by the American system, where academia’s focus is on fundamental research. I would like to see more effort to strengthen academia and industry and I think the best way to do it is through entrepreneurs.

What do you think academia can offer to industry in a collaborative project? Do you think there is a reason why collaborations need to take place?

I do. I know a lot of colleagues who work really well with industry. Industry is full of interesting research problems, and it is full of folks who are interested in creating short-term, medium-term, and maybe even long-term gains. So, for them it makes sense to partner with industry. That is one way to do it. I tend to favor the other approach, which is to do fundamental research, and if it is good work, then attract the interest of entrepreneurs who are willing to learn about the new technology. There are also technologists who have an entrepreneurial spirit, who team up with domain experts and figure out how the technology could be used.

Where do you see the hurdles when technology is transferred from academia to the industry? Why do you think the transfer process is so slow?

The reward system in academia is very different to the reward system in industry. In academia, especially if you are doing fundamental research, robustness of what you are doing is not really the primary concern. It is new algorithms, new designs … and you don’t have to worry too much if it doesn’t work all the time. Of course in industry, especially with automation, that is a huge criterion. Systems have to work under different conditions; and they have to work all the time, 24/7. A lot of the time, it is about figuring out what is just a lab demo versus what is ready to be deployed in industry.

How do you think one could speed up the process of getting from the demo to a product that is commercially viable and achieving the robustness you mentioned?

Investment. Having entrepreneurs who understand where the technology is at and what is required to commercialize it, and who seek investment to actually do that process. This investment could come from private money or it could come from public funds, but it’s investment that speeds up the process.  I would favor that mechanism more than just giving money to the industry and academia for collaboration and say “commercialize this technology”. If you have entrepreneurs to lubricate this process, it will speed it up a lot.

You have already mentioned the reward systems. When you think of academic and industrial partners collaborating, how can they measure the success of their collaboration in your opinion?

I think one thing is patents, which are rewarded both in industry and academia. But I also think that a big metric of success is the students that are trained. A lot of the time, if it is a real collaboration between academia and the industry, those folks will transition into industry. I think you have to take a longer-term view than just a few years. You have to understand that if there is a relationship being built, it can form a pipeline of really talented PhDs with skills in that particular area, who then often go to work for that company. That is a huge plus, which is not immediate, but will have its benefits in the medium term.

How would you measure technological progress?

That depends on which way you look at it. If you are viewing it from an academic perspective, again, the reward is if it is getting accepted in top peer-reviewed publications. From the industrial side, the question is whether it is a product. And there is some stuff in between. I think you have to look at it on a case-by-case basis. I don’t think there is a simple metric. You have to look at the technology: how many years will it take to be commercialized? What impact will it have? Will something else replace it?

Do you know of something at the moment, for example, in the research with quadcopters, where you think that it has really improved the technology?

I think the limiting factor is not the technology, it is good business models. That is what is missing. The technology is quite far along … I think that once people figure out what the right business models are to commercialize this technology, things will move very quickly.

Do you see different models of technology transfer in North America and Europe?

I know Switzerland and I know the United States, and I think they are similar in many ways. If you develop technology at ETH, and it is done during research that is being funded, the inventors get a fraction of it, and the university gets a fraction of it. If students want to commercialize a technology themselves, they can license the technology from the university. If you personally don’t, the university can license it on your behalf. They get a certain fraction of the profits, the inventors get a certain fraction… That is also how it is done in the United States. I think that works well. It has the right incentives; it is not too onerous, where the university claims everything, which disincentives the inventors from doing it because they do not get any benefit. On the other hand, it is only fair that the university should get some rewards for developing technology and it is really leaving it up to the entrepreneurs to take these ideas and commercialize them. I think in that way it is similar. I cannot really talk about the rest of Europe.

Do you see any current trends in robotics, where you think real progress is being made towards commercialization?

I think people are looking at consumer robotics. But I think in the short term, it is going to be traditional manufacturing, helping existing businesses improve their operation through automation and robotics. That is where the potential is, simply because it is still a semi-structured environment. If you sell something to a consumer, it has to work in so many different situations. Every home is completely different. Whereas, if you are selling to a business that is trying to improve their manufacturing capabilities, for example, they have leeway on changing the way things are done, rearranging things, instrumenting things to help their operations. So, that is where I think you will see the big in-roads in robotics and automation.

Are we currently in a situation of increasing the market share and selling more, or are we seeing real progress?

No, I don’t think the market size is increasing. Before, it did not make sense to use automation in certain areas, but now the technology is improving in such a way that it does make sense to do so, and to do so in more industries.

Do you think academics lose interest as soon as something reaches market readiness?

Yes, I think that is the case. Academics are interested in doing fundamental research, and there is a different set of values and constraints when you want to commercialize something. For example, robustness – which I think is actually an interesting topic itself – has a lot of interesting research problems associated with it. They tend to be more at the system level. And I think only a few people are really interested in doing those sorts of things. So, I think, for the most part, academics lose interest. Not all, but a fair amount do.

Do you think that is a good or a bad thing?

I do not think it is a bad thing. I think that, again, you need something in between university research and commercialization to facilitate this process. There are many ways to do that. In Germany and in a lot of other European countries, there are strong industry-university collaborations. In the United States, it is more the entrepreneurial folks that are forming the glue. I tend to like the US model best. I think that it enables the quickest transition. Entrepreneurs are the ones who take on the stuff that is not ready for prime time, so to speak, and actually do it. And it takes investment. So, it is not bad that this happens … because without it, you would not get fundamental research done. So, I think that is perfectly fine.

In your TED talk, you talked about science, engineering, and arts coming together. I think you did not mention business and industry as such. Do you see these worlds connecting well or do you think they are all individual communities that tend to not communicate?

I think that they are separate entities, but there are people who are always at the overlap. You need enough of these people to ensure that there is a transfer of knowledge and interests.

Do you think the business, engineering, and science communities communicate enough?

I think they do. They could do more, of course. And I think the right way to do it is to provide the right incentives.

If you liked this article, you may also be interested in:

• The full ECHORD interview series
• Why fostering high-tech innovation and entrepreneurship matters to the economy, and what universities can do about it
• Live at TED Global 2013: Tweets, photos and video for quadrotor fans
• Raffaello D’Andrea on “Do robots kill jobs?”
• Raffaello D’Andrea on “How will robots shape the future of warfare?”
• Raffaello D’Andrea on “What funding scheme is the most conducive to creating a robotics industry?”
• Robots Podcast: Warehouse robots

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Welcome to Robohub, Toshio Fukuda. Can you tell us about your research?

In my lab at Nagoya University we do many different kinds of research: micro/nano robotic systems, medical robotics, and particularly medical simulators. We have many people working on medical robots and surgery robots right now. We are working on an intravascular robotic surgical system that could be used for brain surgery, for example, but one of our important areas is simulation-based medicine.

Let me explain simulation-based medicine … Before a medical doctor goes into surgery, he has to plan the surgery, and he used to have to do this in his own mind. But now he can use our simulator. For example, if you could practice an exam before taking it for real, you would be more likely to get a good score. Likewise, a medical doctor can use our simulator to rehearse his surgical plan so that when it is time for surgery, he can carry it out with confidence. This is especially important for intravascular surgery, where you have a team of doctors doing the surgery together.

Our simulator is not just for general purpose: we can create patient-specific models. We developed a system called EVE – it is short for EndoVascular Evaluator. It is for training, evaluation and simulation of endovascular surgery. We start by taking a CT scan of the patient and then we make physical model of the patient’s vascular system out of silicon using a 3D printing system. A fluid control unit is then used to simulate the patient’s circulation. Doctors can test their surgical plans on this simulator – it gives them the sensation of a real endovascular surgery.

I think this is the only such simulator available in the world so far. Of course there are virtual-reality-based simulators, and these can be good enough for a beginner, but there is a big difference between a virtual-reality-based simulator and a real physical simulator that can be used by a professional. It makes a big difference if a group of medical doctors can get together and make a plan in advance on how to carry out the surgery using a surgical model where they can see it and practice together on it … In a real situation, if you make a mistake, you have to stop what you are doing and sometimes you cannot carry out the surgery. This way a team of surgeons can make a careful plan in advance.

We have done this since 1998, and we have a built business from it since 2005… we have a company now called FAIN-Biomedical and are selling to the US, Italy, South Africa and also China, Korea and other places.

Can this system be used for training, too?

Of course you can use the simulator to train, but the big advantage is rehearsing just before surgery.

You have done micro-robotics for a long time. Can you tell us about that?

For intravascular surgery, you use a catheter; catheters are small wires about 1mm in diameter that can fit inside a patient’s brain veins. Also we are now expanding to the liver and to the heart, to do bypass surgery …

Your heart has 15 main blood vessels. These are very small and if one of them gets choked up, then you have a strain and the heart does not work properly. The doctor usually carries out bypass surgery, bypassing the choked up place to unblock the problem. That’s the conventional way. But with bypass surgery you have to open up the patient’s chest and then after that you have to close it up again and it takes a couple of weeks to recover from it. However in theory it’s just a catheter surgery that takes one or two hours and after that the patient can walk away from the hospital the same day or the next day. That’s what we call minimally invasive surgery or MIS and it’s becoming very popular because it doesn’t give much burden to the patient. Even after catheter surgeries in the brain or the heart the patient can walk away from the hospital the same day or next day.

The problem is that the doctor must have good skills to navigate the catheter, for example in brain surgery, from the thigh up to the brain where he uses a coil to remove the blockage. For this you need to be skilled, and if you make a mistake, there is no way to fix it.

I have been working with micro-robots for many, many years now. When I moved to Nagoya 25 years ago, a medical doctor came to me and looked at my micro-robots. At that time I had a small 7mm diameter micro-robot going inside of a tube, a kind of low-impact robot. This doctor looked at my robot and asked me if I could build a similar robot to be used for medical treatments in the blood vessels. But it’s not easy to make a 1mm- or 2mm-diameter robot that works inside a catheter. The head of the catheter can be shaped in such a way that it can make right or left turns, the same as steering functions. But there aren’t any commercial products yet … for more than 20 years people have been working on making a functional robotic catheter, but nothing has been commercialized yet. So companies like Johnson & Johnson or Boston Scientific, they don’t sell it yet.

Why is that? What’s the obstacle?

Because of reliability. Also, one catheter costs $2000 and is for only one-time use. In surgery, up to four or five catheters might be used and that brings the cost already to $10,000. But more important is the patient safety: patients are exposed to x-ray during the procedure to guide the catheter and many catheters means the patient will be exposed to the x-ray longer.

Is there a natural connection between micro-robotics and medicine?

Yes, of course … because micro-robots are small and the human blood vessels are also small. If you make robots too big, there are many procedures you cannot do. Think of it in terms of an endoscope: if an endoscope is big it can cause pain. But endoscopes are getting smaller and smaller, and some you can even swallow. As robots get smaller, that’s a good thing from a medical perspective, because they can therefore go more deeply into the human body … even the eyes for corneal surgery.

Think even of the Da Vinci … why is it good? Because the human finger is so big … maybe 10mm in diameter, but the Da Vinci device is 3-5mm in diameter and can get inside the body to reach much smaller places than the human hand with less burden to patients.

Does the application of medicine push the field of robotics to be more interdisciplinary?

Yes, medical robotics is a very interdisciplinary area. For example Pierre Dupont conducts research at Boston University and is also now working at the Children’s Hospital. He is not a medical doctor; he is a robotics scientist. Many people in my field work the same way … engineers going into the medical field and making micro-robotics tools to help people and save their lives. If you visit universities engaged in robotics, you will find at least one person working on medical robots. More than 20 years ago, my friend demonstrated using an industrial robot for biopsy or cutting of bone – specifically the hipbone. Many people, especially the elderly, have hipbone joint problems that could be helped with an artificial joint. And sometimes robots can do a better job than the doctors of cutting the bone very precisely … though of course the doctor has to have good knowledge of the subject and operation, and the robot, as I have said, is just a tool.

Medical robotics is a very competitive field. Germany, France, Japan, US … everywhere.

Do you also see people moving from the other direction, from medicine into robotics?

Yes, there are not so many, but there are some. I have met some medical doctors who are very enthusiastic to use robotic technologies for surgeries.

Professor Yamato, who gave one of the plenary lectures at IROS this year, would be a good example?

Yes, he is a good friend, and a medical doctor who is self-taught in robotics …  he understands engineering. Other people are also looking mechanical robotic systems for surgery … maybe they are doing complicated surgeries and want to use more advanced technology, so they are making advances.

Before we finish, I was hoping you could tell us briefly about your recent trip to the Philippines after the typhoon … You were there on behalf of IEEE?

I’m a director of IEEE’s Region 10 – this is the Asia Pacific Region, including Japan, China, South Korea, New Zealand, Australia and other countries. After the typhoon I flew to Manila to meet with the local Region 10 officer and the chairman of the Philippines section of IEEE. The typhoon destroyed all the houses and infrastructure in the affected area; it was completely destructive. So we are working together on humanitarian technology program. We don’t give water or food … we give technology-based support. Like Internet recovery, energy and such things.

We have a lot of knowledge in this area from other disasters like the earthquake that befell a big province in Pakistan several years ago. Using experiences like this one in Pakistan or the tsunami in Indonesia, we are now making a list of priorities of what we should do to assist them. In most cases what we need first is to restore communication, Internet … after the typhoon in the Philippines there was no electricity and we therefore needed solar panels to keep the PCs working.

Is disaster response an especially important area for robotics in your region?

Yes, of course. There are disaster robot teams in Japan and also in IEEE Robotics and Automation Society, there is a technical committee, and we also have a humanitarian technology conference in Region 10. But in the Philippines case, disaster robot systems have played a very minor role so far. Priority is gathering information and restoring communication. In the first stage after a disaster, this is very, very important. But we are looking at later stages when we can bring some robots for taking care of people.

When will that happen?

We will be there in a couple of months … we have to hurry up. People need order to live. For security we want to bring robots to do monitoring, and to take care of the people from a psychological aspect, too. Many people have no houses to live in … it doesn’t feel good. For example we can bring PARO. PARO was made by Dr. Takanori Shidata, who was my graduate student in my lab at Nagoya … we want to bring this kind of robot to the Philippines. The PARO was also used here in Japan after Fukushima to help people with psychological aspects of recovery.

There are various stages where robotics can help after a disaster. We are making our priorities now.

Thank you Toshio for your time!

You are welcome.

If you liked this article, you may also be interested in:

• Minimally-invasive eye surgery on the horizon as magnetically-guided microbots move toward clinical trials
• Crowdsourcing new strategies for cancer treatment: Towards swarming nanobots
• Medical robotic systems market to reach $13.6 billion by 2018

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

First, I asked him to give me an idea of the “state of robotics” at the present time – so that I might grasp where the field is today compared to where things were be even 5 or 6 years ago. Tjin refers to the fact that after an initially slow past decade in robotics, he’s seen a lot of progress over just the last half-decade, which for him seems promising. “We went from robots that couldn’t play soccer (the traditional competitive activity of Robocup)… were unlikely to be used at home… to robots that can open the fridge, get a beer, open the beer for you, and bring it to you.” He’s seen everything from robots flipping pancakes to serving drinks at bars – and the variety and complexity of these tasks increases noticeably year-after-year at Robocup.

What has enabled this transition? A number of congergent factors have come together to help take robotics to the point that it’s reached today. First and foremost, says Tjin, the X-box “Kinect” and mobile phone companies have given robotics teams in and out of academia a series of cheap and highly effective sensors that would have cost the teams years of time and millions of dollars to create themselves.

In addition, with many of the Robocup competitors being from universities, the purpose of competition is the growth and development of the field – allowing open-source robotics software to be developed, and more useful skills to be shared among the brightest and most eager minds in the field. The result? Tjin reports that teams in remote and unknown areas can often come into Robocup competitions and do just as well as the better-known schools, because they have software and knowledge access that now rivals anywhere else in the world. Robocup itself is intending to facilitate these developments, and by bringing robotic teams together head-to-head, the field is furthered by real, competitive aims (rather than merely new or novel innovations). A set of rules, a bunch of excited scientists of all ages, and a clear objective means a lot of specific learning for everyone involved.

As for the future of robotics, Tjin shared some insights that I had never heard from any of my other interviewees. “I would not be surprised that if in 30 years there were several robots per person, with specialized tasks. This means there will be a huge commercial applications… and these robots would have to adjust differently for different cultures.” He admits that he’s prognosticating, but he sincerely feels that the cultural differences between us will be manifested in the robots around us (much like McDonalds offering different foods in different cultures). He uses the humorous example of a more design-focused country like Italy which wants its sewage-cleaning robots to look good – despite their job.

In addition, different countries and cultures may have different ideas about what a robot should be or do. “In Japan, people see robots as a friend, they want a robot buddy. In northwestern Europe… nobody really cares about this… because we want to robots to do the tasks that we don’t want to do, or which are too dirty.”

One of the biggest areas where Tjin feels that robotics will have home applications is with elderly communities. He mentioned that the percentage of the elderly population is on the rise and that it will be unreasonable to hire individual around-the-clock helpers to care for the elderly – and that it may be a job perfectly suited for a robot which can provide telepresence capacities for loved ones to “visit” the elderly, as well as remind elderly individuals to take medication, or to notify neighbors when an accident or fall happens. Tjin’s own startup in the robotics world (Assistobot) is geared towards serving elderly individuals in managed care facilities and hospitals, a body of work that he – and the teams of talented researchers at the Robocup events – will continue working on.

If you’re a diehard Star Wars fan, or if you have a background in robotics, you might already know who Michael McMaster is. In the past, he’s partnered with Mike Senna to build two life-sized Wall-E replicas, and the little robots were featured on Tested.com. Now, McMaster has two more robots in his sights – an R4-P17 and R2-D2. He built an R2 unit ten years ago with the help of the Yahoo! Group: The R2 Builders’ Club, but he aims to best himself this time around.

I had the chance to sit down with McMaster and ask a few questions about his own personal experience, the droids and exactly how someone can build a droid for themselves.

AA: Hi, Mike. So, in terms of your background, what started all this robot building stuff? Do you have an education background in robotics?

MM: No robotics background, but I have been building things since I was a little boy. I used to make elaborate spaceships from cardboard and lumber, then slowly stepped up to go-carts and more practical things. I grew up on a citrus farm, so my Father designed much of our farm equipment, and I was tasked with building some of it myself. Through the R2 Builders group, I learned many new things. I had a little electronics experience from high school, but learned so much more in the first few years in the club. I was really excited to learn how to mold parts I had made, so I could reproduce them. I also learned how to properly paint and weather props through some very good teachers who are professional prop builders themselves.

So, you’ve done an R2D2 in the past. What makes the new R2D2 different than the old one? And what’s an R4-P17?

I built my first R2-D2 replica over ten years ago, during the early days of the R2 Builders Club, so it was slightly off-scale if a few places. The new robot will be as accurate as humanly possible and will also have most of the tools and gadgets you see him use in the original trilogy. He will be much more robust as well, utilizing the latest advances available today. The R4-P17 was seen in the newer Star Wars films, and was used by Obi-Wan on his Jedi Starfighter. What drew me to that character was the color scheme, as well as the fact that there were very few of them out there. It seems everyone has done an R2-D2 at this point!

How do you get accurate details on the R2 and R4 for the build? It sounds like it might be a tedious process. 

I am fortunate to be involved with a diverse group of talented people in the R2 Builders Club. Many of them have already done a ton of research on the smallest details on each of the droids, which has helped by tremendously. Other than that information, I have used screen grabs from DVDs, and other printed sources. There are quite a few books on the making of Star Wars with some great behind the scenes photographs.

How will you control these fellas? 

I use an RC switch relay and a few limit switches to make it all work. The timing is critical, so it takes some tweaking to get it just right. I will use [linear actuators] to move the center foot and out legs. Both droids will have the ability to transition from two leg to three leg mode. I typically use the PA-14s, as they fit in small spaces, are cost effective and very powerful!

If someone wants to be a part of the R2-D2 Builder’s Club, how do they do that? 

You can visit the R2 Builders Club, which is a Yahoo group or visit Astromech.net. These groups offer an incredible resource, and the members are always willing to help and give advice. Anyone can build a droid!

I’m excited to see the build progress and will be watching closely to see how these two droids are brought to life. Good luck, Mike.

See also Mike McMaster’s robot holiday video.

Aisoy, a spanish robotics startup, is motivated by the goal of building intelligent, personal, “social” robots, which make our lives easier and funnier. Their robot, the Aisoy1, is their first step towards achieving this vision. Robohub recently caught up with the team, to talk about social robotics, the Aisoy1, and the startup culture in Spain.

Can you briefly give the readers an overview of the Aisoy1 robot, and the audience/experience level that it is targeting?

Aisoy1 is an affordable, social-robotics platform designed for anyone over the age of 7 years, who wants to experiment with personal, emotional and social robotics, or just explore robotics and have fun. Our audience includes kids, adults, educators, artists and creatives who want to learn about social robotics technology.

Aisoy1 is based on ROS and uses the Raspberry Pi (other single-board computers will be available soon). Beginners can program the Aisoy1 using AIDIA (a web based graphical environment) or using MIT’s Scratch programming environment. For advanced users who want to delve deeper, programming in Python or C++ is also an option.

What do you mean by “social robotics”?

Social robotics is a deep concept. For us, a social robot is a robot that is:

• Intelligent in the sense that it can understand our desires and can have fun with us.
• Emotional in the sense that it can interact with us in a natural way, while also being able to express emotion and give the impression of feeling and evolving.
• Personal in the sense that it provides a different and unique experience for every person.

What vision or idea motivated the development of the Aisoy robot?

Aisoy is motivated by the idea of creating “life” with emotional (social) robots. We want to build emotional links between us and our physical and digital worlds, by giving emotions to the objects and services that we use every day. The first step we’ve taken is the creation of the Aisoy1, a robot that can interact with us in a way that we as children only dreamed of.

In the creation of the Aisoy1 (our first robot), we wanted to build a platform that included cutting-edge technologies in areas such as pattern recognition, unsupervised learning, emotional and decision frameworks, natural language processing for dialogue management, automatic speech recognition, text-to-speech, and computer vision.

For me, one of the most interesting features of the Aisoy is the integration with cloud-based services, such as audio processing and a text-to-speech engine. Is offloading complex services to the cloud a future direction you see for robotics? What are the next services you have planned for Aisoy?

The cloud is right now a key part of the ecosystem of our life and in the near future, we believe cloud robotics will be fundamental in the development of more advanced and capable robots. We are seeing the first steps in this direction already: Amazon are developing cloud-connected drones for usage in deliveries, and Google is taking a stance in the robotics world. Two companies whose major businesses lie in the cloud – it will be very interesting to watch how their access to cloud-computing resources inspire their robotic developments. In the not-too-distant future, we can certainly imagine ourselves living among connected robots, each of them sharing knowledge with us and the other robots, in a kind of shared intelligence.

For a more concrete example, consider cloud services, such as speech recognition and text-to-speech, which have become commodities for our smartphones – common features, like Siri or Shazam, couldn’t exist without them. The same is happening with robots: there are capabilities, such as SLAM or object recognition, which are computationally very expensive (currently too expensive for the majority of microprocessor-based robots), but can be delivered effectively through cloud integration. Such capabilities, we believe, will become standard for future robots.

We’ve bet on this since the beginning, and have already integrated cloud-based speech recognition and text-to-speech. We’re currently migrating our emotion and decision engines to the cloud so that you can use it from your robots and from other connected devices. By porting these services to the cloud, we want to give people a new kind of experience using social robotics.

Basing the Aisoy on standard, (partially) open source hardware and software such as the RaspberryPI and ROS makes the Aisoy a very customizable platform. What is the coolest thing you’ve seen done with the Aisoy?   

There are many people working with Aisoy1 and their inventions constantly surprise us, for example: a Star Wars Encyclopedia programmed in AIDIA or the integration of Aisoy with turtlebot for making an assistant guide inside a shop.

Socially, one of the coolest projects is taking place in Alicante (Spain), where several schools are preparing a stage play where the main characters are Aisoy1, programmed by children under 11 using Scratch.

We were also very proud when Aisoy1 was the host of Telefonica Talentum and introduced the CEO. It was amazing.

What are the next steps in Aisoy development?

Our development efforts are guided largely by the feedback and requests we get from our customers. A few things they are asking for (and we are currently working on) are:

• Sophisticated personalities with clear focused functions (e.g. security or energy saving)
• Visual development tools for users with low technological background (this is motivating our Scratch integration)
• A mobility device for Aisoy1
• Integration with more cloud services

Now I’d like to change the focus and look at Aisoy as a company. We often hear stories of US-based startups, but very little from the rest of the world, hence I think it would be very interesting to get an insight into the startup culture in Spain.

Is the Spanish ecosystem (funding, mentoring, etc) supportive of startup companies with new and novel ideas?

Spain still has a long way to go in this area. Companies such as Facebook, Apple or Google, could never have been founded in Spain – their ideas were too innovative, and too unproven. Spanish investors usually invest in companies whose technology has proved successful in existing markets and who have equally successful business models – minimal-risk investments. They are happy to invest once the company has an income, but are extremely conservative in supporting young companies with innovative ideas to get off the ground.

What were the major difficulties in the early stages of the company, and how / with what resources did you overcome these challenges?

In the early stages of Aisoy, we had two major challenges: Finding funding, and surviving the economy.

Finding funding was difficult for us. In Spain it’s easier to reach 200K€ in a “Friends, Family &Fools” first round, or even 10M€ for well-established companies with proven business model, than it was for a company like us, in the end-phase of a long-term technological development, to reach 2M€. We are very thankful for our first funding from the founders and CDTI (The Centre for Industrial Technological Development, a Spanish Public Business Entity) through the Neotec program for technology startups.

Relatedly, our first years of development and production were during the high-point of the Spanish economic crisis and the Fukushima disaster (which affected hardware supplies). It was not easy for us to survive during these years, but we love robotics, believe in our vision, and have fought hard to keep pushing Aisoy forward. We’re now very proud to see the results of our efforts over the last years!

In the last few years we’ve seen the development of many different strategies and frameworks for running a startup. Did you take guidance from such resources, or find them helpful in the early stages of AISoy?

Nowadays people are using or talking about these methods (eg. Steve Blank’s “The Startup Owners Manual”, Alexander Osterwalder’s “The Business Model Canvas”, Eric Ries’ “The Lean Startup”), but five years ago such guidelines were hard to come by. In our case, we used similar methodologies without knowing that we were using their principles. In the end, most of it is just common sense.

Looking back on the last five years, what knowledge/experience that you have now would have been useful to have when you started? What would you do differently, and what would you do the same?

Jeez, where should we begin… Experience is experience! But if we were to do it all again, we would:

• Use affordable 3D printers (such as the Prusa i3 we have recently built), to drastically improved the speed of prototyping, thus reducing risks by testing more and reaching the market faster.
• Make the hardware more modular so that we can reduce the risk of component obsolescence or supply problems affecting the whole system.
• Develop the SDK and developers oriented tools first, making it easier for us to develop for the system

In the end though, these things are a matter of continuous evaluation and improvement!

Any closing words?

Thank you for giving us this opportunity to reach your audience (we’re among them). We’re really looking forward to showing you the upcoming tools we’re developing, such as the new AIDIA and our latest development, a Chrome App to bridge with Scratch and cloud services. Stay tuned!

Thank you to the Aisoy team! If you are interested in more information, check out their homepage!

The Girls of Steel – a competitive FIRST team located in Pittsburg, PA – is on a mission to draw more young women into engineering. We’ve already heard what it’s like to be part of an all-girls robotics team, we now catch up with the team’s mentors, Theresa Richards and George Kantor, to hear about their roles in inspiring and mentoring the team. Here’s what they have to say …

How and why did you become a mentor?

George: My lab, the Field Robotics Center at CMU, was approached by Patti Rote about creating an all-girls FIRST Robotics Competition Team in the summer of 2010.  I have been involved in various K-12 robotics outreach activities dating back to the 90s, and I had always wanted to learn more about FIRST.  So I offered to lead the team.

Theresa: I became a mentor because I was very interested in learning more about FIRST robotics and about how I could work with the FIRST program to support and encourage girls in the interdisciplinary field of robotics.  The study of robotics includes all STEM fields.

Did you have a mentor when you were growing up? If so, can you tell us about this person and the impact they had on you?

George: Growing up, I had a swimming coach who taught me lessons about setting goals and perseverance. In graduate school and throughout my time as a researcher at CMU I have had a series of mentors who have helped shape my technical career.

Theresa: I had several mentors when I was growing up. Most of my mentors were my high school science teachers who offered opportunities for me to do activities with them outside of school. For example, my biology teacher invited me to participate as his teaching assistant at an evening class he taught. My chemistry teacher helped me get a part-time job and she introduced me to my college – the same one she attended, and my zoology teacher took a group of us on a special field trip and she arranged for an “extra credit” dissection lesson after school. These awesome teachers recognized my interest in science and made sure to offer opportunities. When I think back about it now, I feel so grateful that they took extra time with me. That’s what I try to do for my students.

Do you have any suggestions for teachers or others who may want to get started as a mentor?

George: Just do it! There are plenty of opportunities to mentor young people, and it is an extremely rewarding experience.

Theresa: Other teachers who are interested in finding out about a FIRST robotics team should go to the website and click “What teams and events are located in my area?” I bet they’ll find a robotics program that will welcome them with open arms. If the teacher feels unsure about being a robotics mentor, he/she should attend a meeting or two before making a decision. Working with the students is inspiring because of their interest in creating something and learning so many new things. And it’s amazing to hear a student say, “So that’s why we learned this in math or in physics” or “I never would have considered engineering if I weren’t on this team.”

What’s the hardest part about mentoring?

George: It is important to encourage people to dream big, but on the other hand aspiring engineers often have unrealistic expectations about what they can accomplish. Helping students define achievable goals often involves encouraging them not to pursue some of their more ambitious ideas, and delivering that message in a positive way can be a challenge. I want my students to believe they can do anything, but I also need to teach them to approach their dreams in a realistic way.

Theresa: The hardest part about mentoring – there really isn’t a hard part, except that having meetings during after school hours from 5 pm until 8:30 or later means we have some late nights, especially during the 6-week build season. Mentoring is a rewarding experience on multiple levels. I enjoy the experience of working together with students, and colleagues and parents who have similar interests of encouraging girls in STEM fields. Like they say in FIRST, “It’s the hardest fun you’ll ever have” – that goes for mentors as well as students.

Do girls need female role models?

George: Yes. It is inspiring for girls to see women in the roles they aspire to, especially in fields like computer science and engineering that are currently dominated by men.

Theresa: Girls need female role models in STEM fields in order learn about various careers and to be able to see themselves in various positions or careers in STEM fields. Girls also benefit from belonging to a network or group of girls who share similar interests while in high school, college, and beyond.

Which students should participate in robotics competitions?

George: All students, especially those with technical interests. Robotics competitions are easily accessible, and they provide an opportunity to learn valuable technical skills in a fun way. They also provide an opportunity to learn life skills that are useful regardless of what career they decide to pursue: teamwork, goal setting, time management, etc.

Theresa: I believe all girls will benefit in some way from joining the Girls of Steel or another FIRST robotics team. The level of the benefit will depend on their level of participation.  Participation on the Girls of Steel FIRST robotics team can range from 4 to 20 hours a week during the year. The number of hours depends on the “season” – preseason, build season or competition season – and on a girl’s particular role on the team – e.g. leaders spend time preparing for team meetings and have additional leadership meetings to attend. Why do you recommend robotics competitions for everyone? Not only do the students apply what they’ve learned in computer science, math, science, and engineering classes they also learn that it can be fun to work hard. They learn so much about themselves by being a part of the team. For example, I’ve heard girls say things like “only girls on the team understand why I like ….”, “I found my best friends on the team”, “now I know the difference between a personal goal and a team goal” , “ Only you get my jokes,” and more.

There are many robotics competitions for students. Why FIRST?  

George: I really like FIRST’s approach to competition. Winning is important, but greater importance is placed on cooperation and professionalism. FIRST has invented words that capture this spirit and are central to everything that FIRST does: “coopertition” and “gracious professionalism”. In addition, FIRST goes to extra efforts to make its events exciting. The games are well designed, with attention paid to making the matches interesting to watch. The competitions are narrated by professional emcees and the field set-up has a high production value. The result is something that is a cross between a rock concert, a sporting event, and a science fair.

Theresa: I really don’t know much about other competitions except that some involve “fighting robots” rather than “game playing robots,” and the robots vary in size. FIRST® (USFIRST.org) is an organization that promotes robotics competitions for all ages that involve more than just a robot. Our team competes at the FRC (FIRST Robotics Competition) level that involves designing and building a 120 lb. robot over a period of 6 weeks to play a game with 5 other robots on the field. The awards structure of FIRST promotes other aspects of the team – like fundraising (to raise money for registration fees), outreach, teamwork, public speaking, leadership, media, and more. We’ve found that having a FIRST team seems to be like running a business with students having the opportunity for major leadership roles. High School level FIRST teams are encouraged to support STEM awareness for younger people in the community. Publishing Lynn’s comic about the 2012 FRC season on our website and on Robohub is just one way the Girls of Steel FIRST robotics team contributes toward that goal.

Thank you both very much for your time, and best of luck to you and the Girls of Steel in future competitions!

My interest in mechanical things came about from the age of five. I distinctly remember carrying around a little sketchbook with a pencil everywhere I went. Whenever I had an idea for something, I would draw a picture in a page of the book, maybe writing a word or two on what it was. Most of the time these ideas were Rube Goldberg machines designed to do mundane tasks like watering plants or making hot drinks. As I got older and played around more with LEGO Technic, the complexity of these machines and the information I’d write about them increased. By the age of eight I was starting to become more interested in electronics. I loved designing mechanical machines, and with electronics I could make the machines I was designing much more complicated. I began to pull apart my toys due to an insatiable curiosity to see how they worked. My knowledge, of course, was very basic – the only electronic parts I knew of were the ones with a visual function, such as a DC motor or an LED. Even the sketchbooks were getting an electronic overhaul; I remember a few of them had the LCD clock faces on the front cover that I pulled out of my wrist watches.

By the age of ten, the designs I was drawing in my sketchbook were more product designs. I designed phones, watches, cars, computers and even game consoles. The designs had all the key components diagrammed and labelled with labels like ‘CD reader’ and ‘Big board’ (Motherboard). By this stage the designing process became a key interest as I spent more time trying to make the designs look good so that I could show others my work.

I think the step between machines and product designs to robots was due to two key things that were happening during that time. The Star Wars episodes 1-3 were released during my early childhood and, when I was old enough, we hired the DVDs. I was instantly hooked and loved the idea of C3PO. The fact that Anakin made C3PO in order to help his mother specifically seemed to target me, it was like a sign that I needed to do the same.

The other reason I became interested in robotics was actually to do with my cousin. Every Thursday night I would go to his house and play computer games with him. One night I went there and he had this really cool game called Robot Arena 2, in which you design battle bots. The game let you customize everything from the body shape to the internal components and the weapons you used. I spent hours playing this game and absolutely loved it.

I borrowed all the books in my school library that had anything to do with robots (there weren’t many) and started to design robots.

At the age of fourteen my parents bought me the Lego Mindstorms NXT. The NXT was a Lego kit that let you build and program your own robot. I played with it for around two years and had great fun with it, but soon my hobby was going to take a turn for the worse.

I was in high school now and things were getting tough. I was starting to get really stressed at school and began to play video games as a release. Video games quickly took over my life. I would get home from school at 4PM and jump onto my computer to play. I would only get off for tea around 7PM and then get straight back on until I had to go to bed. It was a poisonous cycle and I regret those years of my life as I felt like I achieved nothing. My video game addiction meant I didn’t do any homework and spent no time revising what we covered in class. My grades began to slip and my gaming habit only became worse.

The cycle was only broken when it came to crunch time, Year 12, the final year of high school. I realised what I had gotten myself into and was determined to make it stop. I put my head down and worked like I never had before. My grades picked up dramatically in all subjects but Math. I studied every night and even finished assignments early. Sadly it was too late for Math, I was never good at it when I was young and the period of not doing homework absolutely destroyed any understanding I had on the subject.

During the mid-year break I wanted a new hobby, a hobby that would not only be something challenging like computer games, but also something that would actually benefit me. I remembered how much fun I had designing and making robots and decided it was time to go back to that.

I went online to research other ways I could make robots other than using Lego, that was when I heard of this thing called an Arduino. I didn’t really understand what it was but I bought one called an Arduino Uno. When it arrived in the post I looked up tutorials to see how I could get it to do things.

July 18 2012 I uploaded my first video to YouTube on Arduino. It wasn’t my first creation, but it was the first one that I had recorded. I recorded it on my phone and just uploaded it straight to YouTube. I had uploaded previous videos before on YouTube and thought it would be cool to show off what I had made.

From that point on I recorded all of my creations and uploaded them to YouTube, each time writing a little about them on my blog.

First I covered all the basics, such as making sounds, lighting LEDs in patterns and detecting light changes. When I had used up all the components that came in my starter kit I researched ways to make things move with the Arduino. I found these things called servos and read they were easy to use so I went on Ebay and bought a few of them cheap from Hong Kong. Once I had worked out how to use them I made my first little bot. It was nothing complex and didn’t even have any autonomous capabilities. The robot simply moved the servos to certain positions to make the robot hop like a frog. Money was scarce so I made things out of whatever I could. The body was made of cardboard and the legs were the servo arms with heat shrink over the top to add grip.

I had experience with programming before and I was pretty good at computer software, so once I understood how I could make robots with the Arduino, I began to make them smarter and more complicated.

I became interested in program in particular called GlovePIE. GlovePIE is a program that allows you to program gamepads to act as a keyboard and mouse. I wrote a code that allowed me to have full control of my computer by simply talking, I called it JARVIS.

After programming JARVIS I realised that I could use voice recognition to control the Arduino. To test this theory I made a humanoid robot that was able to express emotions and communicate.

I also learned how to use infrared commands and turned my Arduino into a voice controlled TV remote.

It had only been two months but I absolutely loved making things with Arduino and couldn’t stop. Every time I wanted to learn something new I researched for hours on how to do it and watched so many YouTube videos I used up our internet allowance. I went from controlling a remote control car with my iPhone, to making a webcam that tracked my every movement.

My creations were becoming more and more complicated and the views my videos were getting was increasing. I was getting asked so many questions a day I decided I would start doing tutorials. I was getting more comfortable with YouTube as the response to my creations was positive. I kept making tutorials covering all the basic things you needed to make robots with Arduino.

My robots were starting to get very complicated and had various autonomous capabilities. My friend from school had a big box full of old robot kit parts that he had collected as part of a magazine series, he knew I was interested in robotics so he kindly gave me all the parts. I salvaged everything and turned the brain of the bot into an Arduino.

I began to understand more about the electronics inside robots and my knack for software proved invaluable in my first big robot project.

I had five tutorials released by this stage and they were becoming very popular, with thousands of views. My old physics teacher, who I am good friends with, suggested I come into school and teach the students about robotics. I loved the idea and since then I have been volunteering every Friday during their lunchtime to teach the students about robotics and programming.

I kept making robots and developing new skills. My videos were getting a lot of views now and I had just over five hundred subscribers. I was now recording everything I did, whether it was about robots, making things to help with videos or just me going out to conventions.

I thought it was time now to start making videos focused purely on making robots. I made several videos on the matter and three Instructables on three different bots, Ben, James and Yogy. I realised now that people were really starting to like my creations as my bots had started to win me prizes.

Yogy the yoghurt-tub robot!

After spending a semester at the University of South Australia, studying a bachelor of Mechatronic Engineering, I decided to take a one year deferment. I choose to do this for several reasons. The first major reason was the job prospects. We’d often have people from the field come into lectures and talk with us about what they do. The one thing that I hated about what they all said was that none of them were hands on, they all sat behind a desk and worked for the ‘big man’. As I spent more time volunteering at the school I realized that perhaps teaching is the job for me. I will always love making robots, but sometimes I feel like sitting behind the desk just isn’t for me. Teaching seems to bring my love for robots and the enjoyment I get out of teaching together.

I now have over one thousand subscribers on YouTube and am starting to make of money off of it. Recently, a publishing agent contacted me asking whether I would be interested in producing a video course for them on the subject of ‘Making Arduino Powered Robots’. I agreed to their offer and am working with them now to produce the course.­

In the next few weeks I will make the transition from not knowing what to do with my life, to turning my forever interest in robotics into a job. I am going to be an author and also be helping a robotics kit creator in writing a teaching curriculum and helping with instructional videos for his kits. I feel like this is the start of a lifetime journey and I can’t wait to see where robotics takes me.

I don’t know where I am going, but I feel like I am surfing the wave of opportunities. I hope one day to be producing my own kits or perhaps working with the likes of MAKE promoting robotics for kids.

I would also like to mention one last thing. Without the people on LetsMakeRobots.com I probably wouldn’t be where I am now. They are the people who have stood behind me, giving me support and help when I needed it, and have been my slingshot into having a robotics-based career. If you are interested in making your first robot, or have been making robots for a while, go to their site and join. LetsMakeRobots has a great community for both beginners and veterans alike, they have a step by step instructional on how to make your first bot and you can even show off your robotics projects.

I hope you can draw similarities between my story and yours. Remember, stories are ongoing and the more effort you put in, the better your story will be.

To understand what the Drones for Schools program is, and how it came about, it helps to know a few things about my background. I didn’t originally set out to be a STEM educator. I had some science education from studying mechanical engineering as an undergraduate, but eventually wound up with a master’s in journalism. I didn’t think the two halves of my life would merge until I finished my master’s and took a job at a K-12 STEM education grant, at the University of Illinois – they needed someone with a scientific background to handle communications and study the social networks that teachers were forming as part of the grant.

At the time, I was researching how journalists could use aerial robots, commonly called “drones”, to capture data during natural and man-made disasters. As part of this research, I was spending a great deal of time in my basement “hacking” together a small drone from a radio controlled airplane and open-source hardware. It became apparent that this learning experience mirrored what teachers were doing in the STEM grant, and so I approached my boss, a department head in the College of Education one of the co-principal investigators for the grant, with a pitch: let’s have students design, build, and fly drones at our partner schools. He assembled a team of STEM educators to guide the process, and the “Drones for Schools” initiative was born.

Two students collaborate to build a “drone”.

In some ways, the program resembles a nesting doll for concepts in science, technology, engineering and mathematics. A drone is a complex machine, and once you peek inside, one STEM concept leads to another. To design the aircraft, you begin with concepts like Bernoulli’s principle, which when applied through a mathematical formula, determines parameters such as stall speed (how slow the plane can fly before falling out of the sky). Stall speed dictates the power you need to fly, which opens an opportunity to deliver content about electricity and electric motors. And delivering content about electricity and electric motors segues nicely into lessons about battery chemistry and capacity. All the while, students are applying this content knowledge as they make key decisions about how this aircraft will be designed and deployed.

Because this is a semi-autonomous aircraft that is dependent on a small microcontroller, computer programming is interwoven into the Drones for Schools curriculum. Students take this full-function microcontroller, which uses hackable open-source hardware and software, and learn programming by getting it to do some things it wasn’t intended to do. They learn about electronics prototyping by wiring up the microcontroller to external LEDs, and by writing C code to enabe the microcontroller to blink out a Morse code, they learn about variables, arguments, loops, functions, and libraries.

Soldering is one of the many skills that students learn through the Drones for Schools program.

These individual lessons are valuable on their own, but the program’s biggest gains come from students assuming the roles of engineers. This program was designed in part to meet the demands of the Next Generation Science Standards (NGSS), which were developed by the National Research Council to help prepare K-12 students for an world where science literacy is increasingly necessary. Perhaps one of the biggest challenges and opportunities presented by NGSS, is the incorporation of engineering education into the K-12 science curriculum, something which hasn’t been attempted on this scale. To help students understand what makes them engineers in the Drones for Schools program, and how that’s different from what scientists do, we’ve had a graduate engineering student explain how engineers make models, design solutions, and test those solutions. The engineering student also provided high school students with information on how their interests in the Drones for Schools program could lead to an education in engineering, and what an engineering career path looks like.

Drones for Schools is just one of around 30 collaborations at the National Science Foundation grant EnLiST, which stands for Entrepreneurial Leadership in STEM Teaching and learning. Each collaboration is unique, but they are all “entrepreneurial,” meaning they draw on resources from other classrooms, university faculty, community organizations, and businesses. Drones for Schools is no different than the other collaborations in the sense that we rely on community partners to provide a useful application for the drone. The drone is not built for the sake of building a drone; rather it is built because it can provide high-resolution images of the ground which can be stitched together in a map. And that data has use for farmers, city planners, researchers, first responders, and many others. Simply put, we think there is a need for drones in the community, and our students are trying to fill that need.

Testing out one of the class’ inventions

AUVSI, the Association for Unmanned Vehicles and Systems International, predicts that 30,000 drones will be flying by 2020, with the majority of those serving as farm equipment. Depending on the definition of a drone, the US may already have exceeded that number. The goal of this program isn’t to ensure that one of our students will engineer one of those drones someday, although technically they’re already building and flying one of those 30,000 when take part of our initiative. Rather, we’re interested in our classroom data, which suggests the initiative increases interest in engineering. And in that sense, I think we’ve succeeded.

UIUC faculty and staff associated with this grant are currently seeking other grants to specifically enrich middle school STEM education. Because of the tremendous learning opportunities packed into these flying robots (some of which are small enough to fly in a classroom), and because of their real-world importance, we believe that drones will be an integral part of STEM opportunities for some time.

As part of our series on ‘High-Risk / High-Reward’ robotics, I interviewed SRI International‘s Director of Robotics, Rich Mahoney, who’s role there is to help identify important emerging robotics technologies, align them with the needs of funding sources, and bring them successfully to market.

[Hallie Siegel]
Tell us about yourself, SRI, and your role in the R&D and entrepreneurial space…

[Rich Mahoney]
I’ve been at SRI for five years, and my position before coming to SRI was in an early stage venture-backed environment as the general manager of a company introducing and developing a new robot for stroke therapy in the US market. Before that, I did product and business development, and before that I did university-based research just after finishing my PhD. So, I have a broad background in applying robotics, from university research to commercial product development to venture-backed early-stage companies.

What’s interesting is that, because of SRI’s business model as a nonprofit, I actually practice my background in all of those areas in just one job. SRI’s primary business is contract research and development, but our overall mission is to invent, apply and commercialize technologies. As the director of the robotics program, I’m responsible for running the business of our robotics program. I have profit and loss responsibility for our performance in this nonprofit environment where we’re really working towards a higher mission of impactful R&D.

It’s a very aggressive and difficult environment, and it’s for people who want to work in a place where they can have a lot of impact: where they can work on applied solutions, and where the problem-solution is the focus, rather than the PI-driven research that you see at universities.

SRI is an offshoot of Stanford, correct?

You can say it’s a spin off from Stanford University. SRI originally stood for Stanford Research Institute, when it was founded 65 years ago, but it spun out as an independent nonprofit in the 1970s. It was doing some military-oriented research at that time – during the Vietnam War era – and the students didn’t want that kind of work going on on campus. Instead of closing it down it was spun off and made independent. Since then it’s really developed its own identity, and now we’ve got 40 different programs working on leading edge research. It’s not all hard-core technology like robotics; some of it is education policy…  some of it is pharmaceutical discovery …  so we operate in a very broad base of areas, but always with the mandate to work at the leading edge of applied solutions.

It seems like SRI operates half way between a university innovation lab and an investment company…  

That’s right: there’s university research, and then there’s commercial research, and SRI lives right between those two… in a space often referred to as ‘the valley of death’ [Chuckles.] It’s where things go that get invented, but which just can’t seem to get applied and transitioned to the commercial world. It’s very dry to say our primary business is in R&D: I would say that SRI is a big party tent in the valley of death! [More laughter.] People at SRI don’t just work there, we thrive there: we’re just really good at taking early stage technologies and transitioning them into practical value.

As we were developing this series it became clear that “high risk, high reward” can mean different things to different people … the researchers on our team were talking about technical hurdles, and our start-up and business experts were talking about commercialization hurdles… and then, even if you can make it through these technical and commercial hurdles all the way to the public sphere, maybe the market isn’t ready for you… maybe policy isn’t ready for you… maybe there are going to be legal issues. I find it interesting that you seem to be coming at it from such a broad perspective.

Maybe the environment in which SRI operating is so challenging because there are so many perspectives. We’re trying to actually cross this valley of death, but all the while we’ve got multiple market perspectives, we’ve got the inventors, and we have all this investment that we have to do to get it ready. How do we know how to invest in the right way? It really seems like a big problem…  The person who will grow a company is not the same person who can get it started, and it’s not the same person who can do the initial invention, or the person who can do the engineering effort to get the invention ready for market … it’s very rare to find somebody who can follow something from the very beginning to the very end.

So there is this broad set of competing interests, but the driving perspective for SRI is to align resources with what it takes to be successful – and that’s key. Really understanding what’s going on, being connected to the market by being connected to the technology, having the right people doing the right things… yes it’s broad, but there’s a single focus on finding solution sets, and one thing I’ve learned in this job is that you succeed best by putting people in a position of strength for them.

So where is your work initiated? Are you mostly assisting inventors who are seeking investment? Or are you looking for technical solutions for government and policy-makers?

If you look at just the business part of what we do, our main revenue comes from our contracts to do research. Our customers are organizations that want to fund research and development, and so either they come to us because they have a problem that needs solving and they don’t know have the resources or people to solve it, or they come to us because they know we have technologies that could solve their problem and they want us to develop our solutions further in order to suit their purposes.

On one side, we operate in this business development environment that is essentially being 100% on soft money. We have no endowment or annual corporate budget. If you don’t get the business development right then you are out of business after one year. And so you also have to be really good at evaluating your technology and qualifying the people and organizations that want to fund you to develop them. Part of my job as director is to figure out who the chronic innovators are, evaluate who could possibly fund them, and then do the work of building the relationship and making sure that everyone understands the market needs.

Basically, we are inventing new kinds of technologies that we believe will solve fundamental problems that are aligned with our customers’ needs. In the case of the robotics program at SRI, DARPA is a key customer, and one of their needs is to maintain the US lead in various applications of robotics technology. As a result, we work with them in a very collaborative process where we are educating and interacting and supporting that mission all at the same time.

Why does DARPA need SRI? Couldn’t they just work directly with the innovators themselves?

In rare cases DARPA would work directly with an inventor, but it’s a government agency, and it’s not easy to manage a government contract and to meet all of the costing and security requirements. SRI provides the overall support structure for managing the business relationship with the government. The other part of what we do is managing and documenting the intellectual property.

It goes back to what I was saying before that it’s very rare to have a single individual who has the aptitude to do all aspects of the technology transition process from invention to commercialization. So in an organization like SRI, we have people who are strong in business development teamed up with people who are strong innovators and people who are systems applications experts. By putting teams like this together, you can actually create a better market or industry response.

Do you have a formula or process for working out risk versus reward in terms of the research you undertake and the specific technologies that you select to move forward?

SRI definitely has internal processes for defining the value in its technologies and how it communicates that value. We have something we call “the discipline of innovation,” which is a series of processes that, when applied, allow you to be more successful in transitioning the technology. These include simple things like identifying a champion (and the champion may or may not be the inventor, but is someone who believes in the application of the solution and is personally committed at some level to being successful), and making sure that all the resources in an organization are aligned for success.

We also have a very clear process for communicating the value of our innovations to the clients – and that’s something we call ‘NABC’, which is really just a structured approach to communicating value to the customer. N is the need, so we define the specific need back to the customer to make sure that we understand it. The A is the approach, the actual technical solution. Us hardcore technical personalities love to focus on this part, and what often happens is that we either have a technology looking for a solution or we have the right technology but we focus so much on describing the approach that we forget to really look at the need and align it better with what’s required to be successful… so having a balanced approach is important. B is the benefit: having a very clear understanding of what the benefits are, and at what costs, and this is relative to C, the competition.  We go through this process in a very structured way, and it creates a much better solution set and sense of clarity to our technical and business development staff when they’re beginning to look at whether to apply a specific technology to a problem.

The other thing I’ll add is that there is a culture of iteration at SRI, and its understood and practiced through open communication and assessment of value, so that an NABC is not a single static document – it is a dynamic document that’s constantly being revised.

What would you say are the next breakthrough areas for robotics?

I think there’s a very clear progression that is occurring right now, and there are two different tracks that are going to be coming together in the near future.

One of them is that there’s a drive for robotics technology to become more low-cost and more accessible. A lot of people think about robotics as an independent technology space, and at some level it is. But if you really look at the current state of robots that are getting penetration, like drones/UAV’s and mobile robots, the emergence of these technologies has a lot to do with the fact that their components are cheaper and more available than ever before. And part of that is not because of robotics per say, but it’s because of other markets. In particular, the cost and availability of many sensors is driven by growth in the personal and mobile computing markets. Robotics is drafting off the last 30 years of evolution of personal computing technology, and represents the next stage in that evolution.

The second track is manipulation solutions: robotic arms and hands that allow you physically manipulate the world. If you look at the trends again, you’ve got lots of mobile robot solutions and emerging tele-presence platforms, which generally cost under $10,000 – and some of them cost a couple of thousand dollars – but they’re all lacking manipulation.

There’s been an enormous amount of investment from DARPA to develop lower-cost robot arms and hand solutions, and in the next three to five years, these manipulation technologies will begin to emerge.  And when the cost of these is lowered to the extent that they become accessible to entrepreneurial, hobby-level roboticists, that’s when we’ll start to see a much broader dissemination and application of robotics into a personal robotics market, similar to what happened in the early 1980s with personal computing. That convergence of low costs and manipulation technology is part of an ecosystem that is going to drive innovation in the next three to five years.

Is access to lower-cost equipment making it less risky to invest in robotics now?

It’s not going to make it less risky from an investment perspective because the overall funding required for a company developing something to the point of commercial viability is so much more than the cost of the product hardware components. And often the real risk is with the management team’s ability to properly set up the business model. But it will create products that have a greater potential to get penetration because they can be offered at a lower cost to the end user. Lower cost robotic components will also enable more entrepreneurs to start companies.  So, I would say that the real benefit is that more robotics companies will get started, which will ultimately lead to more successful companies and broader uses of robotics.

What other factors are contributing to this rapid growth in robotics?

The number one place to look is DARPA. If you go back in the entire history of robotics innovation, and look at which technologies are emerging right now in the market and what their origins were, DARPA was the inspiration and funding driver for almost all them at their early stages. It’s really pretty dramatic if you think about it.

Some people look at DARPA programs and think, ‘What are those crazy scientists up to now?’ but every one of those programs represents tens of millions of dollars of R&D funding, and these awards are being made to some of the top people in the country. If you look closely at who those researchers are, what their programs have been doing, and what they are being funded to do, you’re going to see the future of robotics.

Such multi-UAV systems were in the focus of the Sixth Lakeside Research Days organized and hosted by Lakeside Labs and the University of Klagenfurt in Klagenfurt, Austria, from July 8 to 12, 2013.

Concept and Program

The Research Days are a special format of workshop. Besides keynote talks given by invited experts from academia and industry, they feature a demonstration session, intensive group work, discussions on technological and business aspects, and social activities for attendees.

The keynotes were as follows:

• Small flying robots: Design and visual navigation
  By Roland Siegwart (ETH Zürich)
• Fusion of distributed perception in collaborative visual-SLAM approaches on mini-UAVs
  By Darius Burschka (TU München)
• Exploiting redundancy for reliable aerial computer vision
  By Horst Bischof (TU Graz)
• Aerial imaging and wireless communications with multiple autonomous UAVs
  By Christian Bettstetter and Evsen Yanmaz (U Klagenfurt and Lakeside Labs)
• Decisional issues in multi-UAV systems
  By Simon Lacroix (LAAS/CNRS)
• Slalom racing and flight dances: motion planning, control and learning for high-performance quadrocopter flight
  By Angela Schöllig (U Toronto)
• Multi-spectral people detection from UAVs
  By Stephen Cameron (Oxford)

A panel with Roland Siegwart, Hermann Hellwagner (Klagenfurt), Phil Charlesworth (EADS), and Michaël Rischmüller (Parrot) discussed business opportunities and start-up ideas of UAV systems.

Further information can be found in the final program and the Twitter feed #resdays13.

Media Coverage

The event was covered by various media including Austrian television ORF and the newspaper Der Standard (article Drones in peace missions).

Cover of “Der Standard” from 17 July, 2013

Impressions of the First Day

Keynote by Siegwart: Small flying robots

Keynote by Schöllig: Slalom racing and flight dancing

Image Gallery

Photos by Christian Philipp (Lakeside Labs GmbH) and Christian Bettstetter.

Established experts in the field of robotics were recently interviewed by a group of scientists from the ECHORD project at Technische Universität München. Motivated by the fact that industry-academia collaborations in robotics are still limited, they wanted to know what makes such collaborations a success, what can be done to avoid common obstacles and what the differences are between collaborations in the US, Europe and Asia. In this 3^rd interview of our four-part series, Sascha Griffiths from TUM talked to Minoru Asada, well-known professor at Osaka University, at the occasion of IEEE ICDL-EpiRob conference in San Diego, California.

[Sascha Griffiths]
Do you collaborate with industrial partners?

[Minoru Asada]
So far I do not have a direct connection to the industry with my own research. But our department’s curriculum includes projects with two companies.

One is the Daiwa House company. The research topic is the design of an intelligent room. The goal is to control, for example, the air conditioning and the lighting, or to sense the humidity. In order to do that we added face recognition features, recognition of facial expressions, gesture recognition, voice recognition, and so on. So, it is kind of a mixture of that and ambient intelligence.

The second company is Citizen, a clock company. The project was to develop new applications for a watch-like device. We started to design a watch-sized device that measures, for example, the temperature, [the wearer’s] pace, or health related data. We also wanted to include some kind of interaction between the user and this device, for example the ability to understand voice commands. These functions could be included in devices like mobile phones.

Of course, in order to realize the product – especially in the case of the Daiwa House company – we need to do some joint research with them. I myself am the PI, and Hiroshi Ishiguro and another professor are involved as well, and contributed to the project. The other professor does the lighting control by gesture, face, or voice recognition. Maybe, in a couple of years, we will try to realize that as a real project. Not as an entire house, but for certain settings, or for some kind of daily life application. I am not sure when we can realize the product we developed in the project with, maybe in a few years …

In your opinion, how could we minimize time it takes for a product from the development in the lab to being ready for sale?

My idea is that I have my own research agenda: this is fundamental research  – and by that I mean development, cognition, and so on. This might seem to be far away from real applications, but I actually do not think that is true. Suppose that the main goal is to reveal the mystery of human development and cognitive functions, for example how a baby can remember something. This involves a lot of steps with lots of different processes. And to do this research we have to design some kind of material, or certain equipment, or a robot. And suppose that this technology can lead to other applications. For example, very soft skin with sensors can also be applied to non-humanoid robots … to furniture or walls, in homes for seniors, to avoid them getting hurt.

There are lots of gaps between even the best ideas and their actual implementation, and closing them is still very far away. In order to fill those gaps, we need a strong team. In our case, for the Daiwa house project for example, we have a collaboration between researchers and industry teams. We have three professors involved, the researchers at Daiwa and some students, altogether more than ten people. These people put a lot of effort into realizing this idea as an actual product.

Is the robotics industry in Japan bigger than in Europe?

We Japanese robot researchers are very proud of our advanced technology. However we are lacking the integration with the real products. The Fukushima power plant was a typical example of this. Many Japanese people expected Japanese robots to be there and to help with rescue operations. But that did not happen. Instead US robots [iRobot’s Packbot and Warrior] were deployed there for the operation. A lot of Japanese people are very disappointed about this, and they are very passionate about this issue.

Another issue is that we have very advanced technology but we are lacking a real – let’s say – verification. For example, 10 or 20 years ago we had a joint research project in Japan: the government spent a lot of money, and companies and researchers designed a [rescue] robot. The ministry then decided that [a rescue-need] situation would not happen and that the robot should focus on inspection only. So, they designed such a rescue robot – but only once. After that they put it in storage and no one used it.

So, we have advanced technology in Japan but we are not good at integrating it in real situations. My idea is to have a site for realistic robot experiments, like a completely new town. There was supposed to be a new town in Osaka. However, last year there were elections for the mayor in Osaka and this new mayor did not support us. There was also the earthquake, and Japanese industry suffered huge losses. For example, Panasonic used to have a research group for robotic applications in a hospital, but they reduced the funding due to a lack of money. These two issues, the earthquake and the new mayor, made me give up on such a site for robot experiments.

In Europe, Paulo Dario proposed RCC – a robot companion for citizens. He was inspired by Japanese robotics but he started to set up these big robot projects that not only involve industry but also research and education. If he receives funding for the RCC, that is going to be a big movement.[note: the RCC project was not funded by the European Commission]

In Japan we have advanced technology but there is no good connection between researchers, the government and industry. Ideally, these three should form a triangle; they should cooperate with each other. But now the government in Japan is just focused on the disaster situation. The industry on the other hand is weakened. Therefore, researchers are losing their connection to the government and industry. Of course the situation you see depends on which industry you take into account: The robot industry in Japan is going through a very hard time right now. It is supposed to only focus on the disaster situation and lacks fundamental research. For fundamental research we need a long term perspective.

In the wake of the earthquake a lot of companies cut their budget for R&D activities. Would you say that this would be a good time for Japanese universities and industry to move closer together?

Right now Japanese companies cannot afford this, as they are focused on the profit. They have suffered from the deficit and are continuously cutting their expenses. Therefore, it is actually not a good time for industry and university connections. I would propose that we focus on topics that will make companies realize that there is some immediate profit to be made. If a research group has a good idea for a device or material that is immediately applicable to the industry, then it might work. But except for such special cases, it is very difficult right now.

If someone has a good idea that should make it into industry, how should researchers communicate such results?

One issue is that Japan is an aging society. In a couple of years one third of the population will be over 65. So, robot applications for the aging society are a social demand – not only for physical but also for mental assistance. There is a lot of potential there. This gives government, the industry and universities an opportunity to collaborate. Safety has top priority here: Physical contact with senior people needs to be improved, and we need time to do research and real-world experiments.

Another issue is entertainment. The origin of the robot is a clockwork-doll. Robots can also be appliances such as door stoppers – any kind of everyday device. The shape is similar to us but it does something strange – and that is kind of fun. In this case you do not need physical interaction – it is not always necessary.

These applications – both for senior citizens and entertainment – are both far away, but they can share some technology. Therefore, a scientist dealing with fundamental research issues can apply their results to different areas. Industry on the other hand needs immediate profit, so they focus on certain topics.

The industry measures success in profit, and researchers’ measure theirs in publications and citations. When these two worlds meet, how would you measure the success of such industry-academia collaboration?

That is a big issue. Even if there is a lot of media coverage, this does not guarantee the success of a product. A typical example for this would be a car. If an automobile company shows a design for a future car and many people like it, the company will introduce it to the market only to realize later that it is “good” but not bought. It’s just to look at.

Therefore, I would propose that we start a good collaboration with some immediate application to the real world and then gradually introduce it to the market. By collaboration, I mean university and industry again. In my opinion the government’s role is to lead this kind of collaboration.

How can increase industry-academia collaboration be increased, and the current problems be overcome?

I mentioned a triangle – the industry, the government and the university or academia. There are still large gaps between these. Therefore, we need someone to connect these, a kind of gatekeeper or producer. Not a researcher but someone with a sense of the market. They should know the demand and need and then determine what kind of product has a big potential.

In Japan we lack the general scientist to promote research. In the US there are general scientists who also are science journalists. These people are necessary. But in Japan science journalists can’t afford a life. I suppose that we need these kind of people to bridge the gap between industry and university. We need people who are not researchers and not in the industry but have a sense of both.

The second interview of this four-part series about collaboration between academia and industry was conducted at the CITEC Summer School 2012 at the Center of Excellence “Cognitive Interaction Technology”, Bielefeld University in Germany. Sascha Griffiths, member of the ECHORD team at TUM interviewed Hiroshi Ishiguro, who is famous for his humanoids with lifelike appearance. Prof. Ishiguro is ATR fellow and director of the Intelligent Robotics Laboratory at Osaka University.

[Sascha Griffiths]
If you are involved in collaborations between academia and industry, in which areas do you prefer to cooperate?

[Hiroshi Ishiguro]
I have been cooperating with industry for a long time. In 2000 I started a venture company using my patents, where a small human-like robot is being developed. In collaboration with my venture companies, our team was a RoboCup champion four times. There we have lots of good engineers who build pretty good robots. In fact, we were so strong that the people started hating us, so eventually we did not attend anymore.

I am always working with big enterprises that build big systems, for example Toshiba, NTT, Fujitsu or Hitachi. For my venture company the market is smaller. We can develop a very simple, toy-like robot, and we don’t need to have a big market. But on the other hand – in order to change this world – we need to work with big enterprises, and we need to develop more of a general, powerful robot, like for example Honda’s Asimo.

In Japan, almost nobody studies human-robot-interaction. But I have studied it, and therefore all the enterprises are coming to ask my opinion, and we are always running small collaborations. The rest happens at the university. ATR (Advanced Telecommunications Research Institute International, Intelligent Robotics and Communication Laboratories) is very big with the government-supported projects, and its members are big enterprises.

I am involved in university operations and in ATR operations. The university is kind of an incubator. At universities, it is easy to work with venture companies. Once we have good ideas, we can start something new, and then we negotiate with the government about running a much bigger project at ATR. And ATR is no university – it is a private research organization. Therefore it is easier to collaborate with other private organizations, like big enterprises. The companies do not trust universities very much, because a university cannot take any responsibility. It is kind of a collection of amateur people, with so many students. Work at ATR, on the other hand, is different: Everybody is a professional, and we can make any contract.

How would you measure a successful collaboration between a university and industry?

For the companies involved, the most important goal is to have a real product. To have a big success it usually takes a long time. Of course it would be better to submit more patents or journal papers, and to share the know-how; however, the most important thing is internal evaluation, because this is not the paper writing game we play at the university. Cooperation with big enterprises is a very serious game. We do not need to have any external evaluation because we are spending our own money. Universities use government funds, and therefore they need to have external evaluations. That is the difference!

What are the main obstacles in academia/industry collaborations?

It depends on what you want to do. We cannot have pure collaborations. If we want to do real business we should leave the university. If I cannot do it at the university, I just do it at ATR – a private research institute. So my recommendation is to just have two types of cooperation. Obviously, a university is not the ideal place for doing practical work. A specialized institute that is independent of universities is a better platform for exchanging knowledge between industry and academia.

In Asia, is there a very active collaboration between industry and academia?

The situation in Japan is quite strange. We have many good, big enterprises in Japan, but they do not spend a lot of money on Japanese universities. However, they are spending a lot of money on US universities. Their excuse is that they are [already supporting universities by] paying a lot of taxes in Japan. I work at a national university – [and Japanese companies think] that means I am using their tax money. But I do not think that is true! In the US, the situation is probably more natural. Universities are supported by many companies. So, collaboration activities between industry and academia are better in North America than in Asia.

Do you know how the collaboration works in Europe?

I am not sure, but I guess you are doing very well, right? Your universities keep growing and you have a lot of good collaborations with industry, I think. Your universities are very traditional, and you are always working with industry. But in Japan the situation is different. Many people think universities should be independent from industry. In Japan, universities are kind of in a different world that is independent from economics. The fundamental idea is that studying something should be different from earning money. In Japan, the school traditionally was in a temple, and a temple was independent from business matters. A lot of people still think that a university professor cannot have a private company.

In your opinion, how could you improve knowledge transfer and collaboration between academia and industry?

The most important thing is to have the right people. I started my venture company with three people: myself, the president, and a student. The role of my student was technology transfer. He was the best student at Osaka University. I recommended he should get a PhD, but he said he could get a PhD anytime. For a venture company, the timing is quite important, so he said he was going to do something for the venture company. We need to have that kind of person.

How do you measure the success of a robotics research project?

If you can sell a lot of robots, that means a big success. The sales, the real numbers are important. In science, if you can write a pretty good scientific paper, that is also a good success. All we need are two simple evaluations: one for science and one for business. We don’t need to have any other evaluation. Most other kinds of evaluation are artificial. If you find very important things, you will have a good reputation in science, and you can get the Nobel Prize. And if you develop a very useful robot, then someone will sell it.

Thank you very much for your time!

This is the first in a four-part interview series about collaboration between academia and industry, conducted by a group of scientists from the ECHORD project at Technische Universität München. Well-known and established experts in the field of robotics were asked: How can one connect industry and academia effectively? What obstacles are there and what makes collaborations effective? 

The first expert is Rodney Brooks, who was interviewed by myself in Eskilstuna during the Robotics Innovation Challenge 2012. Brooks is professor emeritus at MIT and a successful robotics entrepreneur who just recently launched Rethink Robotics’ Baxter.

[Florian Röhrbein]
What are future research fields in robotics that should be worked in collaborative effort by both academia and industry? Are there specific fields where you think they are especially suited for this sort of collaboration?

[Rodney Brooks]
In general I think there can be collaborations across all fields as long as the expectations are right on both sides. Academia should only be involved if you are not too close to the product. In terms of technical readiness level it would be a 6 or even less. By the time it gets to 6 I think academics have lost interest.

Also the different reward systems of universities and companies seem to be a problem.

Yes, but you can also view this as a strength! With different reward systems innovations are different in the two places and you get the best from both. You cannot view either as a subset of the other.

Are there platforms or tools developed by companies that should be used more in academic robotics research?

It is interesting that ROS, the robot operating system, is now pervasive. One of the systems out of my lab, YARP, started to get used by the RobotCub people. But now ROS has taken over. In ROS you get a lot of tools that then become accessible and can easily be moved around – for example kinematic or dynamic tools. I think it is not so much ROS itself that has become important, but rather that there is now a place people can use as a repository, so that those tools start to become more universal. I think it is like how Open GL made graphics much easier … The same sort of thing is happening with robots.

Looking at this topic the other way round: are there emerging topics that you feel researchers should focus more on in order to ease future collaborations with industry partners?

I think that in industrial robots, and manufacturing in general, we have not really seen the impact of information technology in the same way we have seen it in the office and information spaces. It has been very slow to be adopted. When information technology or computation is brought into manufacturing, it is brought in like it was with mainframes. Imagine, if you had to program in COBOL in order to use this device – it would be useless! But I think that is the level where we are with trying to bring new technologies to manufacturing: right now it is all about how to achieve them and not about making them simple to use. It is the simplicity of use that leads to a high rate of adoption, and I think we have not seen that penetration yet. And this involves both industry and academia. These things have to be people-centric. Adapt them to people, not the other way around. This is where the big payoff is going to be.

Which routes of knowledge transfer do you consider most efficient?

I think the only way it can happen is by exchange of personnel, by people moving back and forth. I ran the Computer Science and AI lab at MIT with 839 people involved. We had a lot of collaborations with large and small companies, but it was only satisfying and successful when people really knew people in the other organization and moved back and forth. Academics were spending time at the companies and vice versa … several months, sometimes a year at a time. The companies and the academics have to be willing to make that investment. But no one wants to make that big an investment if there is no payoff!

What else would you consider a best practice for facilitating this cooperation between industry and academia?

In my experience, the most successful cooperations were those where we set up a joint lab. We did that with Nokia, Quanta Computer from Taiwan and with NTT from Japan. In each case, people from the company and from the university are equals, on all aspects. It is not “We’re the people with the money, we pay you”, or “We’re the people with the ideas”.

What do you think are the main obstacles in collaborative projects between academia and industry and what would be the means to overcome them?

Here’s a generic problem that I’ve see happen many times: The CEO of the company says “I want to bring innovation into my company by working with academia”, goes and signs a deal with some academic group, and then the CEO hands it down two levels in the company. And now it’s someone’s assignment: “Oh, I’m supposed to get innovation from …”

[interrupts] And then it doesn’t work anymore.

It doesn’t work at all. You’ve got to have someone who really understands what is wanted and is part of their mission and not get it handed off to people … they have to cultivate some group that is really going to do it. Within the company, the CEO says “Yes we’re gonna do this.”

Is this somehow collated with the size of the company? It might be easier for small companies because there are not as many levels involved.

Even in a small 50-person company I have seen it not quite working. Surprisingly, one very large company where I have seen it work very well recently is John Deere. The former CEO, Bob Lane, and the current CEO, Sam Allen, keep pushing it, and keep showing up at the places. And everyone in the company sees that this is important, that they really mean it. The CEO has to mean it by participating at some level. So they are running along to demonstrate they are on board.

Regarding upcoming research trends and robotic applications, do you think there are differences between North America, Europe and Asia?

The most important thing is cross-disciplinarity. I think Japan has a large failure on that, because robotics tends to be in mechanical engineering departments. These departments are so strong in academia that they do not have computer scientists or electrical engineers working on robotics, but only mechanical engineers. In my experience, places where the boundaries between the departments are much more flexible do better – for example with a lab structure across multiple departments. The differences are not necessarily by country, but also by institution. Some are too department-driven and others, like MIT, have a strong lab culture. It really depends on the individual institutions.

Thank you very much for your time.