talk – Robohub

Dan Mantz: Growing Interest in STEM, Soft Skills, and Labor Shortages | Sense Think Act Podcast #21

Audrow Nash — Tue, 05 Jul 2022 17:22:13 +0000

In this episode, Audrow Nash speaks to Dan Mantz, who is the CEO of the Robotics Education and Competition (REC) Foundation. The REC Foundation is a nonprofit that works with VEX Robotics to build interest in STEM related career opportunities. Dan speaks about the different competitions REC facilitates, why REC is a nonprofit, how REC works with VEX Robotics, on the importance of soft skills, on workforce gaps, and on the future of education.

Episode Links

Podcast info

Maria Telleria: Construction Robots and Working with Unions | Sense Think Act Podcast #20

Audrow Nash — Tue, 21 Jun 2022 17:22:10 +0000

In this episode, Audrow Nash speaks to Maria Telleria, who is a co-founder and the CTO of Canvas. Canvas makes a drywall finishing robot and is based in the Bay Area. In this interview, Maria talks about Canvas’s drywall finishing robot, how Canvas works with unions, Canvas’s business model, and about her career path.

Episode Links

Podcast info

Dan O’Mara: Turning Robotics Education on its Head | Sense Think Act Podcast #19

Audrow Nash — Tue, 24 May 2022 18:45:27 +0000

In this episode, Audrow Nash speaks to Dan O’Mara, who is the founder and COO of Circuit Launch and Mechlabs. Circuit Launch is a space for hardware entrepreneurs to work in Oakland, California, and Mechlabs is a project-based course to learn robotics. This interview is mostly about Mechlabs, but talks about the origins of Circuit Launch, including how it is not a maker or coworking space and its business model. For Mechlabs, we talk about several of its aspects that make it different than a university education in robotics, including how there are mentors not instructors, how projects are scoped, and how people are invited to work on what is most interesting to them. We also talk about the future of Mechlabs and how it fits with current universities.

Episode Links

Podcast info

Afreez Gan: Open Source Robot Dog, Kickstarter, and Home Robots | Sense Think Act Podcast #18

Audrow Nash — Sun, 08 May 2022 16:13:02 +0000

In this episode, Audrow Nash speaks to Afreez Gan, who is the founder and CEO of MangDang; MangDang is a Chinese startup that makes Minipupper, an open source robot dog that uses the Robot Operating System (ROS). Minipupper was inspired by and built with lessons learned from the Stanford Pupper, an open source robot dog. In this interview, Afreez speaks about Minipupper’s hardware, software, and documentation, about manufacturing Minipupper, and about his eventual goal of building a home robot dog.

Episode Links

Podcast info

Dayo McIntosh: A Robot Mixologist to Spark Wellness Talks | Sense Think Act Podcast #17

Audrow Nash — Thu, 14 Apr 2022 16:30:51 +0000

In this episode, Audrow Nash speaks to Dayo McIntosh, who is the founder of Yateou; Yateou is an early-stage startup that makes wellness products and has a customer facing robot, ADE, that personalizes customer orders. Dayo speaks about what motivated her to start Yateou, how she uses the robot arm, ADE, and about her plans for Yateou’s future.

Episode Links

Podcast info

Helen Greiner: Solar Powered Robotic Weeding | Sense Think Act Podcast #16

Audrow Nash — Tue, 29 Mar 2022 08:00:20 +0000

In this episode, Audrow Nash speaks to Helen Greiner, CEO at Tertill, which makes a small solar powered weeding robot for vegetable gardens. The conversation begins with an overview of Helen’s previous robotics experience, including at as a student at MIT, Co-founder at iRobot, Founder and CEO at CyPhyWorks, and in advising government research in robotics, AI, and machine learning. From there, Helen explains the design of the Tertill robot, how it works, and her high hopes for this simple robot: to help reduce the environmental impact of the agriculture industry by helping people to grow their own food. In the last part of the conversation, Helen speaks broadly about her experience in robotics startups, the robotics industry, and the future of robotics.

Episode Links

Podcast info

Christian Fritz: Full-stack Robotics and Growing an App Marketplace | Sense Think Act Podcast #15

Audrow Nash — Wed, 09 Mar 2022 19:33:23 +0000

In this episode, Audrow Nash speaks to Christian Fritz, CEO and founder of Transitive Robotics. Transitive Robotics makes software for building full stack robotics applications. In this conversation, they talk about how Transitive Robotic’s software works, their business model, sandboxing for security, creating a marketplace for robotics applications, and web tools, in general.

Episode Links

Podcast info

Tim Chung: DARPA Subterranean Challenge | Sense Think Act Podcast #14

Audrow Nash — Thu, 24 Feb 2022 16:01:52 +0000

In this episode, Audrow Nash speaks to Tim Chung, Program Manager in the Tactical Technology Office at the Defense Advanced Research Projects Agency (DARPA), on the DARPA Subterranean (SubT) Challenge. The SubT Challenge is a robotics challenge that aims to develop innovative technologies that would augment operations underground. In this conversation, they talk about the motivation of the SubT Challenge, its systems (hardware) and virtual (simulation) challenges, how the resulting technology has been shared with the world, on benchmarking the robotics behaviors in simulation, on the challenge of scoping the SubT Challenge, and on building the final environment for the systems challenge.

Episode Links

Podcast info

Steve Macenski: Open Source Robot Navigation | Sense Think Act Podcast #13

Audrow Nash — Wed, 09 Feb 2022 17:45:35 +0000

In this episode, Audrow Nash speaks to Steve Macenski, who is the Open Source Robotics Engineering Lead at Samsung Research America. Steve leads the Nav2 project, which is an open source robot navigation framework. In this conversation, they talk about the problem and challenges of robot navigation, how Nav2 works at a high level, hybrid planners, and about Steve’s experience working with the Nav2 community.

Episode Links

Podcast info

Matt Robinson: Accelerating Industrial Workflows with Open Source | Sense Think Act Podcast #12

Audrow Nash — Tue, 25 Jan 2022 22:33:10 +0000

In this episode, Audrow Nash speaks to Matt Robinson, Program Manager for ROS-Industrial Americas at the Southwest Research Institute. ROS Industrial is a group that seeks to help industrial users, for example factories, leverage ROS and its ecosystem. In this conversation, they talk about Matt’s background, the need for the ROS-Industrial group and what problems they’re working to solve, ROS-Industrial’s consortium, and ROS-Industrial’s new working group.

Episode Links

Podcast info

CBQ: Commercial-grade Autonomous Mowers, Safety, and Dogfooding | Sense Think Act Podcast #11

Audrow Nash — Tue, 11 Jan 2022 21:57:21 +0000

In this episode, Audrow Nash speaks to Charles Brian Quinn (aka, CBQ), CEO and a Co-Founder of Greenzie. Greenzie make an autonomous driving system for commercial lawn mowers. We talk about Greenzie’s autonomous mowing system, how Greenzie has worked with manufacturers to up-fit their system into commercial mowers, how Greenzie does dog-fooding, safety and standards, and about CBQ’s experiences bootstrapping and with venture capital.

Episode Links

Podcast info

Brett Aldrich: State Machines for Complex Robot Behavior | Sense Think Act Podcast #10

Audrow Nash — Thu, 30 Dec 2021 01:02:38 +0000

In this episode, Audrow Nash interviews Brett Aldrich, author of SMACC and CEO of Robosoft AI. Robosoft AI develops and maintains SMACC and SMACC2, which are event-driven, behavior state machine libraries for ROS 1 and ROS 2, respectively. Brett explains SMACC, its origins, other strategies for robot control such as behavior trees, speaks about the challenges of developing software for industry users and hobbyists, and gives some advice for new roboticists.

Episode Links

Podcast info

Youssef Benmokhtar: Digitization of Touch and Meta AI Partnership | Sense Think Act Podcast #9

Audrow Nash — Tue, 14 Dec 2021 20:19:16 +0000

In this episode, Audrow Nash speaks to Youssef Benmokhtar, CEO of GelSight, a Boston-based company that makes high resolution tactile sensors for several industries. They talk about how GelSight’s tactile sensors work, GelSight’s new collaboration with Meta AI (formerly, Facebook AI) to manufacture a low cost touch sensor called DIGIT, on the digitization of touch, touch sensing in robotics, how GelSight is investing in community and open source software, and Youssef’s professional path in several industries.

Episode Links

Podcast info

Jason Richards: Robotic Food Delivery, Crowdfunding, and Working with Lawmakers | Sense Think Act Podcast #7

Audrow Nash — Tue, 16 Nov 2021 18:23:58 +0000

In this episode, Audrow Nash speaks to Jason Richards, CEO at DaxBot. DaxBot makes a charismatic robot for food delivery. Jason speaks on their delivery robot, their crowdfunding campaign, DaxBot’s revenue model, working with lawmakers to have robots on the sidewalks, and Jason teases their realtime operating system, DaxOS.

Episode Links

Podcast info

Tobias Holmes: Agriculture Robots, Herbicide Resistance, and Education | Sense Think Act Podcast #6

Audrow Nash — Tue, 02 Nov 2021 16:56:23 +0000

In this episode, Audrow Nash speaks to Tobias Holmes, Quality Assurance Manager at Blue River Technologies. Blue River uses computer vision and robotics in agriculture and was acquired by John Deere in 2017. Tobias speaks about herbicide resistance, spraying weeds, quality assurance and testing on hardware, and on encouraging kids to learn robotics.

Episode Links

Podcast info

Sense Think Act Pocast: Erik Schluntz

Audrow Nash — Tue, 19 Oct 2021 08:07:04 +0000

In this episode, Audrow Nash interviews Erik Schluntz, co-founder and CTO of Cobalt Robotics, which makes a security guard robot. Erik speaks about how their robot handles elevators, how they have humans-in-the-loop to help their robot make decisions, robot body language, and gives advice for entrepreneurs.

Episode Links

Podcast info

Sense Think Act Podcast: Melonee Wise

Audrow Nash — Mon, 11 Oct 2021 09:55:02 +0000

In this episode, Audrow Nash speaks with Melonee Wise, former CEO of Fetch Robotics and current VP of Robotics Automation at Zebra Technologies. Melonee speaks about the origin of Fetch Robotics, her experience at Willow Garage, her experience being acquired by Zebra Technologies, challenges in the warehouse setting, on autonomous cars, and on the future of robotics.

Links

Subscribe

Sense Think Act Podcast: Adrian Macneil

Audrow Nash — Tue, 05 Oct 2021 15:57:10 +0000

In this episode, Audrow Nash interviews Adrian Macneil, Co-founder and CEO of Foxglove. Foxglove makes Foxglove Studio, an open source visualization and debugging tool for robotics. Adrian speaks about the origin of Foxglove, Foxglove’s business model, web and robotics, and gives advice to those interested in getting more involved in robotics.

Episode links

Subscribe

Sense Think Act Podcast: Dave Coleman

Audrow Nash — Sat, 02 Oct 2021 09:00:22 +0000

In this episode, Audrow Nash interviews Dave Coleman, Chief Executive Officer at PickNik Robotics. Dave speaks at a high level about what MoveIt is and what problems it helps roboticists solve, they talk about supervised autonomy, including a collaboration with NASA and MoveIt Studio, and Dave talks about MoveIt 3.0.

Episode links

Subscribe

Sense Think Act Podcast: Ryan Gariepy

Audrow Nash — Tue, 21 Sep 2021 22:42:56 +0000

In this episode, Audrow Nash interviews Ryan Gariepy, Chief Technology Officer of both Clearpath Robotics and OTTO Motors. Ryan speaks about the origin of Clearpath, how Clearpath focuses on making reliable robots, and the future of robotics.

Episode links

Subscribe

Robots that feel by seeing

Carlo Sferrazza — Sat, 06 Mar 2021 09:30:07 +0000

Humanoid robot hand. Futuristic cyborg concept.

While modern cameras provide machines with a very well-developed sense of vision, robots still lack such a comprehensive solution for their sense of touch. At ETH Zurich, in the group led by Prof. Raffaello D’Andrea at the Institute for Dynamic Systems and Control, we have developed a tactile sensing principle that allows robots to retrieve rich contact feedback from their interactions with the environment. I recently described our approach in a TEDx talk at the last TEDxZurich. The talk features a tech demo that introduces the novel tactile sensing technology targeting the next generation of soft robotic skins.

Design

The sensing technique is based on a camera that tracks fluorescent particles, which are densely and randomly distributed within a soft, deformable gel. The randomness of the patterns simplifies production of the gel and their density provides strain information at each pixel of the resulting image. In addition, the technique does not make any assumption about the shape of the sensing surface, which can exhibit an arbitrary geometry.

Data Processing

While the images capturing the particles’ motion are intuitive and to some extent visually interpretable, the extraction of accurate physical quantities is challenging. In order to overcome the complexity of modeling the behavior of soft materials in real-time, the information extracted from the images is mapped to the distribution of the applied (shear and pressure) contact forces in a data-driven fashion. Specifically, a neural network is employed, which is trained entirely via accurate finite-element simulations to extract the aforementioned mapping.

Applications

This technology has the potential to impact several application fields, the obvious one being robotic manipulation. In a recent proof-of-concept, we have demonstrated how highly dynamic manipulation tasks can be achieved via the sole use of tactile sensing, as shown in the video below.

In addition, the versatility of this approach makes it suitable for various products beyond the robotics domain. In fact, an artificial sense of touch may find applications in smart prosthetic systems, with the potential to restore tactile sensations to people who have lost limbs.

How to build a robotics startup: the product idea

Ricardo Téllez — Tue, 23 Feb 2021 13:03:46 +0000

In this podcast series of episodes we are going to explain how to create a robotics startup step by step.

We are going to learn how to select your co-founders, your team, how to look for investors, how to test your ideas, how to get customers, how to reach your market, how to build your product… Starting from zero, how to build a successful robotics startup.

I’m Ricardo Tellez, CEO and co-founder of The Construct startup, a robotics startup at which we deliver the best learning experience to become a ROS Developer, that is, to learn how to program robots with ROS.

Our company is already 5 years long, we are a team of 10 people working around the world. We have more than 100.000 students, and tens of Universities around the world use our online academy to provide the teaching environment to their students.

We have bootstrapped our startup, but we also (unsuccessfully) tried getting investors. We have done a few pivots and finally ended at the point that we are right now.

With all this experience, I’m going to teach you how to build your own startup. And we are going to go through the process by creating ourselves another startup, so you can see in the path how to create your own. So you are going to witness the creation of such robotics startup.

This episode is about deciding the product your startup will produce.

Subscribe to the podcast using any of the following methods

ROS Developers Podcast on iTunes
ROS Developers Podcast on Stitcher
ROS Developers Podcast on Spotify

Or watch the video

The post 89. How to build a robotics startup: the product idea appeared first on The Construct.

IEEE RAS Soft Robotics Podcast with Ali Khademhosseini: Biomaterials, soft robotics and the Terasaki Institute

Marwa ElDiwiny — Tue, 26 Jan 2021 08:54:37 +0000

Interesting discussion with Prof. Ali Khademhosseini, CEO of the Terasaki Institute, and one of the pioneers of the Bioengineering field. Prof. Ali’s journey from Harvard and UCLA to the Terasaki Institute is truly inspiring. What does the institute do to bring a product to the real world? Learn about the design challenges of biomaterials, organs on a chip, and soft robotics in this episode of the IEEE RAS Soft Robotics Podcast.

Soft Robotics Podcast · Ali Khademhosseini "Biomaterials, Soft Robotics, Terasaki Institute"

You can also listen to this podcast on Spotify and Apple Podcasts, and explore support options on Patreon.

IEEE RAS Soft Robotics Podcast with Hod Lipson: Can we design self-aware robots?

Marwa ElDiwiny — Wed, 13 Jan 2021 15:35:20 +0000

Interesting discussion with Hod Lipson, head of Creative Machines Lab, Columbia University in New York. Can robots be self-aware? Can they design other robots and self-repair? Why should we evolve robots to do tasks that animals do so well? Why don’t we have useful autonomous robots in the real world yet? Find out Hod’s answers to these questions and updates on VoxCAD development for designing and simulation of soft robots in this episode of the IEEE RAS Soft Robotics Podcast.

What’s more, Hod gave his personal advice to roboticists being interviewed for an assistant professorship and to 1st-year robotics PhD students looking for a thesis topic, and he also commented on his approach to the ethical dilemma of military funding scientific research. I hope you enjoy listening to the episode! You can check Hod Lipson’s portfolio here: https://www.hodlipson.com/

Soft Robotics Podcast · Soft Robotics With Hod Lipson

You can also listen to this podcast on Spotify and Apple Podcasts, and explore support options on Patreon.

Talking Machines: The long view and learning in person, with John Quinn

Talking Machines — Sat, 23 Sep 2017 17:34:00 +0000

In episode nine of season three we chat about the difference between models and algorithms, take a listener question about summer schools and learning in person as opposed to learning digitally, and we chat with John Quinn of the United Nations Global Pulse lab in Kampala, Uganda and Makerere University’s Artificial Intelligence Research group.

If you enjoyed this episode, you may also want to listen to:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: Machine Learning in the Field and Bayesian Baked Goods, with Ernest Mwebaze

Talking Machines — Fri, 08 Sep 2017 15:37:14 +0000

In episode eight of season three we return to the epic (or maybe not so epic) clash between frequentists and bayesians, take a listener question about the ethical questions generators of machine learning should be asking of themselves (not just their tools) and we hear a conversation with Ernest Mwebaze of Makerere University.

If you enjoyed this episode, you may also want to listen to:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: Data Science Africa, with Dina Machuve

Talking Machines — Fri, 25 Aug 2017 06:25:45 +0000

In episode seven of season three we take a minute to break way from our regular format and feature a conversation with Dina Machuve of the Nelson Mandela African Institute of Science and Technology. We cover everything from her work to how cell phone access has changed data patterns. We got to talk with her at the Data Science Africa conference and workshop.

If you enjoyed this episode, you may also want to listen to:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: Getting a start in ML and applied AI at Facebook, with Joaquin Quiñonero Candela

Talking Machines — Tue, 01 Aug 2017 20:56:00 +0000

In episode five of season three we compare and contrast AI and data science, take a listener question about getting started in machine learning, and listen to an interview with Joaquin Quiñonero Candela.

For a great place to get started with foundational ideas in ML, take a look at Andrew Ng’s course on Coursera. Then check out Daphne Kohler’s course.

Talking Machines is now working with Midroll to source and organize sponsors for our show. In order find sponsors who are a good fit for us, and of worth to you, we’re surveying our listeners.

If you’d like to help us get a better idea of who makes up the Talking Machines community take the survey at http://podsurvey.com/MACHINES.

If you enjoyed this episode, you may also want to listen to:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: Bias variance dilemma for humans and the arm farm, with Jeff Dean

Talking Machines — Mon, 03 Jul 2017 15:38:00 +0000

In episode four of season three Neil introduces us to the ideas behind the bias variance dilemma (and how how we can think about it in our daily lives). Plus, we answer a listener question about how to make sure your neural networks don’t get fooled. Our guest for this episode is Jeff Dean, Google Senior Fellow in the Research Group, where he leads the Google Brain project. We talk about a closet full of robot arms (the arm farm!), image recognition for diabetic retinopathy, and equality in data and the community.

Fun Fact: Geoff Hinton’s distant relative invented the word tesseract. (How cool is that. Seriously.)

If you enjoyed this episode, you may also want to listen to:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: Overfitting and asking ecological questions, with Professor (Emeritus) Tom Dietterich

Talking Machines — Fri, 16 Jun 2017 11:00:14 +0000

Credit: Wikipedia Commons

In episode three, season three of Talking Machines, we dive into overfitting, take a listener question about unbalanced data and speak with Professor (Emeritus) Tom Dietterich from Oregon State University.

If you enjoyed this episode, you may also want to listen to:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking machines: Graphons and “inferencing”

Talking Machines — Tue, 30 May 2017 22:00:02 +0000

In episode two of season three Neil takes us through the basics on dropout, we chat about the definition of inference (It’s more about context than you think!) and hear an interview with Jennifer Chayes of Microsoft.

If you liked this article, you may also want to read:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Hosts of Talking Machines: Neil Lawrence and Ryan Adams

Talking Machines — Fri, 28 Apr 2017 14:51:00 +0000

Talking Machines is entering its third season and going through some changes. Our founding host Ryan is moving on and in his place, Neil Lawrence of Amazon is taking over as co-host. We say thank you and goodbye to Ryan with an interview about his work.

If you liked this article, you may also want to read:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Brian Cox presents Science Matters: Machine learning and artificial intelligence

Robohub Editors — Wed, 11 Jan 2017 18:41:50 +0000

We’re beginning to see more and more jobs being performed by machines, even creative tasks like writing music or painting can now be carried out by a computer.

But how and when will machines be able to explain themselves? Should we be worrying about an artificial intelligence taking over our world or are there bigger and more imminent challenges that advances in machine learning are presenting here and now?

Join Professor Brian Cox, the Royal Society Professor of Public Engagement, as he brings together experts on AI and machine learning to discuss key issues that will shape our future. Panelists include:

Professor Jon Crowcroft FRS, Marconi Professor of Networked Systems at the University of Cambridge
Professor Joanna Bryson, Reader in AI Ethics, University of Bath and Affiliate, Center for Information Technology Policy, Princeton University
Professor Sabine Hauert, Lecturer in Robotics at the University of Bristol, and President and Co-Founder of Robohub.org
Professor Murray Shanahan, Professor of Cognitive Robotics at Imperial College London
Professor Peter Donnelly FRS FMedSci, Director, Wellcome Trust Centre for Human Genetics, and Professor of Statistical Science, University of Oxford

https://www.youtube.com/watch?v=IunNpGGt3H8

You might also enjoy the following articles about AI and AI/robotics policy:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: ANGLICAN and Probabilistic Programming

Talking Machines — Fri, 04 Nov 2016 10:30:19 +0000

In episode seventeen of season two, we get an introduction to Min Hashing, talk with Frank Wood the creator of ANGLICAN, about probabilistic programming and his new company, INVREA, and take a listener question about how to choose an architecture when using a neural network.

If you liked this article, you may also want to read:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: Restricted Boltzmann Machines, with Eric Lander

Talking Machines — Fri, 09 Sep 2016 10:09:00 +0000

In episode sixteen of season two, we get an introduction to Restricted Boltzmann Machines, take a listener question about tuning hyperparameters, plus, speak with Eric Lander of the Broad Institute.

If you liked this article, you may also want to read:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: Generative art and Hamiltonian Monte Carlo, with Doug Eck

Talking Machines — Wed, 17 Aug 2016 13:40:00 +0000

Generative Art on PBS/YouTube

In episode fifteen of season two, we talk about Hamiltonian Monte Carlo, take a listener question about unbalanced data, plus, speak with Doug Eck of Google’s Magenta project.

If you enjoyed this article, you may also want to read:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: Machine learning and the Flint water crisis, with Jake Abernethy

Talking Machines — Wed, 27 Jul 2016 14:36:00 +0000

Testing lead in water during the Flint water crisis.

In episode fourteen of season two, we discuss Perturb-and-MAP and answer a listener question about classic artificial intelligence ideas being used in modern machine learning. Plus, we speak with Jake Abernethy from the University of Michigan about municipal data and his work on the Flint water crisis.

If you enjoyed this article, you may also want to read:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: Automatic Translation and t-SNE, with Hal Daume

Talking Machines — Fri, 08 Jul 2016 21:24:00 +0000

Reuters dataset (in 2D) landmark t-SNE using semantic hashing. Source: vdmaaten.github.io/tsne

In episode thirteen of season two, we talk about t-Distributed Stochastic Neighbor Embedding (t-SNE), take a listener question about statistical physics, plus, speak with Hal Daume of the University of Maryland (who is great to follow on Twitter).

If you enjoyed reading this article, you may also enjoy:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Talking Machines: Fantasizing cats and data numbers, with Iain Murray

Talking Machines — Mon, 20 Jun 2016 21:43:00 +0000

Generating faces with Torch. Photo source: torch.ch

In episode twelve of season two, we discuss generative adversarial networks, take a listener question about using machine learning to improve or create products, and lastly, speak with Iain Murray from University of Edinburgh.

If you like this podcast, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

ep.210: ICRA 2016 Exhibition (Part 1 of 2), with Scania, PAL Robotics, Husqvarna and AnyBody Technology

Audrow Nash — Sun, 12 Jun 2016 18:21:17 +0000

This is the first of two episodes where Audrow Nash interviews several companies at the International Conference on Robotics and Automation (ICRA). ICRA is the IEEE Robotics and Automation Society’s biggest conference and one of the leading international forums for robotics researchers to present their work. The 2016 conference was May 16-21 in Stockholm, Sweden.

And don’t miss our #ICRA16 photo essay!

Interviews include the following:

Soheil Salehpour, Development Engineer from Scania, speaks on autonomous trucks.

Judith Viladomat, Corporate Communications at PAL Robotics, discusses a humanoid robot for research labs.

Bjorn Mannefred, Research Engineer at Husqvarna Group, talks about autonomous, robust lawn mowers.

Mohammad Shourijeh, Research and Development Engineer at AnyBody Technology, speaks on human body simulation.

Links:

Talking Machines: Spark and ICML, with Sinead Williamson

Talking Machines — Mon, 06 Jun 2016 18:16:00 +0000

Source: Pexels/CCO

In episode eleven of season two, we talk about the machine learning toolkit Spark and answer a listener question about the difference between Neural Information Processing Systems (NIPS) and International Conference on Machine Learning (ICML). Plus, we speak with Sinead Williamson from The University of Texas at Austin.

If you like this podcast, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

ep.209: INNOROBO 2015 Showcase, with RB 3D, BALYO, Kawada Robotics, Partnering Robotics, and IRT Jules Verne

Audrow Nash — Sun, 29 May 2016 04:07:19 +0000

In this episode, Audrow Nash interviews several companies at last year’s INNOROBO, a conference that showcases innovation in robotics.

Interviews include the following:

Oliver Baudet, business manager at RB 3D, speaks about the exoskeletons displayed at the showcase.

Baptiste Mauget, responsible for marketing and communication at BALYO, speaks about BALYO’s robots for warehouse automation.

Atsushi Hayashi, an Engineer at Kawada Robotics, demonstrates a humanoid used in factories in Japan.

Abdelfettah Ighouess, Sales Director at Partnering Robotics, describes their robot for indoor air quality control.

Etienne Picard, a Research and Development Engineer at IRT Jules Verne, speaks about a large cable driven robot for manufacturing.

Links:

Talking Machines: Computational learning theory and machine learning for understanding cells, with Aviv Regev

Talking Machines — Tue, 24 May 2016 12:00:00 +0000

Stem cell. Source: CC0

In episode ten of season two, we talk about Computational Learning Theory and Probably Approximately Correct Learning originated by Professor Leslie Valiant of SEAS at Harvard, we take a listener question about generative systems, plus we talk with Aviv Regev, Chair of the Faculty and Director of the Klarman Cell Observatory and the Cell Circuits Program at the Broad Institute.

If you like this podcast, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

ep.208: Ladybird, with James Underwood

Podcast team — Sat, 21 May 2016 11:31:53 +0000

Transcript below.

In this episode, Ron Vanderkley interviews James Underwood from the Australian Centre for Field Robotics. Underwood discusses his work on an autonomous vegetable harvesting robot, Ladybird.

Ladybird is a lightweight omni-directional electric vehicle, inspired by the Coccinellidae insect (Ladybird or the ladybug insect). It is equipped with sensing, manipulation, communication hardware and software. Various user interfaces will be developed for the growers, contractors, and harvesters so that they can control the robot and use information derived from the system.

James Underwood

James Underwood is a senior research fellow at the Australian Centre for Field Robotics (ACFR) at The University of Sydney. James is an expert in the area of perception systems for field robotics – the study of how outdoor robots working in complex, unstructured environments can make sense of their world using science and technology in multi-modal sensing, data fusion and mapping.

Links:

Transcript has been edited for clarity.

Ron Vanderkley: Good morning James! Can you introduce yourself to our podcast listeners?

James Underwood: Hi, my name is Dr. James Underwood. I’m a researcher at the Australian center for field robotics, which is a robotics research group at the University of Sydney. I’ve been working there for the last several years, in applications of robotics to agriculture, specifically focused on ground-based crops like vegetables, and recently cereals and grains, and also tree crops.

Ron Vanderkley: Great. We’ve heard a lot about a particular robot you are working on. I don’t know if it’s the only one, but it’s the one that seems to be predominant, called Lady Bird. If you could describe the work that you’re doing on that particular machine.

James Underwood: Yeah, the Lady Bird is the robot, we custom designed it for the vegetable industry. Where we’ve also done a lot of research on a different platform called Trent, which we’ve used on tree crops. That was more of a general purpose. This one has really been designed specifically for the vegetable crop industry. We built this as a kind of research platform which would have a number of different senses, that are relevant in terms of a new wave of research in crop sensing, and so on. It’s also equipped with a general purpose manipulator arm, so we can actually interact with the crops.

In terms of the physical construction of the system, it has omnidirectional wheels so all four wheels point any direction. It’s a very maneuverable platform. It can go from one row to the next, without using very much of the headlamp, because we can kind of come out and just go straight sideways and back in. Another interesting feature of the platform, is it’s covered in solar panels. These actually have two functions. They have covers, so we can actually shield some of the senses underneath that, from the direct sunlight so we get better imagery of the crops that we are looking at.

Also, then of course we can cover up the solar panels, and use the energy that we are getting, in these typical, very open field applications. On a sunny day, getting a lot of energy into that all electric powered system that allows us to go for days on end without recharging.

Ron Vanderkley: You briefly talked about the senses on the Lady Bird, etc. The frequencies you were talking about were infrared, optical, there’s laser scanning, 3-D imaging involved. Are there any other type of sensory equipment involved in the machine itself?

James Underwood: We also do, we use, I don’t know if you mentioned stereo, that’s just a variation in general, allowing us to estimate the 3-D shape of instruction on the camera that we’re looking at. We also have a couple of senses, like in terms of GPS guidance and things like that, we have international sensor on board as well. The list that you’ve mentioned basically covers it. We’re looking at a lot of the different parts of the optical spectrum, as you say. From visual, to infrared and so on. We have high spectral senses to look at that spectrum in detail.

Also, with the femoral imaging, which is long wave infrared as well. So covering a large range of the electromagnetic spectrum, sense of approach there.

Ron Vanderkley: What are the difficulties in autonomous pest control, actually not just pest control, there’s weed control. Are you using other things apart from chemical sprays, steam, microwave? Mechanically, have you looked at things like that?

James Underwood: We have done a little bit of an explanation in terms of, as I mentioned, we have this general purpose manipulator robot arm on the platform. We can put a utensil on the end of that to actually mechanically remove weeds and some. But we haven’t really, at this stage, done extensive testing. There is quite a lot of work there to be done, in terms of particular, how fast could that type of thing be done. We could stop and then identify the weed, then position the arm in order to remove it.

Would that be fast enough for realistic commercial applications, is the question we don’t really know the answer to at this point. There are some other groups around the world who have looked at, for example, mechanical weeding mechanism, in particular using a stabbing mechanism. Which so rather than actually carefully hauling out the weed like that, you just sort of punch it directly from the barb. That type of technology looks like it might show some promise, and allow these sort of things to be done more rapidly.

Certainly our initial viewpoint on this, is using a rapid target and spray mechanism, which can be used even when the vehicle is in motion, for example, it can find weeds and shooting at them as it rolls past. While it is desirable to remove the chemical element all together, I think it’s probably a good first step. We can probably reduce the quantity of the chemical that is used, but maybe keep up the pace of the robot while it goes over and deals with these weeds. Certainly mechanical weeding is something that we’re rather interested in, because we can get rid of that chemical element all together. That may be a very interesting way to deal with herbicide resistance, and problems of that nature.

Ron Vanderkley: Great, I don’t know if I brought it up, but microwave was looked at at one stage. I don’t know how viable that is. Is that something that has been acknowledged?

James Underwood: I think there are a few groups who have looked into that. To my knowledge, I don’t know that there are any viable systems of that nature yet, but that is also something we are very interested in. There are many challenges that have to be solved with the microwave type application. For example, the power consumption that that device would require. Safety considerations in terms of actually having high voltages in the system, and things of that nature. I believe that the research shows that it can be an effective mechanism for dealing with weeds. Certainly in terms of constructing such a system with a robot and so on. They could be put together, there’s no reason why not.

I think that there’s more work to be done in that regard, to do a proof of concept of a useful whole system.

Ron Vanderkley: This is an interesting one for me. How do you create the section of a weed or a pest in an algorithm to create software to deal with it. Is it visual references, colors, leaf shapes, like in botany, or the location of it versus where the plant is?

James Underwood: Well this is an interesting question for us, too. This is exactly the type of algorithms that we’re interested in, and so on. A lot of the things that you mentioned are relevant. Obviously what you need to be able to do, is to detect those weeds reliably , so to minimize false detections where you think part of the crop is the weed, and you would then spray it. You want to avoid that kind of thing. Similarly, you want to have a sufficiently high rate of detection to actually deal with the weed infestation.

It’s no good if you go through a field and miss a large percentage of the weeds that are there. So the objective is to obviously do this reliably, with low error rates. Now, all of the things you mentioned can be used to detect weeds. So you can use the color camera, and in some cases, the weeds are a different color. You could use high spectral sensing, which let’s you, really the same sort of thing as a color camera, although maybe it tends to go into the infrared range as well. It’s a very detailed way of looking at color.

You where you might have two things that are almost the same shade of green, if there’s a kind of a statistically significant, albeit nuanced difference in the color pick that out more, then the cost of the sensor goes up. It may make the algorithms more simple, and the whole system may be more reliable , but of course if you can do it with the color camera, that’s a better approach.

The key with all of these things, where you’re using color, you’re using texture, all of those things together to try and separate the difference between weeds and the crop is that you have to be able to deal with the variability that you get. Not just in the different instance of the same species of weed will look different, but you also you have lot of variation just due to sunlight changes and things like that. So being able to do this reliably is really a key thing. It’s quite a challenging task.

In a lot of cases the approach that we take, are from state of the art in terms of computer vision, is to try not to specify hand coded rules. We try to avoid saying a weed is something that is this green, and it’s this size, or something like that. We try to get large data sets where we can, and learn directly from the data, what facets of that data from the senses really best eliminate the weed class and the crop class in question. It’s kind of a standard approach. It’s generally based on color, and it’s based on texture, but not necessarily in a human specified way, like a particular shape or a particular color or something like that.

You mentioned the position of the weeds location on the ground. That’s another really important one, because crops tend to be planted in straight lines with the seed. The probability of any vegetation that you may find off that seed line, the probability is much higher that it’s actually a weed. You can encode that knowledge into the system, and actually let it learn what weeds look like, by finding the examples that are off the seed line. Learning from those, and then learning automatically to find those same weeds amongst the seed line as well. There’s the kind of approach that we are taking.

There can be a lot of sophistication in how that’s done, and that’s kind of necessary in order to get those accuracies up, and to be able to do weeding once the field is no longer of color basically. When the field is basically fallow, detecting weeds is easy, because anything that’s green you just shoot with commercial applications with that already.

Ron Vanderkley: Cool, so what are the advantages and disadvantages of autonomous harvesting, fertilizing and planting today?

James Underwood: There are a number of different advantages. One of them is, obviously, labor saving and a lot of the growers that we talk to, that’s kind of the top of there list. They have difficulties with labor supply in Australia, and I understand there are similar issues in similar countries around the world. A lot of the work is done by temporary migrant workers.

We have an aging workforce. The average age of farm workers in this country is actually surprisingly high like 55 or 60 years old and the younger generation don’t seem to be showing as must interest to get involved in that area. When we talk to growers, the cost of labor and the security of the labor force is pretty much number one on their list. So clearly, autonomous systems have an advantage in that regard.

There are many other advantages you start to get when you have an autonomous system, such as the repeatability, the ability to operate 24/7 without getting tired and keeping that attention to detail at 100% capacity for the entire time that it’s working. There are advantages as far as buyer security, applications in terms of disease detection and management. And advantages having fewer instances where people handle the food that we eat. All these things have benefits for the quality of the produce and the safety of the food.

Whichever stem you pick up, you look on there and see a whole number of different advantages that might not necessarily be the top of the grower’s list but when you start to look at them all coming together in this one system that can provide many different advantages altogether, then it starts to look like a pretty good way to move.

Ron Vanderkley: What would be the ideal goal in crop management using multiple small robot tractors?

James Underwood: There are a number of advantages. One of the key things with using smaller autonomous systems, and so on is that you have a much lighter mass in terms of the vehicle. That has benefits in terms of soil compaction. Productivity in farming over the last few decades has really been driven by getting bigger and bigger equipment so that the one person driving a tractor can tend to normal hectares per day, simply by virtue of having a bigger machine. The cost of that is much greater, soil damage and soil compaction that that causes. This has been kind of mitigated to some extent by having control of traffic farming. The vehicle puts its wheels over exactly the same position so the extent of where the damage is caused is minimized. The size of these machines really does do a lot of damage to the soil. Smaller vehicles means really avoiding that problem.

As I said before, there are all these advantages jumping out at you. For example, if, by virtue of having a smaller machine, you might need to have several of them to replicate the same number of hectares per hour that you could do in a much bigger machine. But now, if one of your ten smaller machines breaks down completely, you’re still running at 90% capacity. Whereas, when that one big machine, which cost a fortune incidentally, breaks down, you’re down to 0% capacity and there’s very little that you can do about that. By having teams of smaller platforms you get a scalability and you get a resilience to malfunction, you can keep going with a subset of that team.

A small farmer could have just one of these systems. A really big farmer, property, could use a hundred of them or however many are required to scalable, better for breakages and things like that, better for soil compaction.

Ron Vanderkley: It’s also an approach to integrated farm management using farm bots both aerial and ground in the production, is that a viable option as well?

James Underwood: It doesn’t exist today as a system you can just buy and use. We’re talking in the research, where the technology is heading. There would definitely be scenarios where aerial vehicles could team up with ground vehicles to very effectively manage certain aspects of a farm. We haven’t seen it enough in practice to get a hands on view of what those benefits actually look like. It takes longer for a ground based vehicle to scan across an entire [inaudible 00:16:28] where it is much quicker for a flight vehicle to go over the top at a higher altitude and get a lower resolution snapshot of the status of the crop. If there was an identifier from that data, then they could go and do some sort of targeted management response based on what you saw from the air. You could then send vehicles specifically to the locations on the ground that need attention, whatever that might be. It is likely that there will be combinations of different bits of technology that work together and combine in that way.

Ron Vanderkley:
From the grass-root approach, What are interests local farmers? Where do they want us to go with robotics?

James Underwood: I mentioned land saving being the top of the list of the growers we have spoken to. I think they’d like to see more technology coming online that can … other assisted technology that allows greater areas to be managed in some sense using this technology rather than doing everything by hand or technology that in some cases can completely replace certain functions that are currently done now by manual labor. I think growers would like to see that. This could include a number of different things, for example, weeding is something that depending on the crop and the time of year, certainly there is a manual labor component involved in that. I have been to farms where they have to send people out to just look at the fields. They walk the whole field and look at the crop to make sure that there are no contaminants in the field. That could be leaves that blow in from adjacent, native trees and things like that. You don’t want gum leaves in your bag of spinach. As a consumer we’re fussy, you could have robots going out and doing that job.

There’s a lot of labor saving type approaches that tend to be high up on list. Where we are now we are doing some early prototypes. Growers are going to want to see the technology be actually ready to be used in a commercial sense. We still need to move in that direction. Early prototypes are based on functions. We need get a few more miles under our belt before we’re ready to sell the equipment.

Ron Vanderkley: Where to you see the future in robotics saw in the next ten or twenty years, for instance in agriculture?

James Underwood: Ten to twenty years is a very very long time, especially in terms of technological progress. It’s really hard to predict that far forward with any kind of accuracy.

Within that time frame, I think it would be good to see entire processes of the production, certain types of crops, could be done in a digital fashion. I don’t necessarily mean that there are no people involved. I mean that you have technology at all the right key points in the production cycle. All the information that’s relevant to growing the crop in an optimal kind of way. It’s all recorded and stored and used in terms of making optimal decisions and putting down the right inputs and someone to grow the best possible crop, that crop to be as resilient as possible, [inaudible 00:20:07] all using digital technology. That can be a big combination of different types of actual hardware like robotics is one part. It could also be stationary senses. It could be satellite data or UAV data. It could be handheld senses, senses mounted onto conventional vehicles, a whole number of things like that.

All that data is then coming together to really address all the information requirements across the whole production cycle. That, coupled with various automated and optimized processes, in terms of harvesting. If we’re saying in twenty years time then I hope by then that we manage to really digitize this whole system. In the near term, I think that we’ll start to see individual applications be kind of kicked off the list. Not a general type of robot, in the next five years, can do all of these things but maybe a package that you can add to a tractor that will specifically target weeds in a non fallow period. That’s a piece of technology that will come into commercial existence fairly soon. It could be integrated fairly easily into current practices.

You might then see certain crops that now are not harvested in a mechanized way, start to become solved in the mechanization. A lot of the crops are harvested in a mechanized way, like with spinach, you have a machine that has a sword in the front and cuts it off and on to a conveyor belt it goes. It’s all manually driven but it’s quite an efficient method for harvesting. Where those crops that are currently not mechanized, there is opportunities to kick them off the list. Not necessarily one special harvesting robot with arms that goes out and deals with all the different crops but say an asparagus harvester and it’s using modern robotics technology to do the job and it just does that one job very well. In the next five years or so we’ll start to see one by one these kinds of specific applications getting kicked off the list and eventually you start to get this kind of matrix of connectivity between all these different applications that start talking to each other and dealing with the whole production cycle in the bigger picture.

Ron Vanderkley: What do you think of the options for new players to robotics, for instance in agriculture, are there a lot of opportunities for a graduate be engineers?

James Underwood: Yeah, I guess that’s a fairly broad question. I think so. I think the industry by and large is sort of seeing the potential in this kind of technology. Throughout the whole chain we’re starting to see more funding in fundamental Blue Sky Research, more funding in applied research, leading towards commercialization opportunities, more commercialization funding to get those off the ground and go forth. There are more commercialization opportunities popping up all the time. There are brand new opportunities in building service companies that supply the services.

We’re sort of at the start of the ramp I think at this stage. We will start to see some of these companies become … their products will start to be used more universally and there will be the best practice in terms of farming. Then we will have hundreds and hundreds of these systems, whatever they look like, out there in the field. They will need servicing and maintenance, just like tractors do today. There is no magic behind robotics that somehow you don’t need to maintain them and look after them and always develop them. Every stage in the pipeline, there will be increasing opportunities from Blue Star Research all the way through to the actual filing practice.

We hope that actually … the demographics of those who work on the farm today is kind of an aging population and the younger people tend not to be so interested in farming as a career choice right now. There are kind of hopes that as it becomes more high tech, that it might actually attract again the younger generation into those sorts of roles and make those kinds of jumps, and fix that kind of demographic issue.

Ron Vanderkley: James, thanks on behalf of the Robots Podcast for a most interesting session.

Talking Machines: Sparse Coding and MADBITS, with Clement Farabet

Talking Machines — Mon, 09 May 2016 08:58:22 +0000

In episode nine of season two, we talk about sparse coding, take a listener question about the next big demonstration for AI after AlphaGo. Plus we talk with Clement Farabet about MADBITS and the work he’s doing at Twitter Cortex.

If you like this podcast, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

ep.207: Evolutionary Approaches for Flying Robots, with Guido De Croon

Abate De Mey — Sun, 01 May 2016 13:13:18 +0000

Transcript below.

In this episode, Abate De Mey interviews Guido De Croon about Evolutionary Robotics and its use to design behaviors for flying robots. They discuss a recent paper by Kirk Schepe et al., in which the DelFly UAV robot learns to fly through an open window when trapped inside a room thanks to a controller optimized using a Genetic Algorithm. The controller is programmed using a Behavior Tree Framework, which is more intuitive and adaptable than the traditional Neural Network framework. This helps the user to manually adapt the controller to handle the differences between the simulation and the real world. They go on to discuss the challenges and benefits of using Evolutionary Robotics to learn robot behaviors.

Video of the Evolutionary Robotics strategy used to develop a controller for the DelFly robot:

Guido De Croon
Guido de Croon is Assistant Professor at the Micro Air Vehicle lab of Delft University of Technology in the Netherlands. His research interest lies with computationally efficient algorithms for robot autonomy, with a particular focus on computer vision and evolutionary robotics.

Links:

Transcript
This transcript has been edited for clarity.

Abate De Mey: Welcome to the Robots Podcast. Could you please introduce yourself to our listeners?

Guido De Croon: Hi, I’m Guido De Croon and I’m assistant professor in the Micro Air Vehicle Lab of TU Delft in the Netherlands. I work on the artificial intelligence of tiny robots.

Abate De Mey: You also do work with evolutionary robotics. Can you talk about evolutionary robotics, and what are some real world applications of them?

Guido De Croon: Evolutionary robotics is a domain in which we chase the following dream: we would like to have a general method that if we give a task for a robot, we only need to come up with the fitness function. A way of saying how good a robot is doing the task. Then, there’s an artificial evolutionary process that takes over the design work and evolves the robots, like the body, sensors, actuators and the controller do this task.

Abate De Mey: What are some things that you optimize using evolutionary robotics tactics?

Guido De Croon: Typically, it’s a task you want the robot to do. You come up with a fitness function, which is actually a difficult part of evolutionary robotics. In the last few years, we’ve also seen special fitness functions like rewarding robots to do novel stuff. We call this novelty search. In my case, I’ve been evolving robots to do different tasks, such as finding odor sources. Recently, I’ve been evolving small robots to perform the task of flying in a room, avoiding obstacles, finding a window and flying through the window.

Abate De Mey: What are some limitations of evolutionary robotics?

Guido De Croon: I don’t think there are fundamental limitations to evolutionary robotics because we will have technologies in the end to perform an actual evolution with it. I think the major limitation at the moment is that if you want to do evolution on real robots, it will happen in real time, which means it can actually take a very long time. The worst case it really takes as long as a real evolution. You can imagine that … Especially if we want a complex robot to do a complex task, then this might take a really long time.

That’s why most people in evolutionary robotics evolve robots in simulation. You have a simulation with your robot in it. It has a task to do. You can test out thousands of what we call individuals to different robot solutions. You can have the best ones procreate and create offspring. Then at the end of this artificial evolutionary process and simulation, you need to transfer the best solution to the real world. This is a difficult step because simulation goes fast also because you don’t model everything that’s happening in the real world. The real robot you use is typically different than the one that was used in simulation, so this is a challenge.

There’s difference between the simulation and reality in the field of evolutionary robotics. It’s called the reality gap. Crossing the reality gap is another major challenges at the moment.

The second limitation that’s there is the complexity of the task it outperforms is pretty limited. We have really quite studies on finding light or avoiding obstacles, things that can be done with simple controllers. The question is: how well the approach scales also to more complex problems? Finding other sources in turbulent conditions or really navigating in an environment, being able to find something and then return to the home location.

Abate De Mey: What are some tactics you use to overcome the reality gap?

Guido De Croon: There are different tactics that have been developed. One way is to do an evolution in simulation, then taking the solution and continuing the evolution on real robots. You perform a lot of the evolution in simulation, and then just perform final adaptations on the platform. Still, the real robot may be quite different. It may take generations before you get the right solution. This way of handling the reality gap still requires a long time.

There are other ways though. There are ways in which the mismatch between simulation and reality is modeled during evolution. What they do then is once every generation, for example, one in every hundred generations, they try out an individual on real robots. They take the difference in the performance. They try to model this in the simulation and try to adapt the simulation and predict if solutions will work in the real world. That’s another approach.

I think that the final approach, which may be very successful, is to combine evolutionary learning, which is a slow process, with developmental capabilities. They call this evo-devo. It means that you evolve the robot and its controller, but you also evolve adaptive mechanisms that will allow the robot to also learn online, so during operation.

Abate De Mey: Have you implemented evo-devo before in a project?

Guido De Croon: Yes, some while ago. Until now it’s generally acknowledged that this will be important, but the golden bullet there has not been discovered yet. I do think that in the future this is promising.

The question why not is perhaps a good one as well. Why didn’t we try that yet? Also, because the learning mechanism has to be really fast. In my case, I focus on robots such as small flying robots. Suppose you combine evolutionary learning with reinforcement learning, still the robot will have to do reinforcement learning. Also reinforcement learning itself can take quite some time. It involves risks for the robot. Then you don’t really solve your problem. You really need to find the right combinations of developmental algorithms and evolutionary algorithms.

Abate De Mey: Could you tell us about a project where you have implemented evolutionary robotics?

Guido De Croon: Yes. I’ve implemented the evolutionary robotics in quite a few projects. I think one of the ones that’s perhaps a very good example is recently we implemented evolutionary robotics for our tiny flying robot called the DelFly Explorer. This is a dragonfly-like robot. It weighs 20 grams. In those 20 grams, we have a stereo vision system so two cameras and a small processor with which the DelFly can see obstacles and can do obstacle avoidance.

Now, we developed this DelFly Explorer a few years ago. At the time, we showed that it could avoid obstacles, fly around completely by itself, without any human intervention or without any off-board computer. Our PhD student Sjoerd Tijmons, he made what I think is a very elegant algorithm to control the robot. The main idea behind the algorithm is that the DelFly always keeps a space in front of it, a free space, large enough to make a turnaround maneuver. If it sees an obstacle coming into the zone, it will do this prefixed maneuver. It always turn in front of the obstacle and then it can circle as long as it wants until it finds a new flight direction. This, I think, can be compared a bit with some insect behaviors that you see in nature. We were, of course, very happy with it.

Then we thought now we want to do another task so so we want, for example, our DelFly to fly through a window, to fly in a room, find a window and fly through it. Do we then have to search another Sjoerd, another PhD student again to come up with another very nice algorithm? The answer actually is no. It will be great if we can give this task to an evolutionary process and that this evolutionary process finds a solution that’s, not only effective and robust, but also as computationally as efficient as the solution we found ourselves for obstacle avoidance. This is very important, for example, for these small flying robots because the processor that we have onboard, for example, it has only a 192 kilobytes of memory.

The nice thing about evolutionary robotics is that you can beforehand define the structure that evolution has to evolve, and you can ensure that it’s computationally efficient enough. If it then finds a solution, then it fits within the constraints of the onboard platform. This project we did last year, it gave us really some insight into this matter and into how to cross the reality gap.

Abate De Mey: How successful was the DelFly, which was optimized with the evolutionary robotics framework, at flying through the window?

Guido De Croon: That’s, of course, the most important question. The approach we took is quite different from what is typically done in evolutionary robotics. In evolutionary robotics it’s, of course, bio-inspired, inspired by natural evolution. This also typically implies that neural networks are used as the robot’s controller because this is also bio-inspired. It’s parallel. Of course, neural networks are very capable, especially also the deep neural networks. The problem of neural networks in evolutionary robotics is that it’s pretty hard to understand what evolution came up with. I’m not saying it’s impossible, not at all.

For multiple tasks, I’ve evolved neural networks, for example, also for odor source finding. When you do that afterwards, you can study it. You have the neural networks. You can, for example, clamp neurons to a certain value, or you can influence the input that the robot has, and you can analyze what happens then to the actions. Actually you can understand what it’s doing, what the strategy is.

When I was doing a study on odor source localization, I had robots in simulation performing really well. I tried to understand the networks. Then I was like, “When can you say that you really understand this?” In order to test whether I understood the strategy, I re-implement the strategy that evolution found in the neural network, and I re-implemented it as a finite-state machine. Then I ran the finite-state machine, and it got a similar performance as the original robot.

Then I thought, “Hey, this finite-state machine, if you look at it, you understand immediately what the robot is doing.” Suppose that you have this type of controller and you put it on a real robot, if something is not working correctly, then you can probably understand what is going wrong and adapt it yourself. That’s why when we wanted to evolve a controller for the real DelFly, we took a similar kind of controller. Instead of a finite-state machine, we took a behavior tree.

Behavior tree is a controller that is used a lot in the gaming industry because it’s pretty intuitive. In comparison with finite-state machines, it does not have what we call a state explosion. When you add complexity to the controller, it doesn’t immediately lead to a huge explosion of the controller structure. With the behavior tree, you can solve complex problems and the controller will still look pretty compressed, so pretty small.

In our case, for the DelFly flying through the window, we evolve the behavior trees. Evolution came up with a very elegant solution that was smaller than the hand designed one that we made ourselves and that had a higher success rate in simulation. Then, of course, the question was if we have this behavior tree, can we now understand it? Will this help us if we go to the real world?

We took the controller from simulation and put it into the real DelFly. Of course, the first time we flew it or the first few times, it would just crash against the wall. That was not very nice. Since we could easily see in what part of the tree it was active, we could easily adapt some parameters in the behavior tree to really boost the performance and get it more similar to the simulation performance.

Your original question was how successful was it? Well, in simulation it would get through the window 90% of the time. After tuning it on the real robot, it only flew through 53% of the time or 55. Don’t test me about the exact number. That’s, of course, much, much lower and really not satisfactory. The reason was that the robot which was evolved in simulation would, let’s say, take a bit more risk. It would fly very close to the window border which in simulation didn’t matter. In the real world, we were testing in a pretty artificial environment so a small room that we built indoors with window in it. It turned out that even in this indoor space, there were drafts and these drafts were coming through the window, and they were pushing the DelFly away from the window basically. If you then have a risky strategy of going a bit close to the border and you’re pushed away by the draft, then you actually hit the window border a bit more often than in simulation.

It taught us, yes, these behavior trees, they’re easier to understand. You can then manually adapt them for the real robot, but it doesn’t solve the reality gap completely which, of course, would have been almost too easy, of course, if that would have been the case. The robot exploited the fact that there was no wind in the simulation environment. Currently, we added, of course, wind to the simulation environment, and we’re re-evolving and seeing how that goes. It’s not the full answer to this problem, but it is a very promising step.

Abate De Mey: How adaptable would the controller that you created with evolutionary robotics be to a room with different parameters, different dimensions?

Guido De Croon: That’s also a very good question. Let’s say, partly a different room would be no problem. For example, our DelFly uses stereo vision. If you change the texture in the room, it would be okay. If you would make the walls completely textureless, that could be a problem actually. Then you would need other visual inputs, for example.

In this specific article that we wrote on this study, we only use one size of room. I think it was square and then it would fly through the window. One of the pitfalls of evolutionary robotics is that the solution that you evolved in simulation will exploit everything it can from that environment. If you only evolve in a square room, then it’s possible that it will generalize rectangular rooms. It’s also possible that your solution will be less good. You have to vary the parameters in simulation that you think will vary in reality. That’s, by the way, another thing that we’re doing at the moment for this task.

I think that there’s hardly an answer to that because I think the real answer would be to have the robot also develop by itself in the real world afterwards. There’s still a few other strategies. For example, one of the problems we had in this first study was that in simulation, we had the robot just set a rudder command. No, sorry. An aileron command. That means that it would directly set the ailerons of the robot. If you do that in real life, then this never gives the same result because, for example … Ailerons are like little things behind the wings that make it turn around, that make it change heading.

These ailerons on the real robot are already a bit asymmetric, for example. In simulation, we didn’t have a very accurate model not at all of the DelFly. If you would give an input of plus one, it would turn, for example, with I don’t know, some degrees per second to the right. Minus one would be the exact opposite reaction. In reality, this is not the case. It will turn perhaps easier to the right than to the left. It can even depend on environmental conditions whether it’s really turning with, I don’t know, 10 degrees per second or not.

One of the things we did now as well is to abstract away from those very low level commands and to take a bit higher level. The robot now sets, for example, a turn rate. It has a lower level controller that can actually execute this turn rates. In that sense, you also narrow the reality gap. We’re searching through all these kinds of possibilities and see how far we will get with that.

Abate De Mey: What are the largest hurdles that evolutionary robotics has to overcome to have widespread usage?

Guido De Croon: I think one is this reality gap that I’ve been talking about. One of the main reasons to use evolutionary robotics, especially for, for example, small robots, is that with fairly limited resources, computational, sensory, you’re still able to perform complex tasks. The way in which they do this is typically pretty ingenious. For example, we were talking about this odor source localization or if you think about fruit flies in your kitchen, they always find the … Well, at least in my kitchen, they do. They always find the bananas and the apples, especially in the summer. They really invade my kitchen. They solve this problem of finding the banana mostly by using the scent.

When you look at roboticist, then some of the robotic solutions are pretty complex. What they do is, for example, in the robots, the robot in its mind, it’s going to simulate how odor spreads through turbulent air. When it smells some odor, it will run some probabilistic dynamical model to see what’s the highest probability in its environment are to contain the source of the odor. That’s one approach. It’s computationaly very complex.

If you look at fruit flies, then they actually just alternate between two behaviors. If they don’t smell any odor, they will do casting, which means that they fly into the wind. If they do then smell something, then they will start surging, which means that they move upwind. When they lose the odor again, they will start casting again. This, of course, a very rough description of their behavior. By alternating between these two behaviors, in the end, they find the fruit. They do this in a computationally efficient way.

Now, what we want with evolutionary robotics is for any given task, for any given robot to be able to find such very smart strategies that work for the particular robots. If you want evolutionary robotics to be widely used, it really needs to work on the real robots. You need to cross the reality gap and the challenge then is not only to cross it, but to cross it and still keep this ability for finding these elegant, sensory motor solutions. That’s one large hurdle.

I think the second hurdle, I think I also mentioned it, the complexity of the task. It’s, of course, nice to have a robot that goes around and avoid obstacles. How can we evolve robots to do all kinds of tasks? The robot, for example, has to fly around in a greenhouse, has to deposit some larvae on plants or something and at the same time look for ripe fruits, come back to its nest to recharge, collaborate with the other drones in the greenhouse in order not to fly in each other’s way and to effectively cover all the area in the greenhouse. How do you use an evolutionary algorithm to come at such a complex solution?

I think also there’s a huge challenge. At the moment, we’re looking at these behavior trees also for this reason because behavior trees, they allow for certain sub behaviors to be very easily reused as a sub behavior in another behavior tree. I don’t know if this, of course, will be a step towards such a solution. I think that’s a large hurdle as well because if you wanted to be widespread, it needs to work on the actual robot, and the robots need to do useful and sometimes complex tasks.

Abate De Mey: Are evolutionary algorithms a one size fits all strategy for creating robot behaviors?

Guido De Croon: Not yet but that’s the the dream, Abate. We’re trying to replace ourselves as roboticist. That’s the dream that you can really give us a task. We just put the same evolutionary algorithm at work. It will be great, of course, in the future if you can evolve both the body, and the sensors, and the actuators, and the brain to come up with the ideal solution for this task.

Abate De Mey: How can our listeners learn more about evolutionary robotics?

Guido De Croon: That’s a good question. There is a book that was written a while ago by Stefano Nolfi and Dario Floreano. They talked about the earlier work in this area. I think there’s really some exciting work going on at the moment in evolutionary robotics. The book is a good starting point. More recent work, I would say Google is your friend.

Abate De Mey: Well, thank you very much for coming in today and discussing evolutionary robotics with us, Guido.

Guido De Croon: You’re welcome. It was my pleasure.

Talking Machines: Remembering David MacKay

Talking Machines — Fri, 22 Apr 2016 17:05:00 +0000

Recently Professor David MacKay passed away. We’ll spend this episode talking about his extensive body of work and its impacts. We’ll also talk with Philipp Hennig, a research group leader at the Max Planck Institute for Intelligent Systems, who trained in Professor MacKay’s group (with Ryan).

If you like this podcast, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

ep.206: Mecha Monsters, with Silas Adekunle

Audrow Nash — Sun, 17 Apr 2016 15:33:10 +0000

Transcript below.

In this episode, Audrow Nash interviews Silas Adekunle, Co-founder and CEO of Reach Robotics. They speak about Reach Robotics’ first product, Mecha Monsters: legged, gaming robots that are controlled by a smartphone.

Through Mecha Monsters, Reach Robotics is attempting to connect the physical and digital components of the gaming world. Adekunle also discusses the startup process, including technology accelerators.

Below is a video where Silas reveals Mecha Monsters.

https://youtu.be/D7uHQMB6WE0

And an image of a Mecha Monster.

Silas Adekunle

Silas Adekunle is the Co-founder and CEO of Reach Robotics, a company developing intelligent gaming robots. He established Reach Robotics while still an undergraduate at the University of the West of England, UK. Before Reach Robotics, Silas managed a “Robotics in schools” program in partnership with UWE, the UK government, and HP for 4 years, leading a team of engineers going to schools to develop the interest of young students in STEM through robotics. As a robotics engineering graduate and avid gamer, Silas’ goal is to make robotics more accessible to both young and old through play.

Links:

Transcript
This transcript has been edited for clarity.

Audrow Nash: Hi, welcome to Robots Podcast. Can you introduce yourself to our listeners?

Silas Adekunle: I’m Silas Adekunle, CEO and co-founder of Reach Robotics. I’m from the UK, but we’re currently in San Diego, where we’ve just taken part in the Qualcomm Robotics Accelerator, powered by Techstars.

Audrow Nash: Great! Tell me a bit about Reach Robotics.

Silas Adekunle: Reach Robotics is developing the world’s first intelligent gaming robots. I don’t know how many people are familiar with the movie Big Hero 6, by Disney. There’s an underground battle scene where the kid brings out a robot. It’s like everything you fantasized as a kid — having a real robot companion that can battle, and upgrade. It’s an entertainment consumer product, but the vision’s a lot more than that.

Audrow Nash: What do they look like? They’re called MechaMonsters, correct?

Silas Adekunle: Yes, they’re called MechaMonsters. The branding is about to change, we’ve spent a lot of time in the accelerator so there is a lot of new stuff coming. They look like four-legged robots, quadrapeds, who are well-articulated so they move fluidly. They’ve got weapons on their backs, or weapon ports, and shields, like something out of a sci-fi movie.
Audrow Nash: What are weapon ports? Can you attach new weapons, do they actually do anything?

Silas Adekunle: What we’re doing is supplying game play mechanics. You start with your robot at level one. The more you train by battling other players, or by engaging in activities, the stronger they become.

That means you can start to add on weapons, shields. They don’t just change the way your robot looks. You can tailor your MechaMonsters any way you want, and it also improves your performance in battles. They’ve got electronics that allows the robot to know what functionality to add to the robot itself.

Audrow Nash: How would you describe the weapons? Is it a click on piece that has electronics the robot registers?

Silas Adekunle: Yeah, that’s right. So each weapon has an IDE that plugs into the robot, where we plan to add different types, different levels of functionalities. Weapons will be moving parts, so a bit more premium. Weapons by themselves are little, but look awesome. Each weapon, or add on as we call them, has a piece of electronics that has an ID for the robot.

Audrow Nash:
Are shields entirely in the user’s imagination? Or does it register on the smart phone controller?

Silas Adekunle: When you put the shield on, it clips to the leg of the robot. So it’s a physical item that clips on.

There are ports on the leg of the robot and on the back. We call them hard points. These are all the game play mechanics or terminology for the MechaMonster world we haven’t put out yet, because we’ve been trying to get the foundation right. Over the next few months we’ll be more engaging with early adopters so we know exactly what we put into market next year.

Audrow Nash: What does a battle look like with two users controlling the MechaBox?

Silas Adekunle: You’ll be able to knock your opponents over for higher damage. It’s interactive game play, the way you’re trying to use the world around you to your advantage, hide behind the table leg, things like that. You try and outsmart your opponent. Because the robots are maneuverable we’re focusing on the strategy of the game play. For example, there are skills where you can stun your opponent’s robots for a few seconds. The robot will just stay there and not be able to move because they need five second to wait. That gives you the opportunity to get behind your opponent.

It’s one of those things that’s better experienced than described, sorry to say!

Audrow Nash: What kind of sensors are on the robot? Do they interact in game play or battle?

Silas Adekunle: We’ve got sensors for the robot state, in itself. We have an IMU [Inertia Measurement Unit] in there so that the robot knows its orientation, a sense of where it is. We have infrared images, so that robots know distance from each other and orientation. Those are the metrics that plug into the role, the game play.

Silas Adekunle: For the infrared emitters, do you just have one that emits in all directions, and then so the robots can tell their relative distance from each other? Typically, an infrared LED it’s very directional.

Audrow Nash: Which direction are the infrared emitters and receivers? Are they in the front of the robot?

Silas Adekunle: The four sides of the robot.

Audrow Nash: Are you changing the way they emit based on the side? A sort of light signal that you can detect to tell if the robot is turned?

Silas Adekunle: Exactly. You brought different IDs, so to say, for each side so you know that I’m reading 1 for front, 4 for the back.

Audrow Nash: That’s how you make sure that if the robot attacks and it’s facing the wrong way that it doesn’t register. For an attack to hit, the robots have to be within a certain distance and facing the correct direction?
Silas Adekunle: Exactly. There’re different types of attacks, and this will be for the people that want to get into the game play. For example there are some attacks that don’t care about direction. In game play stuff like aerial effects attacks.

Audrow Nash: Like a ground smash?

Silas Adekunle: Exactly. It doesn’t matter where your opponent is, as long as they are within a certain distance, the orientation doesn’t matter.

Audrow Nash: If I was a slick kid, could I cheat by covering up my infrared detectors in certain directions so my opponent couldn’t hit me from the side or something?

Silas Adekunle: Yeah, you could, but that would make you a spoil-sport. Once we start to get into the game-play testing we’ll have our own methods of making sure that people don’t cheat too much, but at the end of the day in every game you have the hackers and the people that will out-smart everyone else. We like that, people are going to be able to hack this stuff to bits, and we want that. We just hope that our customers will be fair players and not annoy their friends too much and be too much of a spoil-sport.

Audrow Nash: Are you making this open to the development community? Do you see research labs using it?

Silas Adekunle: Definitely, it’s part of the story of the company. This all started because I was eventually going into schools, teaching kids robotics, and it’s something that we love very much, and all of us engineers, we want that. We were careful how we wrote it out to make sure people don’t break their robots, but it’s something that’s part of the plan for us, to have versions that people can just take apart and use for development itself.

Audrow Nash: For the hackers, will it have its own operating system on it, like its own little Linux I can pull up?

Silas Adekunle: We’re considering it, we’re still debating exactly how we’re going to do it. I don’t want to make any promises. We want it to be accessible and easy to use.

Audrow Nash: The primary focus for MechaMonsters is to be put in the toy industry?

Silas Adekunle: Yeah, the entertainment industry. We call ourselves a robotics gaming company for a reason, because we think the amount of work that’s going into this product and the experience that we’re putting together it puts it with Skylanders.

Audrow Nash: What is that?

Silas Adekunle: Skylanders is a game, by Activision, where you can buy figurines and use them to unlock characters in a video game. It’s a product that straddled the physical and virtual world, so the lines are starting to blur. For the product in the next 5 years, the kinds of products that are going to be on market it’s going to be hard to define them as toys or games. We find our-self in this spot where we’re halfway between the toy, the intended market, and half of it’s in the gaming market. To kick off we are releasing this product for entertainment.

Audrow Nash: What kind of people do you see buying this? Is it for children?

Silas Adekunle: From interacting with our customers we’ve had interest from children all the way up to adults. These are people with disposable income that are getting nostalgic, they’ve always dreamt of this as a kid and we’re now giving it to them. We’re lucky because we have a product that appeals to so many people: Boys, girls, adults, people that are very techy to people who just want to see their robot beat the crap out of another robot. We consider ourselves to be a consumer product, where the kind of people that will buy it initially will be tech-savvy people until the brand takes off and becomes a general market item.

Audrow Nash: Tech-savvy people to begin with, not kids?

Silas Adekunle: As part of our go-to-market plan we have a campaign in mind that we’re going to launch, and the kind of people online that will have this campaign are going to be people that have access to the internet and things like that. Maybe some parents that want to buy it for their kids, but in the short term it’s going to be someone that’s no Kickstarter for example, that would be able to have initial access to it.

There’s a reason for that. This is a very complex product, a very unique product, and we want to do it justice. We want to give access to people that will be patient enough to give us critical feedback. This is what we like about the product, this is what we don’t like about the product, so that when it starts to reach mass market the people are getting the best experience that we can offer them.

Audrow Nash: Initially, it’s for just testing and feedback so you can refine it.

Silas Adekunle: Exactly.

Audrow Nash: How did you pick your market in the beginning?

Silas Adekunle: It’s a route that’s worked well for other companies so far. It’s not particularly Kickstarter people, it’s where do you find your early adopters. When you’re taking a product to market, how do I get this product end goal?

Audrow Nash: Kickstarter, IndieGoGo Campaign, sites like this, people that are looking for products, or on Reddit. How did you identify your audience?

Silas Adekunle: By taking the product out there and getting feedback. We actually did some testing online ourselves, we took the product out, for example spent a whole day at a Comic-Con in San Diego, we put our product on the internet across different websites, just looking at what people think about this. Did some preliminary research, as much as a small startup can do, and that was how we identified who the kind of people are that would initially be the right early adopter for this, who could give the right feedback.

Audrow Nash: What is a tech startup accelerator? How has this changed in the startup accelerator? I assume that it’s gone through more refinement of who your target audience will be.

Silas Adekunle: A tech startup accelerator is exactly what it sounds like. The company goes in and they must accelerate you to the next stage. That’s different for each company. The accelerator that we were part of, we had companies that just at the idea stage, we had companies that were at the market validation stage, we had companies that were at the growth stage. It’s tailored depending on which accelerator you go into.

There are some that are hardware focused, ours was hardware focused, and it’s there for each company to take them from stage one and rapidly get them to the next stage for them. For us we need a bit of work on the marketing side of things and quite a bit of work on the product development side of things as well. You’re looking at what market you should be aiming for, not just in the short term but the grand vision for the company, and how do we create a product that addresses that.

Audrow Nash: How did that develop? How did you choose for the early adopters in the tech field, and how was that affected by the tech startup accelerator.

Silas Adekunle: The great thing about accelerator was the network you get out of it. You can talk to founders of other robotics companies and what they’d done to get to where they are today. That was how we figured out our strategy, just talking to 4 or 5 past founders. We discovered that a pattern occurred if you’re a consumer product. What worked best is to put it out there to early adopters, and the best channel to reach them on today is by online; platforms like Kickstarter or IndieGoGo are there to make that easier for you. Then, when you reach your early adopters and you have your direct sales and gotten your feedback, then you start to get into your retail channels. It’s really talking to other founders and identifying what strategy worked for them and looking at how we could apply it to our products as well.

Audrow Nash: My impression would be that the early adopters that you find on these websites would be a much smaller market than say the toy industry. How are you guys maintaining funding until you tap into a larger market? Do you seek venture capitalists? They’ve invested in you?

Silas Adekunle: Yeah, we raised money from investors that share our vision and like what we’re working on. Just like any startup, we raise and then use that money to develop before you get to market. The money you raise is to fill that gap before you’re self-sustainable.

Audrow Nash: Going back to MechaMonsters, there has been previous robotic battle toys. One that I had when I was a kid was called Rumble Bots, and the idea with Rumble Bots was they were little remote controlled robots, you didn’t use a smart phone, but you loaded in different behaviors, attacks, that kind of thing by scanning in cards to them. They would read the cards, and then after they were all loaded the devices would battle. This company went bankrupt. What makes MechaMonsters different than other companies that maybe didn’t do as well?

Silas Adekunle: [00:13:00] It’s a question we get asked all the time because battle robots is an idea that people have had. It’s recurring; you’ve got Battle Bots in the US, Robots in the UK, Real Steel and other movies. It’s something people want. It took us awhile as well to figure out exactly what’s different about us, or why we’re going to succeed.

It boils down to creating a battle robot, and there are loads of companies that have tried this in the past. What you’re trying to do is create an experience. We’ve looked at it and said, “Okay, we’re going to create this experience of people being able to battle each other. How exactly are we going to make it the best experience that they can get?” We’ve taken that approach to everything that we’ve done. Initially to kick this off we took a lot of advice from people in the toy industry, and then took a lot of advice from people in the other industries.

Sorry, it’s a bit of a lengthy answer.

Audrow Nash: That’s fine.

Silas Adekunle: The first thing to create a successful product is that there has to be 3 elements: Competition, collectability, and customization. We have all of the elements in it.

Audrow Nash: Where do you get these 3 from?

Silas Adekunle: These were recurring answers from toy industry people. If you look at products that have been successful, apart from the massive amount of marketing budget that were behind them, fundamentally they had these parts.

Skylander, Pokemon, had these if we’re looking at purely the toy industry. When it comes to battle robots, this is stuff that captures people’s imagination. When people say, “I’m gonna buy a battle, or I want a battle robot,” they’re not thinking I want a plastic toy or something, they’re thinking, “I want that epic, awesome robot that I saw in this movie I watched yesterday.” To really deliver that experience we’ve had to focus a lot on the hardware.

For example the Rumble Bots, I think they’re a price about $40, looking at their costs that means their building materials must have been something like $10. This is a while back, right, so looking at the type of technology that was available for that price at the time, the type of components you could get I don’t imagine would be very good.

Audrow Nash: Yeah, and they were before the cell phone boom so, I mean, you get your IMU on your chip then.

Silas Adekunle: We’ve had to think a lot about the hardware. Our robot is very complex, it has 12 degrees of freedom all together.

Audrow Nash: How many?

Silas Adekunle: 12 degrees, so 3 per leg. That means you can make it create awesome moves just like you’d see in a movie. Then, we had to think about what happens if the leg of a robot breaks, which is why the robots are modular. Things we haven’t really announced yet because it’s early in development but will work soon.

We’ve had a massive think, and it’s user experience driven, so we’re creating an experience, we’ve got to create the hardware to deliver that experience. I think that’s what’s different about us, the timing is right and the approach we’ve taken is right. The reaction and the feedback we’ve been getting points that way. When we get to market there’s going to be a lot of stuff that we’ll get wrong at the start, and that’s why we appreciate the feedback that we receive to tailor it.

All of us on the team, we’re gamers, we’re engineers, so we’re not going to put something crappy product out there, that’s what we hope anyway. We’re looking at it from the point of view of, “How are we going to create the best user experience possible,” and that’s what’s driven us so far.

Audrow Nash: What channels are you putting in place for feedback?

Silas Adekunle: We’ve got emails, people can send loads of emails, the accelerator, we’ve sent out some surveys to get feedback from customers. We sent out surveys to about 1,000 people.

Audrow Nash: What would you plan once you unveil it for the early adopters’ audience?

Silas Adekunle: What we plan after that? We sent out our surveys to get feedback, and we had some open-ended questions, non-leading questions. These are questions to confirm our hypothesis or to challenge our hypothesis. You always challenge a hypothesis. Then we’ve gone to live events with the people that we think are our target market, for example as I mentioned Comic-Con here at San Diego.

In the long run when the product is out there, we’re going to be doing a lot of face to face interactions. Once we’re at that state that we can start beta testing, there is going to be lots of invites to get people to try this and give us feedback.

The great thing about this product is that it’s digital in every sense of the word, you’re controlling it from a phone. That means we can look at how people are using the product and get instantaneous feedback, which I think is great for the time we’re launching this product, compared to Rumble Bots for example. We can know exactly how people want to play with this robot, what they do with it, and exactly what they don’t like about it and change it on the fly, so to say.

Audrow Nash: It connects to WiFi? It’s Bluetooth that you’re connecting the robot, but can you send updates over Bluetooth?

Silas Adekunle: Not in itself.

Audrow Nash: It has to connect to WiFi somehow?

Silas Adekunle: This is in the software itself. We’ll have a control infrastructure in there for how you control the robot from the app, so because the app is connected to the internet.

Audrow Nash: The brains of the robot are in the smartphone, it’s just a big actuator that you buy as the robot that receives this message. It’s like Fly By Wire for EJ Kumar’s Lab.

Silas Adekunle: There’s leverage in the power of the smartphone, I think, as part of the game place experience itself.

Audrow Nash: Because the brains of the robot are on the smartphone, you’re controlling what it does from the smartphone? You can add new behaviors to the smartphone to make it do new things, and that’s how you can update it, by updating your app or however you’re running your backend?

Silas Adekunle: Exactly. That’s an idea about how it works.

Audrow Nash: It’s a neat way of controlling it. Going back to the startup accelerator, what was it like?

Silas Adekunle: Hectic, I need to sleep. It was a very intense period.

Audrow Nash: Just for context, you’re at the end of it?

Silas Adekunle: Yes, we graduated last week. That finished last week. It was a very intense 4 months for everyone on the team and all the other companies that where here. You’ve got a target that you’re working towards, there’s demo at the end of it with a fixed date. What you show at the end is up to you, but it’s an opportunity to accelerate it or rapidly take your company to the next level.

Audrow Nash: What are these levels?

Silas Adekunle: For example, for a pre-seed company it would be to raise your seed round so that you can get to market faster.

Audrow Nash: That’s get enough founding?

Silas Adekunle: For other companies it’s to, by demo day, have 100 letters of intent that they’re going to use over at the B2B company.

Audrow Nash: It’s progressing to the next levels, so the next level of the company towards bringing a product to market and being successful in market?

Silas Adekunle: Everyone goes into that territory with this mindset of, “We’ve got 4 months and this our deadline,” and you work extremely hard to make that happen. This is all the while trying to take advantage of the networking opportunities that you’re getting. We met a lot of world class people, founders like the founders of Sphero, founders of other robotics companies like PetRo Robotics, all these people that have lots of success stories that you hope one day want to be like them, you get to meet them. You take in all these feedback, it’s just a massive boiler pot of feedback, development, business development, product development, everything all in one. It’s a very intense period. It’s fun at the same time.

Audrow Nash: Before you decided to be in the startup accelerator, what were the pros and cons of being in a startup accelerator you considered?

Silas Adekunle: Depending on where you are. For us, we had to move out of the UK to the US for it. That was a pro because the weather in the UK sucks so no one complains about going to sunny San Diego, but you’re leaving family behind. It’s all part of the startup roller coaster ride, ups and downs, emotions and all that. You’re leaving a well-known establishment or infrastructure that you have behind, something where you don’t know. In terms of the cons, there aren’t that many cons.

Audrow Nash: The pros you mean? Or the cons?

Silas Adekunle: Yeah, there aren’t many cons. The only con I would think is you’re leaving a well-established infrastructure behind for something unknown. Going into it we were all super excited. The pros way outweigh the cons, it’s an opportunity to take your company to the next level, it’s an opportunity to meet world-class mentors, it’s an opportunity to meet other founders, it’s an opportunity to really spend a whole intensive period with your team and get to know them better, and everyone is focused on the same goal and same target. The pros way outweigh the cons if you have a startup company.

There are different levels of accelerators, Techstars is a fantastic one. If you have an opportunity to apply to Techstars accelerator, take it.

Audrow Nash: It sounds like you’ve learned a lot from this accelerator. What would you say is the biggest lesson you learned?

Silas Adekunle: We learned a lot. This might not be the biggest lesson learned because I haven’t had time to reflect, you should take time at the end of accelerator just to sit somewhere and gather your thoughts.

We learned a lot of great lessons. No matter how big the companies are, at the end of the day it comes down to people: people are what make companies successful. Regardless of the product, idea, market, which are all important, it’s the team that’s most important.
At the very early stages of the company it’s important to put together the right team and treat people you find right because they make or break your company.

Audrow Nash: Would you talk a bit about the long term goals of Reach Robotics?

Silas Adekunle: To take over the world.

Audrow Nash: Naturally.

Silas Adekunle: Naturally, as a robotics company what else do you do? For us the main thing we want is to focus on consumer entertainment, then we see an opportunity for us in education. If we want to shorten it, robotics and entertainment, it should be followed by Reach Robotics.

For us it’s really about getting the product to market, receiving feedback, and always creating something that delivers a great user experience. I think in the toy industry as opposed to the consumer tech industry, there isn’t a lot of brand loyalty because you can just release a new brand with one hit product each year, and then create another one next year.

There isn’t really a name that sticks around as opposed to Samsung, Apple, and all these giants in other industry, in tech. We want to change that and really focus on the user experience we’re developing and create a lasting company, as opposed to a one-hit-wonder.

Audrow Nash: What are some of your ideas on doing that, creating a strong brand?

Silas Adekunle: Community is important. Again, we can’t really dive into the products, but what we want people to be able to do is to create skills for their robots and share them online. If I create a new battle skill I can share it with my friends, so there’s a community aspect. If you design a shield or a weapon for your robot, then you’d like to 3D print, we want that infrastructure in place to provide. We want to focus on brand and community as part of our key defining features for the company.

Audrow Nash: How useful would you say was your university experience, engineering in robotics background, for your current endeavor with Reach Robotics?

Silas Adekunle: Make-or-break, so to say. You can’t go into robotics without having a background in robotics, you wouldn’t know what you were doing at least to get off the ground. I still don’t know what I’m doing, but I knew enough to get off the ground. There’s always new technology coming out and you’re picking it up, but the foundation is important and university education gave me that foundation to have a broad understanding of the robotics space.

There were lots of opportunity for networking in the university. When I was in university I was in this stuff, I think it’s Steam here, we call it Stem in the UK, that was an opportunity to network with other people that were interested in Steam, and it was a university network. I actually found my co-founders though the robotics lab where I was based. I can’t picture being here without the path I’ve taken.

Audrow Nash: Why did you choose to start the company and doing this now immediately after your bachelors, rather than doing more education? How did you make this decision?

Silas Adekunle: Impatience, I suppose. I started the company during my bachelors, during my studies. Whenever you’ve got an itch, you’re not going to wait for anyone you just do it and figure it out along the way.

It’s always been that mentality: we’ll do it live. You get on the road and we’ll just do it live, we’ll figure it out, otherwise you wait yourself to death. It’s never the perfect time and I found that no matter how uncertain you are about something if you just go out there and do it, along the way you’ll find the right people. Even if they don’t have the answer they will point you the right direction towards the answer.

There’s never a perfect time, just go for it, and that was what I did.

Audrow Nash: Thanks for the interview!

Talking Machines: Machine learning and society, with Neil Lawrence

Talking Machines — Fri, 08 Apr 2016 16:11:00 +0000

Episode seven of season two is a little different than our usual episodes; Ryan and Katherine returned from a conference where they got to talk with Neil Lawrence from University of Sheffield about some of the larger issues surrounding machine learning and society. They discuss anthropomorphic intelligence, data ownership, and the ability to empathize. The entire episode is given over to this conversation in hopes that it will spur more discussion of these important issues as the field continues to grow.

If you like this podcast, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

ep.205: Hadrian Bricklaying Robot, with Mark Pivac

Ron Vanderkley — Sat, 02 Apr 2016 21:13:08 +0000

Transcript below.

In this episode, Ron Vanderkley speaks with Mark Pivoc from FastBrick about Hadrian the bricklaying robot. Fastbrick is an Australian robotics firm aiming to disrupt the local bricklaying market with a machine it says will be able to build a four-bedroom house in two days, without any human interaction.

A builder or architect will supply a 3D model, which is then converted into machine code that uses proprietary software to calculate the location of every brick in the building. Laser-scanning technology surveys the foundation once a concrete slab has been laid, and then loads and lays bricks using a 28m articulated telescopic boom. Adhesive is applied to the brick via the robotic laying head, and a laser alignment system ensures an accuracy of within half a millimeter.

Mark Pivac
Mark Pivac is the primary inventor of Fastbrick’s automated bricklaying technology. He is an aeronautical and mechanical engineer with over 25 years of experience working on the development of high technology equipment ranging from lightweight aircraft to heavy off road equipment. Mark has 20 years of experience with pro/engineer 3D CAD Software. He also has high level mathematical experience including matrix mathematics, robot transformations and vector mathematics for machine motion. In addition, he has extensive design, commissioning and fault finding experience on servo controlled motion systems achieving very high dynamic performance.

Links:

Transcript
This transcript has been edited for clarity.

Ron Vanderkley: Welcome to Robots Podcast. Could you introduce yourself to our podcast listeners?

Mark Pivoc: I’m Mark Pivac. I’m the Chief Technical Officer of FastBrick Robotics. We’re developing the world’s first fully automated end-to-end bricklaying robot.

Ron Vanderkley: Why a bricklaying robot?

Mark Pivoc: Well, people have been laying bricks for over 6,000 years and have been trying to automate the process and the intellectual property prior goes back to the times of the Industrial Revolution. The big issues at the moment for automation really are tackling anything that’s dull, dirty or dangerous. Bricklaying actually ticks all three of those. Although it’s not immediately dangerous, over a long period of time there’s a lot of repetitive strain injuries. It’s a job which in Australia is out in the sun, so there’s quite a high skin cancer risk.

Any jobs which are dull and dirty and repetitive, and bricklaying certainly is, it’s very difficult to attract young people to take up the trade. The bricklaying organizations and the builders are having a lot of trouble training people. Even once they do attract a few people to start up the bricklaying apprenticeships, they have over a 50% dropout rate. Really, it’s a supply side driven issue in that there’s not enough bricklayers, particularly in Australia, but we also know that it’s throughout the world. In lower labor cost countries, there may be a pool of labor which is willing to do the bricklaying but they have issues with getting the appropriate skill and the training of those people to become bricklayers.

Although bricklaying is a repetitive task, it’s actually quite a craftsman task because bricklayers don’t just put bricks down. A lot of the other work they do involves setting out the actual size of the house, making sure that it’s the right dimensions and square and that the windows and doors end up in the right place. It’s actually quite a skilled job. Although somebody can be trained up in an few days or weeks to become a productive bricklayer that’s just putting bricks down in a straight line or on a straight wall, it actually takes a full apprenticeship and years of training to really understand the intricacies of building an entire house from scratch from bricks to the modern standards.

Of course, 6,000 years ago people didn’t really care, but these days there’s a whole bunch of different standards and requirements which vary depending on the location of the house, whether it’s in a cyclonic region and needs special roof tie-down straps put in or whether it’s close to a coastal environment and needs different mortar specification or in a bush fire area where the bricklaying needs to meet certain fire requirements or in commercial buildings, again, where you get certain fire requirements.

Of course, the modern requirements for insulation and thermal performance of buildings is meaning that the bricklaying process is becoming more complicated, with bigger bricks, pre-insulated bricks, or insulation added between the bricks. You add up all these various complications, and it really is pretty much a no-brainer that bricks are going to be used into the future of building houses or commercial buildings. It really needs to become an automated process, and that’s why we’ve tackled it.

Ron Vanderkley: Let’s talk about your product. Can get you to describe your robot to our listeners?

Mark Pivoc: Our prototype robot is a 25 ton machine which can reach out 28 meters. The idea of it is you deliver the machine to site, you park it once at the building site, and then the boom is big enough to reach over the entire building site. If you would like to picture how this thing works in reality, the building site will be pre-prepared and the concrete slab laid, and then the bricks delivered to site. We turn up with our machine. A telehandler is used to load the bricks onto the machine. The machine can take three packs of bricks.

There’s automated de-hacking robots which take a row of bricks off each pack and place them onto a shuttle, which then transports the bricks either to a saw or a router or a transfer robot, which then places them onto a chain conveyor where the bricks are gripped by little screw grippers. They are transported then out to a laying head, which is very a dynamic spherical geometry robot that has linear motors and direct drive top motors on it and also some harmonic drive motors. That laying head, the position is monitored in six degrees of freedom. Three linear XYZ coordinates and also the orientation by laser tracker and a smart track sensor.

Those positions are read in a very high speed, some are in the order of 100 Hertz to 1,000 Hertz, into our CNC control system which calculates how much the boom has moved from where it actually should be — that could be due to slight mechanical inaccuracies, wind, or just the dynamic motion of it moving around. We know precisely to fractions of a millimeter and fractions of a degree how that end of the boom is positioned and orientated. We work out where the gripper should be and where it actually is and apply a little compensation there so that the robotic arm of the laying head plucks the bricks off the end of the boom and places them.

Just prior to that, a layer of adhesive is applied to the bricks. We use adhesive for a number of reasons, as opposed to mortar. We have used mortar in the past and we can handle it, but there are a number of issues with mortar. The main one is that it has quite high thermal conductivity relative to the bricks. The modern construction methods are shifting towards a foaming adhesive, which is quite insulating and has less of a thermal bridging effect compared to mortar. We apply just a bed of adhesive, the horizontal joints of the brick.

We don’t do the vertical joints because our initial targeting is for the render construction market. When you apply the render that goes into the vertical joints and it helps the render key on and also provides that bonding on the vertical, what’s called the vertical perp joints. We are in the long run aiming to lay face brick, but we do intend to do that with, again, either a thin bed mortar or an adhesive. The bed will finish on the edges of the bricks to give that jointed effect. We could use mortar, but the modern thermal requirements are moving away from that. Also, there’s quite a big cleanup involved.

The adhesives we use don’t set until they’re exposed to either the oxygen or the moisture in the air. When they’re inside the machine there’s no cleanup problems in the pipes and so on, whereas mortar and thin bed mortars are a cement-based product, which is a hydration reaction. As soon as you mix it with the water they start to set. You’ve got a limited time frame, and then of course you have to clean it out. With up to 16 meters of hose on a machine, that becomes quite an issue in itself. Really, our whole approach has been to look at all the labor and time-consuming processes of either bricklaying or operating a machine and trying to design out those time-consuming and expensive human aspects from the machine.

Ron Vanderkley: On a high level, what type of competition do you have? I’ve heard of robots like the one in the U.S., SAM. What sets your product apart?

Mark Pivoc: Our main difference is that we handle the bricklaying process from end to end. From the moment the bricks are delivered to the building site through to the time of the finished house or structure, we’ve tackled it with an automated approach. Our competitors, like SAM in the U.S., they require manual feeding of the bricks to the robot quite close to where the bricks are being laid. In its current version, SAM can only build one wall at a time. It needs to be taught where the windows and doors are along that wall. It also needs a mason, a trained bricklaying person, to be there to help it out with certain things and do the setup.

Our process is very automated right from the design phase. We take it through a SOLIDWORKS model and we’ve written special API interfaces to that to calculate all the bricks and cuts should happens. Even the generation of our g-code program is quite automated. We handle the bricks automatically from the moment they’re placed on the machine, which of course replacing packs of bricks being done by a forklift or a telehandler. From there right through to the laid brick it’s untouched by human hand.

Ron Vanderkley: When we talked about describing your machine, you mentioned it didn’t have hydraulics. It’s just driven by electric motors and servo.

Mark Pivoc: The entire machine is servo electric. Our first prototype has quite a lot of pneumatic components on it, but our next prototype is moving to be almost entirely electric. The only pneumatics on it really will be used for cleaning and contamination control.

Ron Vanderkley: We talked about the software, and that you’re using a SOLIDWORKS model and you’re creating CNC code directly from that to drive it.

Mark Pivoc: That’s been one of the major parts of our project, has been figuring out a good way to come up with the program for bricklaying. We’ve looked at a lot of different ways of doing it. Although it’s a similar process to 3D printing in that we build up layer by layer, we have to consider that each brick is finite volume and fixed size. Whereas with 3D printing, you’re dealing with something which is a lot more flexible in terms of it’s a much smaller voxel, or 3D pixel if you want to look at it that way. We’re building up a structure out of fairly big voxels, or 3D pixels, which are not cubic. They’re a rectangular prism. We have to deal with that, and we have to manage a lot of complication and issues to do with the way they’re cut.

We can’t just cut off, say, a 10 millimeter thick sliver of brick. How do we handle that throughout the rest of the machine? It’s really, like most computer programming, you can bang out the basic guts of how the program works in maybe a day or two and then you spend the next five years perfecting it to handle all the intricacies and complications. Certainly, that’s been a big part of our process of development, has been dealing with all those intricacies, the cut bricks and corners and getting the bricks to mesh properly and figuring out practical ways or operators to be able to program this thing in a way that is faster than a bricklayer in building the actual house.

Ron Vanderkley: What have been the biggest problems you’ve overcome?

Mark Pivoc: The biggest problem for us has always been, until quite recently, the financial problem. We know that there’s a fantastic market out there. The annual bricklaying market worldwide is just absolutely massive. I got extremely interested in this project when we came up with the concept for an automated bricklaying machine. I really had no idea how many bricks were being used. All I knew what that bricklaying was in the news because there weren’t enough bricklayers available and the price of laying a brick had doubled in 12 months, which is usually an indication that there’s something going on in the market, either a lack of supply or a massive increase in demand.

I jumped on the Bureau of Statistics website and saw that there are over 2 billion bricks a year being made in Australia. I figured that at a dollar a brick to lay them, it was a pretty good market and one worth tackling. We approached the design of the machine without too much concern about how much it would cost because we knew there was a good market at the endpoint. Like all things in automation, as your production numbers go up, the costs come down. We figured that we’d be able to get the costs under control later on. As you can appreciate, it’s a pretty massive automation task that covers a lot of different disciplines and is basically trying to build an entire production line or production plant on something that can be moved from site to site, operated outside in pretty much all sort of practical weather.

From a technical point of view, most engineers know how to deal with those sorts of issues, but it’s getting the funding to sort that out. In Australia for a long time there’s been a reluctance to fund either manufacturing or technology based industries. We’ve had a very busy mining environment. As a consequence, the financial people have been more interested in mining. Unfortunately for Australia as a whole, but fortunately for the tech sector, there’s been a big downturn in mining and commodity prices and the financial capital markets are looking for technology and manufacturing and apps and all those sorts of things. It’s been a case of working at a slow rate for eight years to get the timing right so that it matches up with the financial markets. We’re well funded now, and now we can get in place a lot of technical solutions that just weren’t financially possible before and bring our product to market.

Ron Vanderkley: What comes after this? Currently, you’ve got a prototype. You’re going to be going to production, is it next year?

Mark Pivoc: We have a series of prototypes and production machines coming up. Our first prototype, which we call the 105, is really a technology demonstrator to prove that it’s possible. We know that there’s a number of things with that 105 machine which mean that it’s not likely to be used building houses on site. It would be suitable for some other application, such as a freeway sound wall and things like that, but it’s not ideally suited to the Australian residential market in its current form. We’re pretty well progressed on our next prototype, which we call the 109, and we see that as being the pre-production prototype, which we’ll start building early in 2016. Provided that one passes all of its tests and meets its expectations, then that will become the basis for our production machine.

Ron Vanderkley: Is there anything else planned? We spoke about the possibility of rendering machines. Are there other attachments?

Mark Pivoc: There’s a lot of different possibilities for large scale robotic construction and automation. The key technologies that we’ve developed are what we call course find positioning, so that we can approximately position over a very wide area and then measure very precisely that position and have a fine manipulator, which can then do quite precise work in a much smaller volume. Because it can be moved throughout a big volume, it means we can do tasks which would normally only be possible to automate in a factory with fixed robots. We can now tackle those tasks outside or in a much bigger area. There’s a whole range of different possibilities, ranging from riveting and automated fastening, automated welding, sandblasting, painting, all sorts of kinds of processes, rendering. If you can imagine that there’s a market big enough out there to justify the development costs, then really the world’s your oyster.

Ron Vanderkley: What are the job prospects in robotics in Australia? Are they getting better?

Mark Pivoc: I’m probably the wrong person to talk to. I’m more of a cup is half full guy than cup is half empty. To give you an example, I’ve got three mechatronics graduates who’ve just finished their course. They’re now employed with us here. From our point of view, we think there’s fantastic employment prospects for bright young engineers. Australia as a whole I think has still got a way to go, but it’s pretty widely recognized that there’s not enough engineers around the place. Certainly at the moment in WA there’s probably a few more engineers than the market wants at the moment, but give it a year or two and that’ll probably change.

A lot of that is WA is driven by the resources sector, and that follows through to the manufacturing sector. A lot of Australian manufacturing has disappeared over the last few years because of the high exchange rates, meaning that we’ve become fairly expensive in Australia. With the dollar sitting at about 72 cents as opposed to $1.10, we’ve just become 30% cheaper than we were a couple of years ago. I don’t know too many industries that can become 30% cheaper without doing anything. I think what’s probably a big issue for Australia is that we all understand it’s a global economy and we have to have a floating exchange rate, but floating to the degree that it does in Australia where in my working career I’ve seen the dollar as low as 52 cents and as high as $1.10.

Any long-term planning for industries which have to have presence for 20 or 30 years to become efficient, car manufacturing, there’s such a massive amount of infrastructure, training and technology and associated first, second, third, fourth tier suppliers that feed that production line. You can’t just set up a car industry or a robotics automation industry overnight. It takes years to develop the technology and the expertise to do that. Probably in Australia one of our biggest issues is the fluctuating exchange rate.

We’re going through the same thing now looking at where we’re going to produce these machines. The biggest factor of uncertainty in our production planning is what’s our exchange rate going to be in three years’ time? Nobody can crystal ball gaze that. Is it going to be 75 cents? Is it going to be $1.10? Is it going to be 52 cents? Is it going to be $2.00? Who knows? Which makes it extremely difficult to plan. Yeah, we’ve got some issues in Australia that we have to figure out some way to deal with.

Ron Vanderkley: On behalf of the podcast, I’d like to thank you, Mark, for your time. We’d like to keep in touch and see where your product is going in the future. Thank you.

Mark Pivoc: Thanks, Ron. It’s been a pleasure.

Talking Machines: Software and statistics for machine learning, with Rob Tibshirani

Talking Machines — Sun, 27 Mar 2016 08:56:00 +0000

In episode six of season two, we talk about how to build software for machine learning (and what the roadblocks are), we take a listener question about how to start exploring a new dataset, plus, we talk with Rob Tibshirani of Stanford University.

If you like this podcast, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

ep.204: Satellite Assembly in Space, with John Lymer

Andrew Vaziri — Sat, 19 Mar 2016 17:54:09 +0000

In this episode, Andrew Vaziri speaks with John Lymer, Chief Architect of Robotics and Automation at SSL. They highlight key programs in space robotics from the 1980s through to SSL’s current program to robotically assemble satellites in space.

John Lymer
John Lymer is the Chief Architect of Robotics and Automation at SSL, a company known for its commercial communications satellites and spacecraft systems. He is an expert in automation solutions for space, terrestrial, and underwater operations, and has experience in a broad range of robotics missions including microsurgical and nuclear restoration systems.

Since joining SSL in 2013, John has been named Chief Architect for a NASA Tipping Point project called Dragonfly, which uses robotics for satellite assembly in space. He is also involved in NASA’s Asteroid Redirect Mission (ARM).

Links:

ep.203: Agilic and PiBot, with Harry Gee

Abate De Mey — Mon, 14 Mar 2016 09:00:33 +0000

Pibot, robotic kit for the Raspberry Pi. Source: PiBot.org

In this episode, Abate De Mey interviews Harry Gee, founder of the robotics startup Agilic. Harry discusses his company and the robots he designed using the Raspberry Pi and Pi zero. He then discusses the basics for creating a robotics start-up and provides advice for aspiring entrepreneurs.

Harry Gee
Harry Gee has formal training in architecture and carpentry, and is a self-taught programmer, web designer, and roboticist. As technology continues to grow, there is an increased demand for tech-related knowledge for the younger generation. To accommodate this demand, Harry Gee focuses on developing educational robotic kits and workshops that empower young people with vital tech and hands-on skills. He believes robotics is an amazing playground for creative solutions to social and global challenges.

Links:

Talking Machines: AI safety and the legacy of Bletchley Park, with Nick Patterson

Talking Machines — Mon, 29 Feb 2016 20:12:35 +0000

In episode four of season two, we talk about some of the major issues in AI safety, (and how they’re not really that different from the questions we ask whenever we create a new tool.) We take a listener question about time series, and we talk with Nick Patterson of the Broad Institute about everything from ancient DNA to Alan Turing.

If you like this podcast, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

ep.202: Automation and Employment, with Michael Osborne

Andrew Vaziri — Sat, 20 Feb 2016 08:01:21 +0000

In this episode Andrew Vaziri speaks with Mike Osborne, Associate Professor in Machine Learning at the University of Oxford. They discuss how advancements in artificial intelligence may change the workforce, the economy and society based on findings from their study on the future of employment.Beyond the big headlines about automation and jobs, Mike discusses how the study was done, the emergence of new jobs, the need for improved education, and ideas for how we might reshape society to spread wealth that results from advances in technology.

Michael Osborne

Michael Osborne is the Dyson Associate Professor in Machine Learning, Official Fellow of Exeter College and Faculty Member of the Oxford-Man Institute of Quantitative Finance, all at the University of Oxford. He co-leads theMachine Learning Research Group, a sub-group of the Robotics Research Group in the Department of Engineering Science.

Links:

ep.201: The Airbus Shopfloor Challenge, with Curtis Carson

Andrew Vaziri — Sat, 06 Feb 2016 12:39:11 +0000

Airbus assembly floor. Source: Airbus Group

Transcript below

In this episode, Andrew Vaziri speaks with Curtis Carson, Head of Research and Technology in Industrial Strategy and Systems at Airbus. They speak about the Airbus Shopfloor Challenge to be held at ICRA 2016, and discuss the need for a new generation of industrial robots for aircraft manufacturing.

More information can be found on the competition website.

Curtis Carson

Curtis Carson is the Head of Research and Technology in the Industrial Strategy and Systems Centre of Competence at Airbus, a leading aircraft manufacturer headquartered in Toulouse, France. Holder of an engineering degree from Ryerson Polytechnic University in Canada, Curtis has extensive international experience in both the aerospace and automotive industries. Curtis’ current role includes developing new and innovative aircraft production processes through the application of emerging technologies. This role spans a host of technologies in the framework of the future factory, from augmented reality to collaborative robotics to digitalization to name a few examples.

Links:

Transcript

This transcript has been edited for clarity.

Andrew: Hello, welcome to Robots Podcast! Can you introduce yourself and describe what you do?

Curtis: I’m Curtis Carson and I’m responsible for the research and technology on the industrial system side with Airbus in France. My role encompasses everything you can imagine around the production of a commercial aircraft from jigs machinery equipment to robotics automation to new innovative techniques, enhancing and optimizing through IOT and connected objects.

Andrew: What is IOT?

Curtis: Internet of things, so connectedness and bridging multiple sensors and systems around our worker or automation on our shop floor.

Andrew: For people who aren’t familiar with Airbus, could you describe your products?

Curtis: There’s a high likelihood if you’ve travelled anywhere, it was one of our Airbus products. We develop commercial aircrafts from a smaller range, which we call single aisle aircraft A320, to the world’s largest passenger aircraft, the A380.

Andrew: What are the challenges in manufacturing aircraft?

Curtis: It might be hard to imagine from an outside perspective, but some of crafts have been developed years ago and we’ve been producing them for many, many years. Our new and innovative products, like our A350, have just gone to our first customers and the industrial system that we built behind to produce all this is quite large. But also expansive in terms of footprint and age and the diversity, is quite significant. We’re not building a factory every year and we’re not designing new aircraft each year. This makes it challenging to find ways to innovate.

Andrew: You said that the lifetime can be significant. What is the average time one aircraft design might be in production?

Curtis: It is quite typical for an aircraft design to be in production for 20, 30 years or more. The time frames are large to very long, the more successful the program the longer the production.

Andrew: How many aircraft does Airbus produce in a year?

Curtis: It was over 630+ aircraft produced last year. This might seem like a small number versus an automotive domain – where you’re producing thousands and thousands of cars – but this is some of the largest quantities of commercial aircrafts ever produced in a given period, or a given year.

Andrew: How many aircraft are currently in the backlog to be purchased from Airbus?

Curtis: I think it’s close to 7,000 aircrafts in backlog for production time-frame, don’t quote me on that. If you take the numbers per year, the backlog is over quite a number of years.

Andrew: To address the large number of aircraft currently in the backlog, what are some initiatives Airbus is undertaking for its manufacturing processes?

Curtis: We’re looking to innovate assembly process in the building of aircraft that are part of that building process and it might be difficult to see this without looking at really what’s inside a production system. As well as looking into automation to help people, or robotics and automation that we can use to assemble and fabricate parts of the aircraft.

We’re looking from across different facets of innovation: from smart tools, that know where they are in the aircraft; to autonomous robotics systems, that can manage and move themselves around in our production environment; to innovative technologies, like 3D printing and what it can bring to production efficiency and performance in the building of our aircraft. We’re looking across a large spectrum of technologies and see how we can effectively boost our productivity and our performance in building our aircraft.

Andrew: What robotics initiatives has Airbus undertaken in the past?

Curtis: In the area of robotics we’re using industrial robots and systems you may see if you were in an automotive environment for specific tasks – like drilling holes on a panel or a skin structure to assemble frames on the aircraft – we have big machines that we use today for building large scale composite parts up to the size of wings. We have a number of different automated systems we use. These are the typical industrial automations that we’re applying. With those typical industrial automations we can target a small percentage of the production activities to the large quantity of parts you can have in an aircraft, and the large diversity of the types of parts.

That’s what exists today. What we’re moving towards is where we can have more light-weight robotic systems that are flexible and agile. Systems that can move in and around our structures versus designing from the ground up a fixed and rigid system… instead you have a robot from within. Once you realize the size and the scale of the aircraft, you’ll see it’s not something that’s readily set up to bring the product to the robot or for the robot to do the work, and then move the product along. We have to be more agile and flexible in our building process.

Andrew: Airbus is sponsoring this year’s challenge at the International Conference on Robotics and Automation. Can you tell us about that?

Curtis: The competition we are proposing this year is drilling. Why would we target something like drilling? The simple answer is by looking at the production in Airbus in assembling the aircraft I mentioned earlier. We’re drilling over 120 million holes per year. Of those 120 million holes I would say we’re only able to automate a small percentage in terms of production tasks. There’s a high level of manual drilling we still do in the production of our aircraft today. Drilling is an interesting aspect. If we can combine what I said earlier – a lightweight and mobile robotic system – and combine with lightweight tools, then we have an opportunity to move into a new arena, and potential. We could implement an automated system without changing our infrastructure or our production lines to accommodate such a system.

A lot of these holes are drilled today by people. It’s not that a large industrial robot wouldn’t work doing that job, but it is not very mobile. And if you make the robot mobile, well it’s still heavy! If you’re drilling holes inside an aircraft structure barrel, it becomes difficult to put the robot inside without rebuilding your industrial infrastructure. This is what we’re interested to see: how can we push the boundaries of robotics capabilities today in this lightweight mobile module and solve these activities? Drilling is just the start. If we can do it with drilling, we can look to other aspects in production operations.

Andrew: In the competition itself, what are you asking teams to do?

Curtis: What we setup might look very simple, but the technical complexities come if we have a metal template. We’re asking them to produce a specific drilling pattern. In this drilling pattern we are asking to produce an automated system with bounding frame or restrictions with regards to size, mass, points for modularity. Then, looking at how quickly they can manipulate the system towards the task. If I have a pattern of holes to drill, how fast can they drill? What’s the level of quality? We’re very, very interested not only to have high speed, but quality.

We need precision. If we adjust the environment and the positions of these templates how can they master their robotic system to accommodate these different positions? How would they simulate the complexities of the geometry and the environment that we’re drilling on? I think for us and during the activities, we’re looking for a robotics system not doing just one task in a repeated mode, over and over, in one position. The types of automation we need, especially when it comes to drilling, are having high volume of repeated drilling tasks over a wide range of different positions.

Andrew: I understand that the robots are to be modular in their end effectors and weigh less than 100kg and you’ve mentioned reasons for both. I did want to ask: why are you asking people to provide open source software to actually control these robots?

Curtis: I would say the simple approach is there in the production context we have today and you see it in a number of industries. With this approach we can look at getting the best of the best. You mentioned end effector. Instead of thinking about a closed complete system that we would go buy and maybe the company we’re buying from is advanced in the modular robotic arm system. However, they don’t develop end effectors. The standard approach in most industries would mean we go buy from a supplier and package. It’s not that it doesn’t work, it can work. But what happens when better end effectors comes along? Or we have another company that’s able to do end effectors better than the company that is integrating the robot and end effectors of the 1^st system?

Well, it becomes difficult and complicated to figure out how to benefit from such an innovative change on the current system. We see the same thing in terms of the software, the control, the algorithms, whatever goes inside. Imagine also we have a huge IT infrastructure on our side across many different aspects of the production lifecycle and this is another level of complexity when it’s closed. We’re then in a mode where we develop each time we have a new system or solution we have to develop a very specific interface for such a solution.

There are a number of different reasons, but I would say 1^st and foremost we want to capture the best capability from the best in the world, wherever they are, for what they’re good at. It’s not common to find somebody that’s good at every single aspect being on all those aspects.

Andrew: It sounds like a large part of this is foreign partnerships. Are you looking more so towards academia or industry, of a mix of both?

Curtis: I would say if we’re really good at it, it will be both. Sometimes you can discover innovative things coming out of industry but you can discover interesting things coming out of academia. I don’t think that’s by surprise; academia and industry are linked in a number of different ways and means. A lot of the new businesses you see in today’s start-up world are from academia. What we’re looking for out there is how can we help stimulate towards academia and/or the outside world?

Where are our challenges, what are we looking to resolve and help that makes it feasible? This is also a good reason for the open approach. The more we are open in accessibility to the outside world, whether academia or businesses, the more opportunities and people that could potentially propose ideas that, perhaps, we didn’t think of in resolving. It creates a level playing field for small start-ups to be just as powerful and effective at proposing something valuable versus the big business and industry we’re used to working with today. It creates opportunity.

Andrew: Beyond just the drilling task that’s been selected for this competition, what are some places where you might see innovation being needed? Or a new approach beyond just making something that’s practical and useful in an industrial setting, but a whole new capability entirely?

Curtis: I guess it depends on what you mean by a whole new capability. What we look for today is predominantly driven by what we’ve already designed and producing. The aspect that we’re not deeply looking from this type of competition is okay. We’re talking about optimizing the current state of our product based upon how it designed. The design may have been a number of years ago, or 5 years ago. How do you combine those new capabilities and products, materials, or techniques? You use the capabilities in the design of the product that you’re doing and optimize both at the same time.

Andrew: As you think about your current production process, what are some examples of other low hanging fruit processes that could be adapted to use these technologies we’ve discussed?

Curtis: The minute we talk about drilling, the next thing is fastening. We have millions of fasteners we’re replacing. We do a lot of surface treatments or ceilings or applying ceilings joints. Painting requires a high demand in terms of manual effort and activity. If you’re painting, then have sanding and abrasion processes. Due to the size and the structure there are confined spaces and restricted access spaces. There are a number of non-ergonomic conditions the workers are in today. This is the whole domain and area of potential when we talk about robotics, especially light weight or new generations of robotics where you see lightweight in modular and adaptable types of systems. We have a huge playing field in terms of opportunity.

Andrew: How would you see the robots and the people interacting as they assemble an aircraft?

Curtis: The approach we focus on is that we don’t believe, at least today, in having a factory full of robots, and that’s it. We’re looking at worker-robot collaboration. We start from our worker, the person doing the work. How can we enable him to be more effective, better, efficient in his day to day job? It’s like a partner. Any difficult and non-human friendly tasks we can pass to the robot. We can ease the non-ergonomic conditions and reduce the repetitive and straining activities from workers. It’s a combination and collaboration.

We know that as robotics systems evolve they’ll be able to do a little bit more. At the same time, like what I mentioned at the beginning, it’s not just about a robot with a worker, it’s also how we can use the new connectivity of internet of things and connected objects and advanced communication to enable the worker to be more efficient and have the right information in a more dynamic way. Whether it’s through Google Glasses or wearable computing, or sensors in the manual drillers showing the quality of the hole they drilled. These types of automated systems allow a new feedback process for the operator to do the job the best he can.

Andrew: Could you give us a story or an illustration of what you imagine a worker in this future factory might to on a daily basis that would utilize these technologies, like, internet of things and the general ability to have more information?

Curtis: I would say the easiest way is just to give you a glimpse of what we’re preparing today. We have tests in preparation for one of our production environments where we will have electronically controlled smart tools. For example, a torqueing wrench for finishing the fastener or a manual drilling pistol for making the hole, with wearable devices. We will equip to three stations or more in one of our production lines and allow them to experience what tomorrow could look like.

The reason I like to focus on the aspect of tomorrow is because it’s not 5 years from now. We have the potential to incrementally put something in place today. What we’re preparing and doing now is the first production trials to see how it would be implemented in day to day for business for our workers.

Andrew: What do you see as the timeline? If tomorrow is the system you’ve just described what is 5 years from now, 10 years, or 30 years?

Curtis: It’s difficult for me to project that far ahead. But five years from I don’t think it’s too far-fetched to imagine that connected objects and types of automation will be a prevalent, common thing in any of our production facilities. It could be that tools you might see today like working from paper or drawings, you won’t see. I imagine you’d see a much cleaner environment. You’d see what we consider a high-tech environment in terms of tools, equipment, digital displays, progress tracking interfaces they’re using to prepare job flow of parts to the job that they’re doing.

I would imagine it’s hard to describe it in words unless you visualize and see what a production facility looks like today. I would imagine the proliferation of such devices is not on a linear scale. You would see it more in an expediential implementation and growth. This is one of the good reasons to consider the open approach. We must have a way, if we want to be fast. It has to be designed in a way that can allow this evolution to happen easily and quickly for robotics systems or connected objects and digital thread that we could use with our workers.

Andrew: In order to keep pace with that expediential rate of improvement, how serious is Airbus in investing in these opportunities? What would you guess would be the activity in the next few years?

Curtis: We’re serious about it, and the challenge is just a glimpse into why we’re motivated in this domain. The challenges are real, it’s a real challenge. It’s a real problem that we face today. It’s not something just made up to make a contest for the ICRA event. It’s a real potential application for our shuffler. When it comes to the digitization and the production efficiency and connectivity, we’re very serious about it and we have a number of initiatives running.

Andrew: On an economic scale people could also be interested, like how much has been spent or planned to be spent. Professors or companies look to the future and will think about possibly partnering with Airbus.

Curtis: I don’t know if I can put a figure to it, but I would say the best way to see how committed we are to it is to open up a dialogue; ask us! As us in your domain area! It really depends on the value we can potentially get, or bring from it.

We have significant portfolios of research and development activity running on platforms and cooperation with universities, technology centers, businesses. There is a significant amount of activity happening. I can’t really quote to the figures to be honest.

Andrew: ICRA is the 15 – 21 May in Stockholm, Sweden. When will you be accepting submissions to be part of this competition?

Curtis: Our submission window for ICRA is open today through March 15^th when we’ll close the submission process.

Andrew: For these teams, what kind of support is available from the competition?

Curtis: We’re offering, in certain circumstances, the possibility to any registered team to apply for help with travel, equipment, materials or tools. It depends on the potential and ideas by the teams. We judge it on the case-by-case basis with the teams who register. There is potential for support, but it requires registering and then talking with us.

Andrew: Where can people find registration forms and more information about the event?

Curtis: They can go directly to the ICRA 2016 homepage to find the link to the Airbus group webpage. The webpage provides details and process for registering, a general description about the contest, prizes and scoring. The best place is directly on the ICRA webpage.

Andrew: Thank you for joining us and putting together the Airbus Shop Floor Challenge! We look forward to seeing the results at this year’s ICRA!

Curtis: Thank you for having me! I’m excited to see the final results from our competitors.

Turning the lens on robotics reporting: An interview with New York Times technology reporter John Markoff

Hallie Siegel — Thu, 28 Jan 2016 19:36:59 +0000

John Markoff filing story from airport. Image credit: Joi via Wikimedia Commons

If you are working in the field of robotics or AI, you may be used to fielding calls from journalists wanting to better understand the technology. But not all journalists are created equal. Pulitzer-Prize-winning John Markoff (@markoff) has been covering the technology beat at the New York Times for almost three decades, and recently published Machines of Loving Grace – a book that chronicles the evolution of computer science, robotics and AI. In this interview we turn the lens around and ask Markoff about what motivates his interest to report on robotics, and how he sees trends in robotics today being informed by people and events from the past.

This interview has been edited for length and clarity.

[HS] Can you tell us about your early days writing about the fields of AI and robotics?

[JM] I began writing about Silicon Valley in 1977 and spent most of my time writing about personal computing and the Internet. In the ‘80s there was a lot of controversy around the Strategic Defense Initiative (SDI) and many computer scientists and AI researchers were opposed to it – it was the first big AI issue in the news. Then in 2004 there was the DARPA autonomous vehicle challenge, and I began covering those events.

Engelbart’s prototype of a computer mouse, as designed by Bill English from Engelbart’s sketches. Image: Wikipedia via Macworld

It wasn’t until a bit later that I really made AI and Robotics my first interest, though. I’d covered computer security for the New York Times forever, but I began feeling that I’d written the same story just too many times. I needed to do something different, and it seemed like AI and robotics were making more progress.

I’ve always loved the social history of the Valley, and trying to understand how things are connected there. Often technologies will bubble around for decades before they actually gain broad consumer acceptance. Look at the mouse, for example. Engelbart invented it in ’64 but it really didn’t become a mass consumer product until Microsoft Windows and Mac adopted it 20 years later. I’m interested in what causes things to gain acceptance even though there were people who believed in them for years beforehand.

There are other examples … In 2010, the historian David Brock came across a 1951 memo written by William Shockley describing an “Automatic Trainable Robot”. Shockley had always been known for his work on transistors, but what the memo showed was that he had really set out to build a robotics company. To discover that early ideas in robotics were at the core of creating Silicon Valley was a bit of a revelation that nobody had really noticed until then.

Shockley’s 1951 memo to president of Bell Labs Mervin Kelly described an “automatic trainable robot” that sounds eerily similar to trainable products on the market today, such as Rethink’s Baxter. For more details, see the IEEE Spectrum article by David Brock How William Shockley’s Robot Dream Helped Launch Silicon Valley.

Your book describes a funny anecdote around that revelation … when Rod Brooks shared that memo with his employees and asked them to guess when it had been written…

Rod gave the memo to his employees, and nobody could figure out the timing of it, because if you just read the memo and don’t think about the date, it reads like a product description of Baxter. So here we have an idea for a “trainable robot” from 1951 … and 65 years later it looks like the timing is finally right.

Machines of Loving Grace starts out describing the tension between two researchers doing early work in computer science – John McCarthy and Douglas Engelbart – whose work eventually led to the creation of two philosophical “camps” in AI research. What was that tension about?

In the ‘60s McCarthy was involved in setting up the Stanford AI Laboratory, and Engelbart set up the Augmentation Research Center at SRI. They were each were about a mile from Stanford, but they had very different approaches. Whereas McCarthy had set out to build a working “artificial intelligence” that could simulate human capabilities (he thought this could be done within a decade), Engelbart was going in the opposite direction – he wanted to use technology to augment intellectual workers.

I came to realize that these two approaches – artificial intelligence (AI) and intelligence augmentation (IA) – went in two philosophically different directions … and that two distinct research communities emerged from those early laboratories – communities that don’t really speak to each other.

The paradox we see now is that the same technologies that extend our abilities can also displace us – and that was the puzzle I wanted to solve in my book.

There were several key figures in your book – Norbert Wiener and McCarthy come to mind – who really broke over differences in their philosophical approach to AI … Can you describe what was at the root of their polarization?

Read Frank Tobe’s review: New book by John Markoff explores common ground between humans and robots

Wiener was an interesting character – an outlier in his day, and rather eccentric.

In 1948 Wiener wrote ‘Cybernetics’, a book that outlined his approach to control and communication engineering. That work led him to believe that it was possible that humans could one day be designed out of the future, and so two years later he wrote a second more philosophical book called the ‘Human Use of Human Beings‘ about the AI/IA dichotomy, where he attempted to lay out the choice we have in terms of how we use technology.

At that point, technological unemployment was a new concern. Industrial automation had already been happening for decades, but Wiener was the first one to really understand the potential of software-controlled machines and he brought that alarm to the US labour movement. He actually wrote letters to many labour leaders to warn them that the technology was going to have an impact on the workforce, saying “This may end up destroying much of your union membership’s jobs.”

We know Wiener was a very controversial character within the engineering and academic communities; he had lots of enemies and his ideals were never mainstream. He was seen as a radical, going all the way back to his opposition to the technical community designing weapons in the wake of World War II.

On the other hand, McCarthy was very clear about saying that he deliberately coined the term “artificial intelligence” in order to set his ideas apart from Wiener. He felt that Cybernetics leaned too far towards the analogue. On a personal level, McCarthy really disliked Wiener, too; he thought that Wiener was bombastic and a bore, and he wanted nothing to do with him.

So there was this tension right at the root of artificial intelligence, and it has all kinds of significance for the field.

For the most part your book chronicles events that take place in the US. Was the same tension between AI and IA happening in other places, too?

That’s an interesting question. Cybernetics was more appealing to the Europeans, and there are Departments of Cybernetics in universities across Europe to this day. I am fascinated by that, even though I really did focus on the US in my book.

Sir James Lighthill. Image via Wikipedia. Watch the BBC Lighthouse Debates on YouTube.

AI actually had its first big failure not in the US but in England as a result of Lighthill Report, which assessed the claims made by the AI researchers in the early 1970s and then cut off government funding for their research, causing what we now call the “AI Winter”. That collapse of funding had a big impact on many people’s careers. Geoffrey Hinton for example is now a well-known Google researcher, and is one of the key figures in development of advance neural net technologies, but early in his career he really was an outcast as a result of that report.

Then there was Joseph Weizenbaum, who had a similar outlook to Wiener. He ran the famous ELIZA experiment at MIT and was aghast that a simple text bot was able to convince his students that they were having real conversations with it. In his book ‘Computer Power and Human Reason’ he was very critical of AI research on a philosophical and ethical level for ignoring human values. He left the US disgusted, and ended up being much more widely accepted in Europe.

What about Japan? Fast forwarding to the present, your book mentions Toyota’s current corporate philosophy of kaizen (or “good change”) as a shift that suggests there are limits to automation. Can you describe that philosophy and how it’s influenced Toyota’s policies?

While Toyota had at one time been pushing towards fully automated assembly lines, more recently they have begun to systematically put humans back into their manufacturing processes. They have come to see that if you fully automate a production line, it might be extremely reliable, but it’s frozen … an automated assembly line doesn’t really improve.

But skilled workers who are on the production line are intimately familiar with the manufacturing process, and can suggest ways to improve efficiency and productivity. So Toyota is rebalancing automation with craft in order to get a more cooperative relationship between humans and robots.

In September 2015 Toyota announced a substantial investment in robotics and AI research to develop “advanced driving support” technology, with former Program Manager of DARPA’s DRC Gill Pratt directing the overall project as Executive Technical Advisor. Toyota will allocate USD$50M over the next five years in a partnership with MIT’s CSAIL (headed by Daniela Rus) and Stanford’s SAIL (headed by Fei-Fei Li) to develop research facilities in Stanford and Cambridge. Photo: Kathryn Rapier

What’s interesting is that just this past fall Toyota invested in the AI departments of both MIT and Stanford to do research on an intelligent car that they clearly describe as being on the IA side of the AI-IA debate. They are not looking to build pure self-driving cars, but instead want to give you a kind of guardian angel that will watch over you as you drive. They really want to design humans “in” rather than design them “out”.

I spend a lot time with the AI community in my research, and I think half of the field really feels strongly that you should use AI technologies in an IA format, keeping humans in the loop. People like Adam Cheyer, Eric Horvitz and Tom Gruber are all well-known AI researchers who see Engelbart as their hero. Horvitz in particular feels strongly about approaching things from that direction.

In 2009 after the AAAI Meeting on Long Term AI Futures you wrote an article for the New York Times with the headline ‘Scientists Worry Machines May Outsmart Man‘. More recently, covering the DARPA challenge, your headline was ‘Relax, The Terminator is Far Away’. Has your perspective on robotics and AI changed in that time period?

The AAAI Alisomar meeting was really motivated by Eric Horwitz in response to WIRED article the founder of Sun Microsystems Bill Joy wrote in 2000 about Why the Future Doesn’t Need Us, where he raised concerns about AI and robotics as threats to human survival.

Horvitz felt that he needed to respond to Joy’s concerns, so in 2009 he called together a group of the best AI researchers in the country to explore the issue of how fast AI technologies were developing and whether they indeed posed a serious threat to humanity. That was where the first headline came from, and I was really a messenger in that case.

Funny enough, the final report from the meeting was somewhat anti-climactic: they basically came to a consensus of that the kind of scenario Joy had sketched out was not an immediate worry.

Now fast-forward to last year, when my article about the DAPRA Robotic Challenge came out. This was toward the end of the renewed AI debate that had just been touched off by Stephen Hawking, Elon Musk, Stuart Russell, Nick Bostrum, and others at the Future of Life Institute who were arguing that, based on half a decade of success in the neural network community, they foresee an exponential increase in the capability of machines coming in the near future.

But many computer scientists think that the possibility of self-aware thinking machines is vastly overstated, and my experience watching the DARPA Robotics Challenge supported that view. You just had to look at the falling robots, and the fact most of them couldn’t even open a door. That’s the state of the art.

I guess it’s still an open debate, but as for my own feelings, I’m sceptical about some of the claims being made about AI.

Douglas Hofstadter was one of the philosophers who originally called out the AI researchers in 1960s and 1970s for over-promising and under-delivering. He argued that the assertions about AI programs and their progress is a little bit like someone who climbs to the top of the tree, saying he is making steady progress on the way to the moon. But at some point the tree stops. I feel like that statement still holds.

I thought it funny when Elon Musk was quoted last year as saying “We will be like a pet Labrador if we’re lucky” and how close that was to Marvin Minsky’s famous in the 1970s saying “If we are lucky they will treat us like pets.” But you seem to be saying that these kinds of statements aren’t just attention-grabbing or click bait – they are also over-promising what AI can do, albeit in a negative way …

Elon has this wonderful ability to say evocative things like “we are summoning the demon”.

Some people have criticized the media for their reporting of the Future of Life Institute’s activities, but I felt that the Institute may have brought this on themselves to some degree. They talk about the “existential risks” of AI, so it’s no surprise that some of the media picked up on this and jumped immediately to The Terminator. Afterall, what is The Terminator if it’s not an existential threat?

I don’t think that existential threat is a bad motif for people’s concerns, but I also think this is a field that routinely over-promises. It’s true that we have seen significant progress in terms of machines that can see, listen and understand natural language. But cognition reasoning systems are still making baby steps.

That said, I’m supportive of the efforts of Elon Musk and his colleagues at the Future of Life Institute because I do believe that machine autonomy is increasingly becoming a reality in everything from the workplace to weapons, and it’s worth talking about these issues whether autonomous machines are self-aware or not.

As a journalist following the field, you’ve had a unique opportunity to watch a whole new crop of roboticists and AI experts mature. Is this next generation continuing to fuel the polarization between AI and IA, or are the two camps starting to merge?

It’s hard to predict what will become of this current debate. Looking at the issue of technological unemployment in particular, I’ve come to feel that the actual impact of automation technologies on the workforce is a much murkier question than either the press or the science community seems to believe.

Once you start reading the economists, you realize there is a multiplicity of issues going on, including things like tax policy and changing demography, and picking it apart is very difficult. It’s a huge debate, and I’ve come away feeling I know less than when I started.

But I’ve spent a lot of time talking to people working in this field – mostly around Stanford and Berkeley – and it makes me hopeful because there is a generation of young scientists and engineers who are actually exposed to some of the issues we’ve been talking about and are really thinking about them. Ultimately, these machines are being built by humans, so it comes down to individuals having to make decisions about how to design their machines, and their values become really important.

Thank you! It was such a pleasure to read your book and to have a chance to talk to you today.

It was great talking to you – I enjoyed it.

Machines of Loving Grace was released in August 2015, and is available in hardcover, softcover and e-book formats.

If you liked this post, you may also enjoy:

ep.200: 200th Episode Special, with Rodney Brooks

Podcast team — Fri, 22 Jan 2016 22:12:21 +0000

Transcript below.

In this episode, we celebrate our 200th episode! That’s over 6000 minutes of robot goodness and nearly 8 years releasing interviews with your favorite roboticists. The podcast is all volunteer run, a special thanks to everyone on the team who’s made this possible! And thanks to all of you for listening in all these years.

To celebrate, our president Audrow Nash has invited a team of old-timers from the podcast team, and one of our favorite recurring guests, Rodney Brooks from Rethink Robotics.

The panel is composed of Sabine Hauert, Markus Waibel, and Peter Dürr, all co-founders of the podcast, and Per Sjöborg who’s been part of the team for many years now. They discuss their favorite episodes, what area in robotics is most interesting, their best memories, and lessons learned.

Rodney Brooks tells us about latest developments at Rethink Robotics, including their new robot Sawyer, and the Baxter Research Robot. He also tells us about the importance of science communication in a field that is prone to hype and disinformation.

Rodney Brooks
A mathematics undergraduate in his native Australia, Rodney received a Ph.D. in Computer Science from Stanford in 1981. From 1984 to 2010, he was on the MIT faculty, and completed his service as a Professor of Robotics. He was also the founding Director of the Institute’s Computer Science and Artificial Intelligence Laboratory, and served in that role until 2007. In 1990, he co-founded iRobot (NASDAQ: IRBT), where he served variously as CTO, Chairman and board member until 2011. Rodney has been honored by election to the National Academy of Engineering, and has been elected as a Fellow of the American Academy of Arts and Sciences, the Association of Computing Machinery, the Association for the Advancement of Artificial Intelligence, the Institute of Electrical and Electronics Engineers and the American Association for the Advancement of Science. He is the now the founder, chairman, and CTO of Rethink Robotics.

Links:

Transcript

This transcript has been edited for clarity.

Part One: Panel discussion with the Robots Podcast Team

Audrow: Our first question to the panel is: “What have been your favorite robots podcast episodes?”

Sabine: That was a really tough question. There have been so many episodes over the years – 200 of them – so it is pretty hard to choose. Dario Floreano was one of them; he’s one of the reasons the Robots Podcast exists and he started this all out of EPFL. I also really liked interviewing Bob Full about bio-inspired robots. I also interviewed Rodney Brooks a couple of times, including at the offices of Rethink Robotics. That was fun because I got to tour Rethink Robotics and have a chat with Rod over in his office.

Then there were things that were really outside the scope of what I was used to discussing or studying, like the interview with Ryan Calo, which was about the legal aspects of robotics. That’s something I hadn’t really thought about before, and talking to Ryan back then was quite helpful for me in terms of framing that discussion.

And also Radhika Nagpal – just because she’s in an area that I love and she’s a great researcher.

Audrow: What about you, Peter?

Peter: I’m with Sabine: it’s been really difficult to choose. As the audio editor for the podcast, I get to listen to all the episodes before anyone else and I’m always looking forward to each and every new episode. If you had to pin me down on one, I’m going to go for #166: Quest for Computer Vision, with Peter Corke. I work with computer vision in my day job and I really liked Peter Corke’s insightful interview and his overview over this super interesting field.

Sabine: That interview was one you did, Audrow!

Audrow: Yes, it is – thank you! … Per, what has been your favorite episode?

Per: I like the episode from the precursor to the Robots Podcast – Talking Robots – with Robin Murphy from Texas A&M, about rescue robotics. Designing robots that actually work and do the task they’re assigned to in a very hard environment – that was great content.

The favorite interview I did had to be the with Mark Tilden in Hong Kong in his amazing apartment. There were probably – I kid you not – 5000 robots there … everything from the Wowwee robots to his BEAM Robotics stuff. It was just amazing to hear his enormous experience and see all those robots, so that was the favorite one to record for me.

But as everyone else has said, there are so many good interviews and I’ve been fortunate enough to meet so many great people. It’s really impossible to choose.

Audrow: What about you, Markus?

Markus: I’ve not been able to pick a single episode either, but I have really enjoyed the business related episodes. I guess that’s how my personal interests have evolved over the years. When we started back in 2006 with Talking Robots, our first 40 or 45 interviews were almost entirely with academics. That was about 10 years ago, and that was what was out there, mostly. But if you look over the interviews we’ve done in the past year, it turns out to be about half industry and half academia, and I think that’s an interesting development for the field.

Sabine: What’s your favorite, Audrow?

Audrow: Like everyone has been saying, it’s really hard to pick. But I think one of my favorites was # 98 on self-organizing systems with Radhika. That episode gave a great overview of the field and it was interesting listening to her describe it. Also #101 on the history and outlook of robotics with George Bekey and Rodney Brooks, because it gave a big picture of what they had seen in field. It was interesting having the interviewer pick into their wisdom, to give me a better perspective on robotics. Those have been my favorite interviews so far.

My next question for everyone is what area of robotics do you think is the most interesting, and why?

Markus: I work with drones every day – small flying machines – so obviously I have a huge interest there.

Apart from that, one area I’ve been really very excited about from the very start is the field of exoskeletons and prosthetics. I think there’s so much potential to increase our independence as we age with these technologies, and as a hobby, I closely follow how these technologies are developing and being brought to market.

Another one is speech recognition. It’s really more AI [than robotics], but I think speech recognition will completely revolutionize the way we interact with technology. Right now data is mostly entered by keyboard or by mouse, and we can’t really communicate with machines in a natural way. I think speak recognition will revolutionize the way we interact with technology across the board if we manage to solve the privacy issues associated with it – which will not be easy.

Of course the next step is then doing something with that speech. I think the prevalent solution right now is to do some type of cloud processing. But with robots like Jibo or Amazon’s Echo, there’s always the big question of whether I really want Amazon or whoever else to listen in on all the conversations I’m having in my house or in my car or in my office. I think it will be difficult to role this kind of technology out in a major way before the privacy issue is resolved.

Audrow: What about you, Per?

Per: Everybody who knows me knows that I love modular robotics and self-reconfiguring modular robotics. There are so many modular systems coming out now (even if they are not self-reconfiguring) – practical systems that are available commercially … off the shelf … and that’s really exciting to me.

I agree with Markus that we’re seeing more startups, and more business being generated around robotics, and also value provided to the customer. That’s probably one of the most exciting things I see in field of robotics as a whole.

Audrow: Peter, tell us what is most interesting to you and why?

Peter: I’m really excited about artificial intelligence. The field has made huge progress over the last few decades. We’ve been able to create computers that can beat humans at chess, and that can not only recognize images (dogs, for example), but can differentiate between them (i.e. different kinds of dogs). They can surpass humans at recognizing human faces, and so on.

We cannot match the human brain (or even animal brains) at certain tasks, but I think we’ve made huge progress in artificial intelligence and I cannot wait to see how intelligent robots will be in the near future. That’s what really excites me.

Audrow: Sabine, what is most interesting and why?

Sabine: Ten years ago when we were starting the podcast, we were worried we wouldn’t have enough material to keep the podcast going every two weeks … that we’d somehow run out of topics. That definitely hasn’t happened. Looking at the topics from the past 200 episodes, they are all very different – it really shows the breadth of the field.

If I had to pick one (and I’m biased because this is the field I work in), I’m quite excited about swarm robotics. The reason is because we have finally reached a point where we can make a thousand plus robots and have them coordinate, and these are the sheer numbers that start to make swarm robotics interesting.

An obvious application is the biomedical field because, whether you’re talking about cells or nano-robots or molecules or chemistry, these are things that work in the trillions. In large swarm numbers, a very simple entity could be helpful for detecting cancers or doing something interesting in the body. That’s the field I’m excited about right now.

Audrow: What are your best memories with the Robots Podcast, Sabine?

Sabine: Starting up the whole podcast, and getting it going.

Markus was doing the episodes for Talking Robots, along with Peter, who has always been there doing the audio. We were all at Dario Floreano’s lab at EPFL in those days, and we used to move the couch against the wall to help soundproof the room so there wouldn’t be any echo. Peter is a professional musician, so he had this super sophisticated setup with real microphones, like the ones used in radio shows. We’d have this whole setup right in the laboratory. It was really fun.

In the beginning it took a lot of time. Audrow, you’ve done an amazing job in terms of continuing the podcast and making sure that we can keep going by having it run so nicely. What’s also fun is to see how it’s evolved into something that works, that’s efficient, and to see just how global it’s become. Per is in Tokyo, Markus is in Switzerland, I’m based in the UK, and Audrow, you’re in the US. Seeing that evolution from all of us in this office at EPFL, up to something that’s this global, that’s been great.

Peter: Now that Sabine is telling the anecdotes from the early days, I have to agree that was certainly fun. I think for me it’s seeing the team grow. I’ve been the constant dinosaur in the background with new people like Audrow stepping up to make the podcast better. It’s been really nice to see.

Audrow: Thanks, Peter! Per what have been your favorite moments?

Per: Being able to work with this amazing team, and being able to go out and meet the interviewees and talk to them about what they’re doing their field. I’ve met so many interesting people doing this, and I’m just amazed by all the people I’ve come to know through this work. If anyone out there is listening, doing interviews is an amazing experience: you should try it!

Markus: I’m with Per on this one. For me, it’s all about the people and the people on this team.

I’ve been deeply impressed and inspired by the excitement, energy, dedication and tenaciousness that people have put forward to make this project come alive stay alive for nearly 10 years. Everybody who’s here today has greatly contributed to making this happen and I’m honored to be part of it.

Sabine: What about you Audrow?

Audrow: It’s working with you all. It was an amazing experience going to the same conferences and seeing everyone in person. We were often on calls like this for our weekly meetings, but seeing you in person and connecting, that’s the best part for me.

The final question I have for everyone is: what have you learned from your involvement in the Robots Podcast?

Markus: I think the key thing I’ve learned is that it really matters what people you work with. I’ve long tried to work with people who are better than me, and I think I’ve been extremely fortunate in that respect and I would recommend that to anyone.

Per: Since I’m not a robotics researcher, what I learned from doing interviews is that, when you’re face to face with this person, you’re really trying to be the voice of the listener in that situation. When I listen to an interviewee speaking, I listen in a very focused, structured way – and that’s been tremendously beneficial for me in in general.

Peter: Listening to all the past episodes I notice that the best researchers and most interesting people have a talent for explaining their research in relatively simple terms. I think that’s extremely difficult and it’s a very useful skill to have. In my opinion, if you cannot explain something in relatively simple terms, you probably haven’t really understood it. It’s quite amazing how the best researchers can explain difficult things because they understand them so well.

Sabine: Tying back to what Peter is saying, what I learned is how valuable science communication is. Of all the skills I learned during my PhD, doing the podcast was probably one of the most valuable. Talking to so many researchers about their work opens your mind to all the different areas you might be looking at during your PhD. You have to learn how to ask questions and to master each topic very quickly so that you can talk to them about it. In terms of selling yourself in the real world afterwards, being able to communicate is a key skill, whether you’re in academia or in business.

Hopefully young PhD students will do the same, and will start blogging and podcasting and doing science communication of their own.

Audrow: Alright, thank you everyone. Happy 200^th episode!

Part Two: Interview with Rodney Brooks

Sabine: Hi Rod, welcome to the Robots Podcast.

Rodney: Thanks for having me. I’m honored to be on number 200! It’s great that you’ve had so many!

Sabine: Yeah, we’re pretty excited. The podcast has been running for nearly a decade now and it’s been great to have you on this show all these years.

How has robotics changed since that first interview when you were telling us about the subsumption architecture?

Rodney: Well, in the last ten years a bunch of things have changed.

In terms of subsumption robots, there are now at least 14 million of them out there in the world, so that’s pretty good. There weren’t anywhere near that many ten years ago. The iRobot Roomba is really a subsumption-based robot, and it’s been pretty successful in that niche.

I think the other big thing that’s changed is ROS. I love ROS, but it could have easily been another competitor. It’s sort of been a winner-take-all situation. The fact that so many people are using it means that people get to work together and exchange things, and it’s really taken off and grown the research arena. It has also let people see what robots could do, helping them move toward a lot more practical applications than we had ten years ago.

Sabine: I think a lot of our listeners might go back and re-listen to your more recent interview about Rethink Robotics. How is the company doing? Can you tell us about Sawyer and how it is different from Baxter?

Rodney: We had always intended to have more than one robot, but with a common software platform. When you buy most industrial robots, it’s like buying a smart phone. The manufacturer says, “If you want a camera, you can buy that as an extra, and if you want a microphone you can buy that as an extra. Oh, and by the way, it’s possible to write programs so that it can operate as a phone if you write the code.” In other words, most industrial robots come as empty shelves.

We decided to have a software stack with intelligence that works across multiple robots, and we use the analogy that iOS runs on iPads and runs on iPhones.

Baxter was our first robot. We learnt a lot from it, it’s a great robot, and we’re still selling a lot of them. At the same time, we saw an opportunity for a smaller robot, especially in China where it could work side by side in close proximity to people. A lot of people still think that China has so much labor they couldn’t possibly want robots, but if you look at the demographics in China, the absolute number of 19 year olds is dropping by 30% over a 10-year period. Nineteen year olds are where the manufacturing labor force comes from, so China is in desperate straits in terms of labor for manufacturing.

Really good, well-run companies that make electronics for the western world will have a labor turnover rate of 16% per month. You can imagine how hard it is to run a manufacturing company when there’s such high turnover. The nice thing about that problem, though, is that the electronics manufacturers in China have had to break down the manufacturing cost into a lot of simple tasks that a new person can be trained to do in just five minutes. That’s ideal for being able put a robot in there, because it’s not a complex task with lots of judgment involved. It’s a fairly simple task that we can train a Sawyer to do in just a few minutes also, and a Sawyer doesn’t quit after a couple of months.

Sabine: The markets are very different between USA and China. Does that warrant different types of robots?

Rodney: If you go into Chinese factories (and I’ve been going into them for over 20 years), people really are just two or three inches from the next person. A Baxter is way too large to sit at one of those work stations, so we had to build a much smaller robot to go after the Chinese market. We also saw that there was a need to be able to handle five kilograms, which is more than the Baxter payload.

Also, people have hips and can lean far forward, but the robot doesn’t have hips, so we needed a long reach on the arm too. Sawyer’s arm is actually longer than Baxter’s, and it’s longer than Universal Robots’ and the Kuka iiwa as well – this is so that it can do the task that a person with hips can do.

Sabine: It’s really great to see the evolution of these robots. We have a Baxter at the Bristol Robotics lab, and that’s the first thing we show everyone when they come into the lab.

Rodney: Yes, that lab at Bristol is great by the way.

Sabine: As we mentioned, it’s our 200^th episode and we’ve been doing science communication for a long time now at the Podcast and at Robohub. I was wondering why you think it’s important for roboticists such as yourself to go out there and spread the word about robotics?

Rodney: I think it’s important so that people can understand what robots can do. We’re seeing in the press a little bit of incorrect information coming from people who are outside robotics and misunderstanding what it can do. The fear of robots taking away all jobs – I think it’s misplaced.

What’s really misplaced is the fear that robots are going to rise up and decide to take over the world. That makes no sense at all to me. But people who are well-known scientists or technologists (but not roboticists) were saying that, and I think it’s doing the whole world a disservice because there’s no chance of anything like that happening anytime soon.

Anyone who works in robotics understands that we can barely get our robots to do most things. They’re not magic machines. I think it’s important for roboticists to talk to the regular press and help them understand that. And it’s important for roboticists to talk to people like you to reach the robot-loving audience and help everyone understand what the significant research problems are that still have to be solved – because there’s plenty of them, and there’s plenty of room for new people coming with new ideas.

I think my role these days is to talk about the plausible things that robots might do in the hope that it will encourage young researchers to come up with brilliant ideas on how to do it.

Sabine: What are these plausible things?

Rodney: I think the thing that’s really going to drive robotics for the next thirty years is eldercare. If you look at demographics in Europe, in North America, in China and even more so in Japan, the ratio of older people to younger people is increasing rapidly. I just saw a statistic yesterday that within a few years a third of all people in Japan are going to be over age 65, and the ratio of people paying into their social security system for every person being supported by social security is going to go from something like 9.5:1 down to less than 2:1 over the next few years.

There are a lot of people who will not be able to pay for the services that they might want, they are living longer, and they – I should include myself – we, we will want our dignity and our independence as we’re living longer, but we’ll need help.

I see the driver assist features of cars turning cars into eldercare robots because it lets people drive safely longer, which lets them have their independence and dignity longer. I also think robots in the home are going to be necessary to help older people.

I was just at iRex, the international robotics exhibition in Japan. There were lot of industrial robots, and a lot of emergency rescue robots following from DARPA Robot Challenge type applications. But it was also interesting to see a whole bunch of booths from various research institutes and universities with robots to help the elderly, to help them with mobility in the house, help them get into bed, out of bed, to help them wash. These robot demonstrations are not going to be deployable in real homes in the next year or three or five, but I think it’s great that researchers in Japan are now turning to this as a place where they can have a real impact.

I think we’re going to need lots of robots to help the elderly so that the rest of society is not completely swamped by this massive relative number of elderly to younger people.

Sabine: I agree that’s a great application. You mentioned the hype and the misinformation. Is that something that’s changed, or has that just always been the case? And what are some of the things that you need to be mindful with your company when you communicate about your robotics today?

Rodney: I think it has changed because of the high profile of some people who have come out and said rather silly things over the last year or two.

But let’s talk about my own obligations at this company. People see our robot and they imagine it can do any task in any factory, and that’s just not true. It can only do a relatively small number of tasks. Our constant thing is to train our sales representatives on what is possible versus what someone would like the robot to do. They have to be able to effectively communicate that, so that people buy the robot with their eyes open to what it really can do for them.

I remember when Baxter was first announced, a prominent blogger said to me, “I’m so disappointed. I thought I was going to be able to have Baxter steer my yacht.” I was thinking, why would you ever think our robot could take the helm and steer a yacht? There are unbelievable expectations of robots, but a general purpose robot – it’s just not going to be able to do something like that for a long, long time.

Sabine: Maybe in another eight years for our 400^th episode?! What can we expect to interview you about by that point in time?

Rodney: I think we’re going to start to see some companies really being in the elder care space. I’m a little more sanguine about all the companies in the social robotics space … I don’t know if there’s enough room for all the companies that are saying they’re going to have social robots in the home. I’ve taken issue with the idea of companion robots – I’m not sure that’s where it’s going to be. I think it’s going to be more about physical help robots. Hopefully in eight years we’ll start to see some of that.

I think the robots we’re building with arms that are safe to interact with will let more people do research in that area. One of the reasons I’ve really pushed the team to make Baxter available as a research robot was that I thought it was important to have a platform where people didn’t need to worry about the safety of the arms, so they could start doing work with the robot in whatever field they chose. And I was hoping some would choose elder care. There are quite a few university labs that are using Baxters to do things like help an elderly person get dressed or help someone in a wheelchair, or quadriplegic to feed themselves, etc. If you used a regular old industrial arm, that would be a pretty tricky and unsafe proposition for coming into contact with people.

Even though this company is aimed at putting robots in factories, I think the technology of these safe arms is going to enable the next wave of research and companies. Within the next eight years I think we’re going to see some startups in eldercare begin to appear.

Sabine: Excellent. Thanks, Rod, for joining our 200^th episode.

Rodney: Thank you for having me. Congratulations on doing this so well for so long!

Flying robots, with Dario Floreano (Part two)

Matt Wade — Fri, 15 Jan 2016 14:38:11 +0000

Last month we caught up with Dario Floreano, the head of the Laboratory of Intelligent Systems at the Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland. Here we continue our discussion, covering acoustic sensing, multi-drone operations and more. Missed Part 1? Check it out here.

Last time we spoke, we discussed your work on visual perception, but you also mentioned acoustic research. So this means we’re talking about ‘listening drones’?

Yes, that’s right. This is particularly interesting in the area of drone swarming, whereby multiple robots need to coordinate their flights very carefully.

The first big problem with swarming is perceiving the other drones. If you have multiple drones in flight, each robot is only a very small spot in the other robots’ visual fields of view. They’re very hard to spot against the background. So we thought, what about sound detection?

The question is, how can a robot hear other robots when it is in flight? Well actually, this is a simple trick. You simply switch the engine off, very briefly, listen using three tiny microphones pointing in different directions, then restart the engine. Just like the eBee does when it is capturing aerial photos. Using this trick we developed drones that can detect the presence and relative direction of other drones acoustically.

Outside of the lab, what do you see are some of the potential end applications of your work?

Sticking with the acoustic approach, this could be useful in search and rescue, being used to detect people’s locations on the ground. If people are lost in snow or in a forest, they often have emergency whistles. We have shown that we can locate a person using a standard emergency whistle and broadcast its GPS coordinates to a rescue team.

With swarming, we’re also looking at using drones as relay networks for communication between rescue and security teams on the ground.

In the event of an earthquake, a fire or a landslide, the first thing an emergency team usually does is to set-up its communication infrastructure. This typically consists of a set of trucks with radio signal emitters and receivers, which operate on a specific frequency. It can take one or several hours to complete this set-up before rescuers can start working on the ground, using their receivers.

However drones can be equipped with emitters and receivers. Therefore a swarm of drones could cover the area, pick up the signals of the rescue team on the ground, and relay these, connecting people via very low power, short-range emitters. Currently with this technique we can connect multiple people, and not only with audio but with video signals too.

We’re also experimenting with multiple people moving around on the ground, with robots following their movement, and we’re considering operations that last longer than a single battery duration of around half an hour. We’re developing an algorithm that allows drones to land for a battery change, while new drones with a full battery launch to continue the service.

So on one hand your work is bio-inspired research, but you’re obviously also thinking about the potential real-world applications of the technology you develop. Which point do you start from—do you look at biology first and then consider how to use the tech that comes out of that research, or do real-world problems drive the direction your biological research takes?

It’s a mix. When it comes to drones, 15 years ago it was more blue sky research. These days however I have more projects on flying robots that could solve problems that people might face, not necessarily next year but in five years’ time. For example, if you will have lots of drones all in the air, you then think about safety. That means thinking about AI and sensing.

The specific applications are not always there though. For example, when the Gimbal was designed we didn’t have an application in mind, just a problem: how to design a robot that could withstand falls and fly again. We started by adding retractable legs, as insects have, but this wasn’t optimal, so we came up with a rolling cage around the drone. Then we realised the application: being able to go where other drones couldn’t go.

Prof. Floreano’s work at the Laboratory of Intelligent Systems (LIS) is bio-inspired; he and his team develop flying robots with animal characteristics, which they also use to learn more about animals.

In swarming there are lots of interesting problems to think about. In the future, as I mentioned, we’ll have a very high number of drones sharing our airspace. We’ll have a lot of crowding in areas such as landing spots and drone airways.

The big question is this: how will these drones avoid each other? Commercial GPS isn’t good enough to enable this avoidance when you fly very close to other drones. So the question is: what are the best sensors and the best control algorithms, both for drones flying in cooperative conditions—meaning the same types of drone flying in the same area—and non-cooperative conditions featuring drones from different manufacturers using different sensors? This is a real problem that some of our research is addressing, regarding the sensors and algorithms we’re developing.

Another real problem comes in transportation: how can we transport parcels very safely? I can’t talk about the solution just yet, but our work there builds on a lot of the topics we’ve been discussing.

Which seems a logical place to wrap up! Thank you so much for all these insights. How can readers who are interested in learning more follow your lab’s work?

The homepage of the Laboratory of Intelligent Systems is at lis.epfl.ch. That page includes our most recent publications. We also have a LIS YouTube channel that people can subscribe to and I am on Twitter at @DFloreano.

You are also involved with the RobotsPodcast, is that correct?

Well, I was involved in its creation, back when it was called Talking Robots, and I supported it from my lab for two years. The podcast grew and grew, and the students that were preparing it then set-up what is known today as theRobotsPodcast. Many members of LIS still work on this on a voluntary basis. It became so successful that it also spurned another, wider organisation, calledRobohub.

Thanks for your time!

You’re very welcome.

Watch Floreano’s ROBOTS14 presentation:

===========================
About Dario Floreano

Dario Floreano (M.A. 1988; M.S. 1992; Ph.D. 1995) is the director of the Laboratory of Intelligent Systems and director of the Swiss National Center of Competence in Robotics. He is co-founder of the company senseFly S.A., of the International Society for Artificial Life, and founder of the popular robotics podcast series Talking Robots (which later became RobotsPodcast).

He has been on the advisory board of the Future and Emerging Technologies division of the European Commission, vice-chair of the Global Agenda Council on Robotics and Smart Devices of the World Economic Forum, and on the board of governors of the International Society for Neural Networks. Floreano is a senior member of the IEEE and received several national and international awards for professional and cultural achievements. Floreano has published four books, more than 300 technical articles, and more than 10 patents in drone technologies.
===========================

ep.199: Microrobots for Harvesting Crystals, with Simone Schürle

Audrow Nash — Sat, 09 Jan 2016 07:24:36 +0000

Transcript below.

In this episode, Audrow Nash interviews Simone Schürle, advisor and co-founder of MagnebotiX, about using small robots to harvest crystals. These crystals can be used to infer atomic structure. An example application is in drug development to see if protein binding has occurred.

Below is a video that demonstrates the technology discussed in the interview.

https://youtu.be/yL9VtAN8jiI

Simone Schürle

Simone graduated in 2009 from the Karlsruhe Institute of Technology (KIT), Germany, with a diploma (German equivalent to M.S.) in Industrial Engineering and Management. During her studies, she specialised in Micro/Nano Systems and Material Science and was enrolled in two international student projects abroad. She was researching at University of Canterbury, New Zealand, in the field of Biomedical Engineering and at the University of Kyoto, Japan, in the field of carbon nanotube based nano sensors which was sponsored by the Heinrich Hertz fellowship and a German state funding. She subsequently joined the Institute of Robotics and Intelligent Systems, directed by Prof. Brad Nelson, at the Swiss Federal Institute of Technology, where she focused on magnetic manipulation techniques for wireless microrobot control in biomedical applications. She was awarded with a Swiss National Foundation fellowship to pursue her postdoctoral studies and joined the Laboratory for Multiscale Regenerative Technologies at Massachusetts Institute of Technology (MIT) in June 2014. Simone is now interested in applying microrobotic control strategies in cancer treatment, specifically in locally enhancing nanoparticle transport.

Links:

Transcript

This transcript has been edited for clarity.
Audrow: Welcome to the Robots Podcast. Can you introduce yourself to our listeners?

Simone: I’m Simone Schürle and my background is in magnetic manipulation of micro nanostructures. I’m a postdoc at MIT and co-founder for MagnebotiX , a start-up company. I work on calibration and performance measurements, optimising electromagnetic systems for micro-nano manipulation.

Audrow: What is the application of this technology?

Simone: You can use the system in an inverted microscope or an upright microscope – any tiny objects you want to maneuver with areas you need to interact with on a small scale, such as cells. You would use this technology if you want to understand how small structures move, or to explore completely new areas, like the low Reynolds number machine.

Dynamics change when moving from large to small. Let’s say you want to manoeuvre tiny objects in crystallography and extract crystals then blast them with x-rays to get the fractal structure so you can get to the atomic structure. These are extremely tiny objects. Here is where you can be guided from our technology in magnetic micro work.

Audrow: Before discussing crystallography I’d like to understand the system and its parts. Firstly, there’s a magnet microscope. Then induction coils … is that how you control a magnetic robot?

Simone: Correct! On the one hand to visualize the robot, we need vision feedback. So first you need a microscope. Then you’ll need a magnetic control system consisting of eight electro magnets arranged in a single hemisphere. We control the current from each of these eight coils.
Then you have a defined work space where you place your robot. It could be a dish, tank or a 96 well PCR plate, whatever.

Audrow: What does the robot look like? Can you give me an idea of its size?

Simone: You can work with any piece of magnetic material. It could be a cylinder, a sphere or a flake of magnetic material. I know you are probably thinking: okay, is that really a robot? The robotics is really a system that generates magnetic fields from the outside to control it. You can also use different oscillating magnetic fields, rotating magnetic fields and dynamic fields that introduce different kinds of motion. You would not pull on a magnetic object where you apply field gradients to make it move, like how a magnet attracts other magnets.

If you have a magnetic helix and apply a rotating magnetic field it will propel forward. This concept is inspired by bacteria called flagella that have a helix and have rotational forward propulsion. You can apply this to magnetic helixes to get a cork screw-like motion. So this is one example.

Another example is a robot developed by a colleague in my former lab called “magmites.” These use a hammering motion – an impact driven motion – based on the oscillating magnetic field, where the magnets are connected through a spring and one hammers against the other to push itself forward. So this is one example of how you can build a surface walker: you just need a cylinder and to apply a rotating magnetic field and it can tumble over a surface.

Audrow: Can you tell me what is being done in crystallography? Why is this important?

Simone: Crystallography is an old technique, over 100 years old. It’s about x-raying fractal patterns of structures. We use crystallography to figure out new structures, understand what they are composed of and what they look like.

Crystals in our body are composed of all kinds of protein arrangements, and one thing we would like to do target specific proteins for research. But first we need to understand the structure of the protein and then the interaction of a potential ligand to bind to that target protein. Then we extract and grow protein crystals externally, adding a ligand, send to a synchrotron, blast it with x-rays and collect the experiment fractal pattern data to figure out if binding occurred and what the structure looks like. This is important to track development.

Audrow: Tell me about your system and how it relates to crystallography.

Simone: In academia we are in the process right now of trying to target certain proteins, and the pharma industry is searching among these huge libraries of potential ligands to find ones that could bind to these proteins.

Audrow: What exactly is a ligand?

Simone: It’s an interaction, another molecule instructed to bind to the protein.

One thing you can do is to model this kind of interaction so that you can knock out potential candidates, and then you conduct the experiment to grow the crystals. Much has been automated: hundreds of crystals can be grown at the same time and they are manipulated by robotic arms to different stations. They’re very finicky because you have to maintain them at different temperatures, and they have to be shaken all the time.

After weeks of growing them, they’re still in this process chain before you can send them off to the synchrotron. Every two weeks most pharma companies book a full day slot where all their crystals will be analysed.
Until recently, one of the steps in this process – harvesting the crystal – has been manual. This step involves taking the crystals out of their growth solution, where they are sitting in little droplets. Previously, this has been done by highly trained physicists using a nanoloop underneath a microscope, trying to fish out crystals that are the size of the root of your hair. These crystals are extremely fragile as they contain up to 90% water; they break very easily, and they like to grow at cold temperatures, so people must work in cold rooms for hours to fish them out.

Clearly, this is an error-prone and unpleasant process. We saw the need for automation and clearly a micro-robot could do this finicky job. We designed a robot called the “rodbot”: it is essentially a polymer rod with soft magnetic posts in it. We can magnetize it in such a way that when you apply a rotational magnetic field to it, you can make it roll.

Audrow: How large are these robots?

Simone: The robot is about the size of the crystal; it’s 60 micrometre in width and up to 300 micrometre long.

Audrow: Can you give me an idea of scale?

Simone: Your hair is about 80 micrometres in width, so it’s about the width of your hair.

Audrow: Looking at the rodbots it looks like there are holes. Are those magnets?

Simone: These are the magnets.

Audrow: So there are several magnets placed along its length and it’s a rectangular prism?

Simone: Correct. We have these posts perpendicular to its length to get the magnetization direction perpendicular to its axis so it can roll on the surface in the way we needed exactly. So initially we thought we could just approach the crystal and push it. In reality, we don’t even touch it because of fluid dynamics due to its size. There’s a vortex generated as soon as we come close to the crystal.

Audrow: A vortex? Like a whirlpool?

Simone: Yes, and that lifts the crystal on top. We have a non-contact manipulation technique, which is very nice for these fragile crystals. They get trapped in this whirlpool.

Audrow: It rides on top of your rodbot because of the whirl of vortex? Where do you bring it?

Simone: Correct. So the rodbot comes in through these crystallography loops which are like fishing nets. It drops the crystals in that net, like a basketball. This net is taken out; the crystal is cooled, and then sent off to be prepared for the crystallography of the x-ray.

Audrow: The rest of the process is automated?

Simone: Yes, our vision for the whole process to be integrated. You could have computer vision to identify your target crystal, automated path planning to approach the crystal, the rodbot to get the crystals, and a robot arm to remove it prior to cooling.

Audrow: From there you can analyse the crystals, check what has bonded, and then use that for drug development?

Simone: Yes.

Audrow: How do you control the crystal? I know you’re using magnetic fields to move it. Is an operator sitting with a joy stick. You also mentioned computer vision, is it automated?

Simone: This is the ultimate goal of true automation: that you don’t need to have an operator anymore. There are already established computer vision algorithms to detect the crystal. It’s not an easy environment where they grow. However, we can improve environmental conditions and growth conditions.
We can identify the crystals and identify the rodbot, that’s easy. The difficulty is going under an object and finding the shortest path to it.

Audrow: The rodbots are single use micro-robots. Why are they only single use when the don’t come into contact with anything?

Simone: When the operator conducted this manually, the loop used to fish it out was a single use item. What we do is combine the single use item with the rodbot. When this crystal has been finished you have to go to a new drop and introduce a new loop anyway. That’s why the rodbot is single use.

However as this can be batch fabricated, the process of making these rodbots uses microsystems fabrication, and you can generate thousands of them at a time, so they are extremely cheap.

Audrow: So you sell the system for “moving” the rodbot? And you sell 100s or millions of rodbots, and those are used for the crystallography?

Simone: Correct. Crystallography suppliers sell the nylon loops that fish the crystals out, and they also loops that have been preloaded with a rodbot.

Audrow: What have been the major challenges to developing this technology? Or turning it into a product?

Simone: One thing you can ask is: why hasn’t this been automated yet? It’s a complex environment, and there’s no commercial solution available. There are different physical manipulation techniques that have been suggested, such as using robotic arms, but so far this environment is too complex to accomplish it that way.

Audrow: Why is that? Is it too small?

Simone: Because of the dexterity that’s required, as well as the size and fragility of the crystal. Classic robotics usually uses contact, but this can damage the crystal. For us the hurdle when you go so small is that you have to deal with capillary forces and other forces that makes things stick, and this can be a problem.

Audrow: If you could go back and give your 20-year-old self one thing, what would it be?

Simone: Learn one thing really well. Have a solid background in one thing, and then branch out and be diverse. Learn as much as you can and try combining these new ideas with your solid background you’ve mastered.

Audrow: Thank you.

Simone: Thanks very much.

This transcript has been edited for clarity.

Interview with IJARS, with Howie Choset and Seth Hutchinson

International Journal of Advanced Robotic Systems (IJARS) — Tue, 05 Jan 2016 17:07:08 +0000

The IJARS team sat down at ICRA with two influential academics in robotics: Professor Howie Choset (Carnegie Mellon University) and Professor Seth Hutchinson (University of Illinois at Urbana-Champaign) to discuss some of the latest developments in robotics, what students of robotics should be focusing on, and what research they most hope to be remembered for.

A bit tongue-in-cheek, but if you ever wondered what engineering minds like to do in their free time there is a humorous Seth & Howie tradition not to be missed! Seth also shares his insight on what makes a science publication truly relevant.

We invite you to watch our interview with the Fantastic Robotics Duo. Enjoy!

Credits: ECE Illinois and Carnegie Mellon

Seth Hutchinson is a faculty member at the University of Illinois in Urbana-Champaign, where he is a professor in the Department of Electrical and Computer Engineering, the Coordinated Science Laboratory, and the Beckman Institute for Advanced Science and Technology. He received his Bachelors, Masters, and Ph.D. degrees in Electrical Engineering from Purdue University. He has published more than 200 papers on the topics of his research interests: vision-based control of robotic manipulators, robot motion planning, and computer vision.

Howie Choset is a Professor of Robotics at Carnegie Mellon University. Motivated by applications in confined spaces, Choset has created a comprehensive program in snake robots, which has led to basic research in mechanism design, path planning, motion planning, and estimation. Choset directs the Undergraduate Robotics Minor at Carnegie Mellon and teaches an overview course on Robotics which uses series of custom developed Lego Labs to complement the course work. In 2002 the MIT Technology Review elected Choset as one of its top 100 innovators in the world under 35. Choset also co-founded a company called Medrobotics (formerly Cardiorobotics) which makes a small surgical snake robot for minimally invasive surgery.

Choset H, Hutchinson S. IJARS Video Series: Howie Choset and Seth Hutchinson interview with the International Journal of Advanced Robotic Systems [online video]. International Journal of Advanced Robotic Systems, 2015, 12:V10. DOI: 10.5772/62191

ep.198: Construction Drilling, with Konrad Fagertun

Audrow Nash — Sun, 27 Dec 2015 11:05:36 +0000

Transcript below.

In this episode Audrow Nash interviews Konrad Fagertun, Chief Operating Officer of nLink in Norway.They speak about a mobile robotic platform for the construction industry. The problem that they’ve chosen to solve is drilling holes in the ceiling. For a shopping mall, 400,000 holes may have to be drilled — by hand. This is a task Fagertun says “no one likes” so nLink built a robot to automate it. Fagertun discusses their robotic platform, how they chose their application, and the future plans of nLink.

Konrad Fagertun

Konrad has been involved with technology startups since 2006 and holds a masters degree from NTNU School of Entrepreneurship (and Boston University), with a cybernetics technology background. As one of the founders, and Chief Operating Officer, he focuses on business development, partner relations and sales in nLink’s quest to revolutionize the construction industry.

Links:

Transcript

This transcript has been edited for clarity.

Audrow: Welcome to Robots Podcast. Can you introduce yourself to our listeners and talk a bit about your company?

Konrad: My name is Konrad Fagertun, I’m the cofounder of nLink Robotics. We make the world’s first mobile drilling robot for construction. In fact, I’ve just started commercializing our first robots.

We’re a small robotics company with only four fulltime engineers and several more who work by the hour. We are developing a robotics system that helps contractors do heavy work on large construction sites. The first task we started is on concrete ceiling drilling. It’s a small niche, but it is heavy and time consuming.

Audrow: Can you describe the robot and discuss more about this issue? Why construction robotics?

Konrad: With large-scale construction, for example, shopping malls or office buildings, electricians and plumbers drill thousands of holes in the ceiling before mounting constriction equipment – like cable trace, light fittings, ventilation, and air-conditioners. You have to measure every single hole and drill, which makes it very time-consuming and creates lots of noise and dust. Nobody – trust me – nobody likes this job!

We solved this by adding an industrial robot arm to a lift, then sensors and a drill. Now the robot can measure and position itself inside the room and drill with better accuracy than manual labor. It’s much faster.

Audrow: How is the user interaction? How would someone set up the robot?

Konrad: In the current version there is a laser station which is a tripod with a laser in the corner of the room. Then using an iPad, that we will supply, we made an easy to use app. You can use it manually and enter one-on-one hole or upload data from a CAD drawing. For example, you can use x and y coordinates and so forth.

Audrow: What kind of pattern does the laser tripod display so the robot knows where to drill?

Konrad: Right now it’s a laser cross in the first position to drill, or as a reference point.

Audrow: Does it create a cross or an angle? Or does it create lines of lasers where at the intersections of lines is where the machine drills?

Konrad: Somewhat. You put one laser cross where you want to start drilling or where you want the robot to look. Afterwards, the robot displays its own laser lines in the ceiling and calculates where to drill the next holes. So there’s no additional measurements it’s just getting that first hole, first light hitting, or a reference point. There may be quite a few laser lines in the ceiling, but that’s the robot trying to figure out if the ceiling or floor is tilted. It always drills perpendicular to the ceiling.

Audrow: What do you mean tilted?

Konrad: In a parking garage the floor is tilted for water allowing different floors and areas to be at alternate heights. When you drive the robot they will, in some positions, be at an angle to the ceiling. The robot will compensate and always drill perpendicular to the ceiling. There doesn’t have to be any accuracy on how to position the robot itself.

Audrow: I see. So the user puts down the laser tripod, uses an iOS or Android app to tell the robot to go, points out the spacing between the holes … is that right?

Konrad: That’s the basic function. You can also use BIM files; 3D drawings that will help subtract the holes from the drawing.

Audrow: BIM are binary files. So something you compile or build?

Konrad: Yes, it is building information modeling. These are getting popular in the US and Europe. You can subtract whole data, for example, x, y and z data – for each hole. That’s easy for the robot to interpret and drill. We even drilled a ‘smiley face’ using the information modeling. I don’t know how useful it is but does show what it’s capable of doing!

Audrow: Is it like a snowman smiley face, like a whole series of dots or is the mouth connected?

Konrad: We made a whole series of dots … 256 actually!

Audrow: Oh wow! What kind of sensors do you use to instruct the robotic arm to go to the correct location and drill the hole?

Konrad: The system uses two different sensors. One is a laser station and can be put anywhere in the room as a reference point. The second is a stereo camera system attached to the robot. This system will always look up at the ceiling and look for the laser lines it makes, then calculate how the ceiling looks if creating a model. That way it always knows where to drill and the exact height of the ceiling. It drills, for example, 1.1 inch up if that’s what want from it.

Sometimes there are restrictions in the concrete, you can’t drill too far. It’s important to control the depth of the hole. That’s very easy combining both the laser system and the camera system.

Audrow: What sensors are you using on the robotic arm so it can orientate itself? Is it the vision system or are you’re modeling the robotic arm?

Konrad: We can control each joint of the robot and calculate where it is. It’s pretty easy and something all industrial robots can do. They can tell where each of the joints of the arm is located every second. We combine with the app so the user sees any progress, like what holes have or have not been drilled. At any point you can monitor if you’re thinking, “Okay, what progress is left on this particular job? Oh, okay, we are 70% done, only a few more holes left.”

If you have any problems with a single hole the system will mark as unsolved. A worker can come back and change or fix that problem hole. There could be something in the concrete or it could be something else. There are always some irregularities.

Audrow: On the website it states this robot is constructed using off the shelf components. Can you talk a bit about that?

Konrad: We mainly use off the shelf components. We do make key components ourselves, such as the software. The largest part is the robot arm and we use a UR10 from Universal Robots, a Danish company recently bought by a US company. We also use the later laser and standard drills – standard cameras and standard vacuum cleaner and other components. We also have a patent pending system and key components that will help the system drill with increased accuracy.

Audrow: Are you also buying the lift that ‘lifts’ the robot up to the ceiling?

Konrad: That’s right. Depending on the height, we use different type of lifts. For high ceilings we need large lifts. Sometimes we need belt-driven lifts instead of wheels when there’s a difficult surface.

Audrow: Do you connect the robot to the lift, or does the operator drive the robot where it should drill the hole?

Konrad: Right now the operator has to drive the lifts.

Audrow: What should we call this particular robot? The robotic arm that drills holes in the ceiling, does it have a name?

Konrad: Actually, so far we don’t have a name. If any listeners have a good name for us we’re happy to hear it! We just call it the “mobile drilling robot” or just Drilli.

Audrow: Nice! Tell me a bit about the business model for this robot. What is the thought process behind renting it out to companies?

Konrad: Most contractors rent equipment like lifts. People are used to renting drills, excavators, all kind of machinery. They’re used to this model and they’ve asked for it so they don’t need to invest in robots. Typically, they will rent a robot for a week or two in a building, so there’s no need to own it. It’s better for them and easier. They just pay the hiring fee and save money and time right off the get go. It’s easy for all everyone, actually.

In the beginning we’ll rent it out with an operator, but eventually we want to train operators at our customers’ request. That way they can operate the robot and just rent the robots, like you rent a car.

Audrow: How large is your company?

Konrad: It’s less than ten. We’re working with large potential distributors and wholesalers who can help with distribution and logistics because these machines have to be moved from location to location in order to be effective. We will need professional logistics partners, we’re looking at Europeans companies to help with logistics in Europe and eventually worldwide.

Audrow: Does having worldwide interest make it difficult since you’re using this renting model?

Konrad: It’s a luxury problem. Right now we don’t have the capacity to fly to Hong Kong, Australia and the US at the same time. For scaling up, this is something we have to solve. In the beginning we’ll start with the two robots we have and build success stories with private customers. Then, we’ll invite people here to train and look at demos and that will make it easier for us to expand. When we want to do a job in — let’s say Brazil — we can go there for a couple of weeks, train them, then leave the robot for them to finish. That’s how we’ll solve this in the short run.

Audrow: How long is the short run?

Konrad: Probably six months, the rest of 2015 and start of 2016. It is important our own crew is with the robot to ensure the good customer care quality and the ensuring customers have no problems. We don’t want to leave customers alone with a robot that turns out to be a problem.

Audrow: Eventually will you do something similar in distribution centers at these locations that carry your robot?

Konrad: Probably, that sounds smart!

Audrow: One of your company ambitions is to revolutionize the construction industry. Can you tell me about that and why you’ve chosen this goal?

Konrad: We believe robotics is the solution to change the construction industry. Maybe in the future, the far future, we’ll see 3D printing of houses as they become more popular and feasible. However right now in the next five to 10 years I believe digital solutions and robotics will be a major part of this significant change in the construction industry.

Every time we show someone the robot they start questioning other things it can do. “Can it drill? Can it paint? Can it lay tiles? Can it do other things?” Of course it can! The important thing is positioning and navigating. When you know how to do that you can put whatever tool you want on the tip. There are possibilities in the construction industry and that’s where our hearts are right now.

Audrow: How did you choose the construction industry?

Konrad: We wanted to do something with robots and began considering industries not familiar with robotics or automation. The construction industry is a little bit behind when it comes to productivity and innovation. It seemed like a good place to start.

Audrow: How did you determine that you’re behind in terms of productivity?

Konrad: If you take a walk at the construction site you’ll see one guy with one hammer or one man with one drill working slow, carrying things. It’s heavy work. I have a lot of respect for people working there. It’s difficult.

The construction industry has also been struggling with productivity. However other industries, like the, car industry has skyrocketed in productivity. One man can produce 10 cars because robot and computer automation. There’s significant difference.

Audrow: What other industries did you consider before settling on the construction industry?

Konrad: We did consider offshore oil and gas, it’s a large industry. We also considered the medical industry and healthcare. Essentially oil, gas, healthcare, maybe education and farming, agriculture, it’s interesting and hard manual labor. Lots of opportunities!

Audrow: Definitely. How did you come on this specific issue of drilling holes in the ceiling?

Konrad: We talked to as many people as possible in the construction industry. We thought about painting and tiling first. An electricians company spoke to us and said, “Well that’s nice, but what we really need is help with measuring and drilling holes. We do this every day and its back breaking work.” They didn’t use those exact words, but that’s how we felt they said it. The idea came from an industry need.

Audrow: So then you present your robot as a solution to this problem. What has the feedback been between industry and what you’ve done with the robot?

Konrad: The feedback is pretty good so far. Everyone from workers on the floor, to CFOs, everyone loves it. The workers are happy not to continue doing a heavy workload and the CFOs see buildings completed a week ahead of schedule.

They get documentation that can be valuable for 50 to 100 years afterwards. When you do maintenance you know exactly where every important part of the building is. The documentation part has positive effects, as health and safety issues are major benefits. Plus, workers are delighted to control a robot instead of just a drill.

Audrow: How has the robot changed since you’ve introduced it to industry?

Konrad: The main form hasn’t changed much. What has changed is the way we measure how we drill in the ceiling. It’s difficult to explain. We’ve changed how we use lasers and prisms so the operator doesn’t have to do it for every single move of the robot. You measure the start and ending points where you want to drill and basically the robot calculates everything in between. There are some major savings there. Also, we have found some new solutions that will help the maintenance and service life but that’s not leasable for the operators. That’s just things we have discovered ourselves.

We’ve also seen the need for more rugged devices that can drive over heavy and difficult construction sites, especially with belts. Robots can drive over rubble and work and position it. That’s what we will learn over the next six months. We’re making a version now that will be a lot cooler!

Audrow: People that drive over rubble? Like tank treads?

Konrad: Yeah.

Audrow: Awesome. If the robot arm and vision system is separated from the lift, and if you’re connecting the two that means the system becomes a lot more complicated. How are you going to find a way so that the robot doesn’t bump into paint cans or something laying in the way? How are you investigating? Is this a current ambition of nLink?

Konrad: Yes, we’ll do this step by step. It won’t drive around at night by itself for now but in the near future we can do that. In the beginning we’ll still need our operator to take care of it and to look after it. We’ll use crash sensors for avoiding people and obstacles. I’m not going to say easy but there are a lot of solutions we can build on so we don’t have to start from scratch.

Audrow: So if the user is always driving avoiding collisions isn’t too much of an issue?

Konrad: Yeah, exactly. Our ambitions are fully autonomous robots but that’s not going to happen this year. But it’s coming.

Audrow: Fully autonomous as in you drive a truck to the construction site and tell the robots to go and they do it?

Konrad: Yeah. Tesla currently has semi-autonomous driving so maybe you don’t have to drive down to the construction sites either.

Audrow: That would be awesome. What are some of the ambitions of nLink moving forward? You mentioned tiling and painting, what is next?

Konrad: We’ll continue with drilling robots and upgrade it to do extra tasks.

Audrow: Extra tasks as a robotic platform? And changing the end of the robotic arm so it can do other tasks?

Konrad: Yeah, we’ll add more tools. I’m not going to go into detail but it’ll be related to measuring and drilling in ceilings, walls and floors. We’ll give it extra features that will make work quicker and easier for people. Of course we’re not going to do this all by ourselves. We’re looking for partners, distributors and logistics to help. We have a couple of very interesting names on our list. This fall and winter it’s going to be very exciting for us and I hope we’ll have good news to report.

Audrow: From all of your experience with nLinks what are major challenges you’ve encountered? Have you learned anything?

Konrad: One finding is that customers think it’s amazing and they are the most important assets. We need customers to help us develop this. Not the math and algorithms but telling us how they usually work, how the process should be, the sound of concrete. Customers have told us, “Oh that’s soft concrete, you have to push a little different or, okay, the drill bit needs to be changed.” We learned a lot that would have been impossible to do in lab.

One thing we have learned is that we have to be out there with the customers. Just helping them or getting feedback. Not getting mad when they’re frustrated. That’s the first thing. Second, we learned about vision systems. There are challenges there we have overcome. 3D vision is not easy. If you can use something that’s already developed – use it! That will save you a lot of time. That’s some of the challenges we have had.

Audrow: What advice do you have for those looking to build a company around some sort of niche industry?

Konrad: We have a defined niche, that’s really the best way to do it. There’s less competition and fewer customers to talk to and the customers you find are very important in that niche. Stay in that niche as long as possible before spreading out in all kinds of directions! Find a team that can handle the technology you’re working with and the business and customer side. If you don’t understand the industry or the business you’re working in, you’ll have a hard time with the technology part.

Audrow: Thank you for being on the podcast!

Konrad: Thanks!

ep.197: Multi-Agent Systems and Human-Swarm Interaction, with Magnus Egerstedt

Andrew Vaziri — Fri, 11 Dec 2015 16:51:31 +0000

Transcript below.

In this episode, Andrew Vaziri interviews Magnus Egerstedt, Professor at Georiga Tech, about his research in swarm robotics and multi-agent systems. They discuss privacy and security concerns, as well as research into interfaces designed to enable a single operator to control large swarms of robots.

The video below shows some of the strategies used by Magnus’ lab.

Magnus Egerstedt

Magnus Egerstedt is Schlumberger Professor in the School of Electrical and Computer Engineering at the Georgia Institute of Technology, where he serves as Associate Chair for Research. He received an MSc in Engineering Physics and a PhD in Applied Mathematics from the Royal Institute of Technology, Stockholm, Sweden, and a BSc in Philosophy from Stockholm University. He was then a Postdoctoral Scholar at Harvard University. Dr. Egerstedt is the director of the Georgia Robotics and Intelligent Systems Laboratory (GRITS Lab), a Fellow of the IEEE, and a recipient of a number of research and teaching awards, including the Ragazzini Award from A2C2.
Links:

Transcript

Andrew: Welcome to the Robots Podcast. Can you introduce yourself to our listeners and tell us about your research focus?

Magnus: I’m Magnus Egerstedt and I’m a swarm robotics professor from Georgia Tech. I’m interested in finding out how to get large teams of robots together, in particular when the individual robots may not be all that talented. The question: how can they make decisions so that beautiful, effective and exciting global patterns in behaviors emerge?

Andrew: Is this limited to physical robots, or are there other systems where this can be applied?

Magnus: I’m mainly interested in robotics, but this question of distributed decision making is really something. It could go from smart invertors on the smart grid to computational units and video games, to biological systems in that order. We have cells, we have birds, and we even have human societies – they’re all distributed systems as we walk through life with limited information. And yet, we’re able to build anthills and levers and human societies.

Andrew: Swarms of robots …. distributed systems … these are very complicated things. How do you generally go about trying to model those?

Magnus: I always start by pretending that I’m a somewhat stupid robot with very limited information, and start by asking: “What would I do? How would I think about this?” After I’ve tried to do that, step two is almost always look to nature, because nature is filled with examples of elegant solutions to our problems. Now, I’m not a biologist and I am by no means trying to mimic nature. I’m shamelessly stealing good ideas that social and behavioral biologists have had. Then, when I’m done stealing from nature, I sit down with pen and paper, work out the math, put it on the robots and find it doesn’t work, because it never works when you go from math to robots. You have to iterate on the actual robots a few times.

Andrew: When you’re working with these robots there are constraints that we don’t have in nature, in terms of the way they’re able to communicate with each other. Is it difficult to model that? What is the approach?

Magnus: When you actually start deploying large teams of robots there are some things that are just impossible to model.

Things happen that you didn’t foresee: robots bumping into each other saturating other communication channels, weird occlusions and the infrared sensors messing with each other. There are always things you cannot model. One of the great arts of swarm robotics is: “How do you go from rudimentary fables or models that are somewhat relevant to things that can be deployed on a large system?” This transition is a lot of times tricky, but it’s also deeply satisfying. Swarm robotics is exciting because in the last 10 or so years we’ve gone from spotty anecdotal understanding of how to structure these things, to large teams of robots on the ground, water, and air that can actually coordinate their actions and build interesting geometries.

We now understand certain things that we didn’t ten years ago. Now it’s common to see 25 robots coordinating actions on YouTube. Ten years ago this would have been unheard of. The basic functionality of how do we build these geometries or get these global behaviors out, we’ve kind of understood both on a practical and on a mathematical level. Now there are a huge number of issues we don’t understand. Let’s say for instance one of the robots is not playing by the right rules, or let’s say it’s broken – or even worse – it’s malicious. How do we identify this? How do we not let our robots be duped by these bad robots? There are a huge number of questions that we clearly don’t understand yet, but do we understand the basics of how to assemble these kinds of structures.

Andrew: I think that’s interesting you mentioned malicious actors potentially being in the system. I know you recently published a paper about differential privacy in multi-agent systems. What is differential privacy, and what is it good for?

Magnus: Let’s say you and I and other robots are trying to do something. But what we’re doing individually is really having our own cost of utility function that we’re trying to minimize. I may like to be where it’s light and you want to be positioned so that you’re not too close to flying robots, because you’re afraid of flying robots. We have to live in this space together, but we have our own objectives. Differential privacy is a way of hiding certain information by sprinkling noise onto what we do. For instance, I may move in a little bit of a noisy way so that you can’t figure out what I’m up to. We’re still respecting each other’s actions, but no one can infer what I am trying to do. The trick here is to insert the noise in a clever way so that we don’t end up doing erratic things. We’re achieving what we set out to do as individuals, but no one can tell what’s going on.

This idea of differential privacy came from the database literature. Let’s say we have a database of all our salaries. I add my salary to the database. If you know how many people are already in the database, and you know how it changed, then you can back out my salary. But if all of us sprinkle in a little bit of clever noise then certain things about the salaries – like what standard salary and so forth – you can still get from the database, but you cannot back out any individual pieces of information. It’s the same idea with swarm robotics. You sprinkle noise on the motions to hide your intentions. This is a flip-side from the malicious behavior where you want to be able to detect a robot isn’t playing by the rules. There are two sides of the same coin. One is to hide your intentions, and the other is you want to be able to figure out what bad robots are doing. Bad robot!

Andrew: I think a theme that we’ve all seen in our local big box stores is that everything wants to be smart these days: smart grid, smart homes. Is that something that we should maybe take with a grain of salt given the concerns of being able to peer into these systems and learn something or maybe we shouldn’t know?

Magnus: Fortunately yes, I think one thing in general when we talk about technology is it’s not going to be decoupled from us; we’re going to be users of technology. We need to – in my mind – build our systems so that it respects us as individuals and maintains our privacy. People are concerned about self-driving cars. We’re not going to wake up tomorrow and be sitting in self-driving cars. This is sneaking up on us whereby we get more autonomy in the cars, but people are engaged. People are almost always going to be in the mix when it comes to distributed robotics systems. Also distributed gadgets and the internet of things; people are going to be on the internet of things together with their things.

Andrew: Differential privacy could be one way to make the internet of things a little bit more private. You mentioned self-driving cars. I know that you’ve done research on the other side of the coin where malicious agents are actively trying to change or affect what the system does. It sounds like a plot from a spy movie. Could you tell us a little bit about controlling an enemy unmanned aerial vehicle by exploiting its own safety systems? How did that work?

Magnus: We were really interested in this question, as we’re about to see aerial drone delivery systems. Google and Amazon, if you believe the talk, are going to be delivering things by aerial drones. The question is then: “Is it possible to take over these systems and take over in such a way that the systems themselves do not know they’re being taken over?” This is not a classic cyber security question; it’s a cyber physical security question.

A lot of these systems including our aerial drones that we were playing on have safeguards to hear our reasonable physical behaviors. Basically it’s – here is the stuff that is within reason! And as long as we are within reason we’re going to be fine. Our question became: “Can we hide malicious intent inside the noise of what is considered a reasonable behavior, and yet, get the systems to do completely the wrong things.” I don’t want to say that we completely demonstrated this is the case, but we’ve certainly demonstrated examples where it is possible to hide and trick the system into thinking that it’s doing the right thing. We’re camouflaging the malicious trajectories inside reasonable and rational behavior.

Andrew: In this particular example how did you inject for that signal, what was the mechanism by which you influenced the aircraft?

Magnus: In my lab we’re not hackers. Since we designed that aircraft we had an error system or a safety system that detects that something was wrong. Then we transferred wirelessly over the network and injected new signals. If someone is really clever they could probably build something to detect this is going on within the cyber security side. That was not what we were interested in. What we were interested in was to see if it is possible to hide the malicious takeover within the cyber physical security system. This is similar to the idea behind Stocks net. Where nuclear centrifuges control systems were made to believe they were behaving right. Not because someone hacked in, but because you made the signals look plausible to the security systems.

We actually drew a lot of inspiration from … this is kind of amusing … from dragonflies. Dragonflies will move in such a way. Let’s say you’re a dragonfly; you’re a male dragonfly and I’m another male dragonfly and now we’re going to fight. I move in a way to have the sun in a straight line behind me, so now you can’t tell where I am. I go in making sure that no matter how you move I’m going to move, so that you only see the sun and then BAM! I’m going to get you. Similarly in control for aerial there’s a notion of an observable subspace which here are things that the trajectory of the system can do that’s hidden from the output. We exploited this by restricting emotion to these unobservable modes of the system.

Andrew: That’s very interesting. This adds a whole other layer. It’s not just cyber warfare, it’s not that you hacked; you changed the code that was running on the target. It’s exactly intact as it was designed by its creator. You’ve changed what it observes and what it thinks it observes. That seems like a very hard thing in general to defend against, to not only take into account the cyber aspects but also all the ways things could physically interact. Is there a principled way to make sure that systems aren’t vulnerable in this fashion?

Magnus: We have been looking into this question and at this point there isn’t a principled way but here are ways people have done it. We looked at what we actually called motion probes. The system every now and then does a somewhat strange manoeuvre, where it’s basically a signature manoeuvre. It could make a small sidestep, or the quad copter makes something that resembles a figure eight. This is unexpected; if the attacker isn’t expecting it then you can actually excite these previously unobservable modes and say, “Hey, something is actually not behaving the way it should be there!”

This is how we are thinking about it and it also goes back to this question of detecting not malicious takeovers, but malicious agents. If we are swarm robots, we’re supposed to do something. If I then make a figure eight I know what you’re supposed to do in response to that, if you are one of my people. If you don’t respond correctly I can detect that perhaps something is fishy here.

Andrew: I’d like to switch gears a little bit and ask about something that’s high risk, but in a different sense. I recall reading, maybe four years ago in a systems magazine. There’s a little blub about you and you wrote about your research philosophy and research style. You mentioned liking to keep a few high risk, high payoff topics on the horizon. I just wanted to ask, what does that mean? What does a high risk, high payoff topic look like to you?

Magnus: The wonderful thing about being a professor is that you’re largely free to explore things you find interesting. I have to pay the bills and, more importantly, make sure my graduate students don’t stave. I need to do things that are five years out which people care about. But I also have the freedom to think about things I truly am interested in that are way out there. If they come to fruition they’re going to be important, but if they don’t chances are they don’t. I always like to have a few of these questions that I think are fundamentally important and interesting down the line on the back burner.

Every now and then I return to them, so right now I have a few. One thing that I’m really fascinated by is this question of human swarm interactions. Let’s say you’re a person with a joystick surrounded by a million robot mosquitoes and you want them to do something. What are the right ways to engage with truly large teams of things? That’s one thing that I’m fascinated by right now. Another thing that drives me absolutely nuts is that people talk now about heterogeneous multi-robot system.

Andrew: What does that mean?

Magnus: It means different kinds of robots. In my lab, we have things on the ground and things in the air. But fundamentally, why is it good to be different? Why should I have fast and slow robots together? What is it that makes that better than just having a bunch of fast robots?

To me, this is an interesting question because we can understand what kinds of robots we should deploy. If you have an earthquake and you want to send out robots to figure out what’s going on in the rubble you can say the optimal thing to do is to have nine snakes robots, 15 aerial robots and 24 ground robots. And here’s what they should be doing. To me this is fascinating. I again look to nature like ecosystems which are highly diverse and consider, why is that? How come hummingbirds’ live next to sloths? Is there something fundamentally good about that?

Andrew: I recall from some years ago you mentioned one of these high risk areas was human swarm interaction. How has it worked out over the past few years there were moments where it seemed dire?

Magnus: First of all, human swarm interaction has become an expectable thing to spend your time on and there are other people engaged with it. I think we’ve understood things, if you have small teams of robots then an effective way of engaging with them is actually to grab hold of a leader and inject information through the leader. That works well up to maybe four or five robots, then it kind of collapses. The reason why is that people get bad at imagining what – let’s say I have ten robots around me and I grab one and I shake it. How does that propagate through the ten robots? We just lose track.

For example look at universities, militaries, big companies … we’re all hierarchically organized. I think the reason becomes we don’t know what happens if we have 250 people to manage, all at the same time. I looked at sheep herding and found you’re not controlling individual sheep. You’re pushing things at the boundary with herding dogs. Instead of controlling individuals you’re exerting forces on the outskirts of the swarm. It turns out we can do some things like that quite well. I have worked with swarming tractors, so the question becomes: What does the farmer do at the side-line of the field with an iPad? Maybe one way of engaging is not to engage with the tractors at all but to engage with the map and say, “Here are regions of interest. Now you tractors, go figure it out!”

Andrew: You did some research with a leader-follower system and in that case it was a haptic interface, could you tell us a little bit about that?

Magnus: Yeah, this goes back to this question of its very unclear for people what it means to control large teams of things. We needed some way of making this information concrete to a person.

We basically computed something that we called the manipulability, which is a measure of how easy it is to move this swarm in particular directions and this we could compute. Then we used that to generate forces. The idea is try to push the swarm in a direction in which it was hard to move it then we’re experiencing large forces. It turned out that this was fairly useful. We did user studies that people could solve tasks that were very hard to solve with only a joystick, but with a haptic joystick they could solve tasks more effectively. This goes back to the question of, what information you actually need about a large swarm. In this case it was just force. If you want them to do more elaborate things you probably need more information. But if in general you just want to move the swarm from point A to point B then we thought the findings in this study where haptic feedback serves a strong and positive purpose.

Andrew: Could you paint us a picture of what the experimental setup looked like? What were the robots like? What were the users looking at and what were they holding?

Magnus: The user was holding a phantom omni, the brand doesn’t matter. It was a haptic interface; it looks like a little pan connected to a ball. You’re moving it up, to the side, or in. You’re experiencing a force depending on what feedback that the little haptic force generating ball is giving you. That was what the user was holding. Then the users were in the lab and we had 10 robots. The robots were arranged in a particular formation. What the user could do is select which robot, he or she, wanted to use as the leader and then drag it in various directions.

You can imagine having a long and skinny formation and the leader robot is embedded inside it in the middle. If you move along the direction of the long and skinny thing then that’s going to be hard! You have to move all these things out of the way and you’re experiencing lots of forces. It has what’s called low manipulability. If you move it away from the long and skinny direction, then it’s easier to melt into a form or shape that follows the leader. The user was looking at the 10 robots, the user did not know how the robots were organized, and basically tried to get them to go from one point to another point while navigating a little bit of an obstacle course, so no robots were allowed to actually get stuck behind an obstacle.

Andrew: How did you assess if it was being effective if the person felt like it was a good interface or not?

Magnus: Yeah, this is always when you do user studies, how do you know if it’s good or not? We used three metrics. One, we just timed them. So how long does it take? Two, we looked at how much did the user have to work. We summed up the control signals that were sent to the swarm. And third, we asked them! Basically we asked: did you find this annoying? There are probably more scientific ways of asking that question, but that’s basically what it boils down to: what level of stress, engagement, or discomfort did you feel when you were doing this?

Andrew: A more advanced interface might be something you alluded to the farmer by the field with a tablet and algorithms that were suited to that. Could you tell us a little bit about that?

Magnus: The backdrop to this is rather interesting. The tractors now are really advanced pieces of machinery. They can largely drive themselves. Put in the GPS of the path and then they can, more or less, do this by themselves. Moreover, cornfields are really not that complicated they’re nice and straight. In terms of the types of environments you’re encountering they’re not that complicated. The manufacturing floor is way more complicated, or my daughter’s bedroom … way more complicated. The farm field is not that complicated. This tractor manufacturer came to me and said, “You know what we know now.” So how do we make these autonomous, which means we don’t need a driver in the tractor.

That means we can make the tractors smaller, and that’s good. Maybe because it has less environmental impact, you don’t have to go back and retail afterwards. These are all good reasons why small is better. Except that it’s not less effective. Instead of having one small tractor lets have ten small tractors. Then the question is but how do you actually engage with these? They literally came and asked, “What should the farmer do, we want to give the farmer an iPad, what should the farmer do?” That was the thing we got rather excited about.

Andrew: What did you end up making, what algorithm and what kind of user interface or display?

Magnus: We ended up going with something that’s fundamentally a good idea, which is that the farmer should actually not deal with the tractors. What the farmer should be dealing with are higher level tasks. In this case what we had is a map of the world and the farmer could outline things that were interesting, or things that were urgent. Or where are where there was lots of moisture, or not a lot of moisture. Things that basically described we used, we call density functions to describe areas of the interest. Then robots run what’s called a dynamic coverage algorithm. That means the robots disperse themselves in such a way that they cover areas of interest where we get the same amount of stuff we’re in charge of. Technically, this is called a voronoi partition. The robots computered a voronoi partition.

Then inside that partition they did their driving around. Then the farmer could do is move their finger around, and on the flight dynamically the robots reconfigured themselves. We took this and this now, which has nothing to do with tractors anymore. But we have this general way of painting event density on areas of interest and then have the robots dynamically respond to that in a distributed manner.

Andrew: Have you received any feedback from either farmers or researchers about some of these advancements?

Magnus: So I have! I did a photo shoot where I was wearing a weird baseball cap and rubber boots in the lab. Don’t make fun of me; I pretended to be a robot farmer. But seriously people are excited about it. The interesting thing about swarm robotics is that it’s up and coming and these kinds of environmental surveillance tasks may be farming or just trying to figure out what’s going on in an area. I had a NASA project where we were monitoring effects of climate change down at Antarctica. We actually never ended up going to Antarctica, unfortunately.

There, you deploy robots for fairly long periods of time and they have to just respond to what’s going on in the environment and this kind of way of painting the world at the user level with areas of interest is a very natural thing. Now if you’re flying two drones, that’s not how to do it, then you need to worry about individual aircraft. If you’re just interested in covering areas then this is a really effective way of doing it. We’ve had quite a lot of funding from the Air Force Office of Scientific Research to pursue this further because in a way it’s a very simple way of describing missions that robots can immediately execute.

Andrew: In closing, I wanted to ask you a philosophical question. Would it be fair to say that you’re a fan of soccer?

Magnus: Ha! Yes, it would be fair to say that I’m a fan of soccer. In fact as soon as we’re done with this I’m going to go put on my soccer gear and go play a soccer game for old men; you have to be over 40 to play in that league.

Andrew: You must be aware of robocup’s robotic soccer league. Currently, they’re pitting teams of robots against each other. Their goal is to field the team of robots to beat the world cup championship human team by 2050. I think that shows a high level of optimism and the future of multi-agents teams and in swarm robotics perhaps as well. What do you see the future as, how will we get there?

Magnus: Robots are already better than people at many things. Robots are really good at not getting bored. They’re really good at responding quickly to things. They’re really good at crunching big numbers. I think if you do a robotic 100 meter dash.

We’re not far away from having bipedal robots that are faster than Usain Bolt, the fastest runner in the world. As a soccer player if you look at the artistry of Messi we’re way far away from that. I think part of that is actually not an algorithmic question. Where we’re lagging right now a lot of times is that actuation and power supplies. The humanoids of today that are not tethered, they have to schlep very large battery packs with them and, if you’re lucky, 3 degrees of freedom foot. Messi’s foot has many more degrees of freedom and he’s small and agile.

Andrew: Perhaps the World Cup is megatronics challenge. What to you would be the swarm robotics coming of age moment where you say that’s the goal?

Magnus: That’s actually a really good question. A big part of what’s driving me is purely intellectual curiosity. I think there are deep scientific questions about how do you get truly emergent behaviors out from simple local rules.

Andrew: What do you mean by an emergent behavior?

Magnus: Simple emergent behavior is how do you send out robots to cover an area where all they’re doing is seeing very locally– what’s going on? A higher level question is that you have these little neurons and all they’re doing is sending spike trains around and, all of a sudden, you have conscious human beings. That’s a very high level and emergent behavior question. Once we understand that to me that’s a really interesting question and it has repercussions way beyond robotics. The big question is not, how do I get 200 robots to mow my lawn more effectively? Even though I do think we’re going to have teams of robotic lawn mowers, small little guys, which are more agile than these big robotic lawn mowers that are out there now.

But, scientifically understand, how do we make these distributed decisions? Along the way we’re going to have robots on Mars. We’re going to have swarms of robots on Mars. We’re going to have robot helpers in hospitals and on the manufacturing floor. These things are coming. Ultimately computers are useful tools, but they’re only useful in as much as we do useful things with them. I think people tend to think of robots as, “Oh my God, we’re going to have an army of terminators!” I hope not. I hope we’re going to use robots to make our lives better. And use them to find energy, food, and travel to remote locations around the globe. I hope that we, as people, will be able to use them as tools for good as opposed to tools for evil … like an army of terminators.

Andrew: A fascinating discussion, thank you for joining us!

Magnus: You bet, thank you for having me!

This audio interview was transcribed and edited for clarity.

Bioinspired robotics #3: Wearables, with Conor Walsh

Wyss Institute — Tue, 01 Dec 2015 21:10:24 +0000

Source: Wyss Institute at Harvard University

Walsh is the founder of the Harvard Biodesign Lab, which brings together researchers from the engineering, industrial design, apparel, clinical and business communities to develop new technologies and translate them to industrial partners. His research focuses on applying disruptive technologies to the development of robotic devices for augmenting and restoring human performance. His current research interests include new approaches to design, manufacture and control of wearable robotic devices and characterizing their performance through biomechanical and physiological studies.

He leads a team of researchers on the DARPA Warrior Web project to develop a soft exosuit that can assist with locomotion that can perform small levels of assistance to a wearer. The exosuit’s function is based on a detailed understanding of human walking and is soft and pliable, unlike traditional exoskeletons that use rigid components. The long term goal is to develop fully portable wearable robots to assist the disabled and able-bodied and further the scientific understanding of how humans interact with such machines.

Source: Wyss Institute at Harvard University

If you liked this article, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

ep.196: Marine Robotics Systems, with Stefan Williams

Ron Vanderkley — Mon, 30 Nov 2015 09:57:00 +0000

Transcript below.

In this episode, Ron Vanderkley speaks with Stefan Williams of the University of Sydney’s Australian Centre for Field Robotics, Marine Systems Group. They discuss the future of Autonomous Underwater Vehicles (AUVs), and a recent expedition where they used multi-session SLAM to map the famous Antikythera Shipwreck (circa 60-80 B.C.), one of the richest ancient wrecks ever discovered. It is located under 55m of water on the NE coast of the island of Antikythera. The site is famous for the first Analog Computer known as the Antikythera Mechanism, a geared device designed to calculate and display celestial information, including phases of the sun and a luni-solar calendar.

The ACFR leads Australia’s Integrated Marine Observing System (IMOS) AUV Facility. IMOS is a nationally coordinated program designed to establish and maintain the research infrastructure required to support Australia’s marine science research. The IMOS AUV facility generates physical and biological observations of benthic variables that cannot be cost-effectively obtained by other means.

Stefan Williams
Stefan Williams is part of the Faculty of Engineering & Information Technologies, School of Aerospace, Mechanical and Mechatronic Engineering, University of Sydney’s Australian Centre for Field Robotics Marine Systems Group.

Links:

Transcript

Ron Vanderkley: Stefan, can I get you to introduce yourself to our podcast listeners?

Stefan Williams: I’m a professor of Marine Robotics at the University of Sydney’s Australian Center for Field Robotics. We undertake both fundamental and applied research in the area of marine robotics and the development of underwater systems.

Ron Vanderkley: What is the main goal of the Marine Group?

Stefan Williams: Our focus over the last decade or so has been to develop tools and techniques for better understanding marine environments. We work in a variety of different application areas, from ecology to archeology and geoscience, and we work in the engineering research areas that underpin these applications – such as vehicle navigation, control planning, machine learning, and computer vision – in order to provide better understanding and information to our end users about the data we are collecting.

Ron Vanderkley: When you talk about autonomous marine surveying systems, what kind of capabilities are we talking about today? Is this a mix of manual and autonomous techniques? Where are the AUVs?

Stefan Williams: The main work that we do is on autonomous underwater vehicles. These are vehicles that are disconnected from the surface. We program them with a particular mission that we’d like them to execute, which might be a big long transect or a particular depth profile, or it might include very detailed studies over a particular feature of interest. Then a vehicle goes down and executes that mission. We typically stay in communication with the vehicle using acoustic modems; the vehicle tells us what it’s doing, it gives us feedback on the data that it’s collecting, and we have some supervisory control to tell it “no, you’re in the wrong spot, please come up” or “there’s some issue with the mission so abort” for example. We also have some ability to redirect the missions while they’re underway.

The data is collected onboard the vehicles. Once we recover them we download the data and build detailed 3D models of these marine habitats primarily using vision data, but we also collect multi-beam sonar, so we have acoustic data as well if we’re looking to map broader areas of the sea floor. That’s the core technology that has been developed around these autonomous platforms, but we’ve also built systems for both shallow water surveys using divers, and also very deep-water exploration using remotely operated vehicles. We also have platforms and sensor packages that we can use in other ways beyond just the autonomous platforms.

Ron Vanderkley: You spoke about the acoustic modems. What level of data can you get out of that? Is it to the level of telemetrics? Can you actually sift some data instead of just downloading it at the end of the mission?

Stefan Williams: Most of the data that we get back at the moment is supervisory in nature. The robot will tell us where it is, how deep it is, how far it is off the bottom, and the amount of data or number of images that it has collected. This lets us monitor the progress of the mission and make sure that things are on track. That modem is also piggybacked on positioning information, so it tells us where the vehicle is as an independent measurement from the ship. We then send both the ship position and the observations of the robot’s position relative to the ship down to the vehicle, and it updates its estimate of where it is using that information. We’re able to correct any drift in the navigation estimate using those external observations.

We’ve just taken on a new positioning and modem package in the past year or so that is capable of higher baud rates. In theory, we can get thumbnails of images. I’ve seen some work where they’re actually using this same sort of acoustic modem to do low-bandwidth video across an acoustic length, but that’s a capability we still have to develop.

Ron Vanderkley: I believe that your group is working on some novel enhancement to acoustic communication from the operator to the robot. Is that to increase the baud rate?

Stefan Williams: We’ve just been awarded an Australian Research Council Linkage Grant, which encourages the research community to engage with interested partners. This is a partnership with Tellus Australia – they do a variety of different developments in the marine space, particularly focused on defense applications. The project is really about managing the communication channels: it’s less about developing new acoustic communications equipment than it is about what information is best to send across the acoustic communication channel so that the operator can most effectively monitor the progress of the mission. In an information theoretic sense, we want to decide what information is best from the point of view of giving the operator an idea of what’s happening, and then allowing that operator to make decisions and query for further information from the vehicle. For instance, it might be used to refine and estimate where particular targets are in an environment, or to request more detailed high-resolution components of a particular area that might be of interest to the operator.

Ron Vanderkley: I also was aware of a multi-session SLAM-based AUV mapping system, and there was an example of a wreck that you worked on. What did that entail?

Stefan Williams: We’ve been involved over the past year in projects together with the Woods Hole Oceanographic Institute and the Hellenic Ministry of Culture and Sport (and a few other organizations) in mapping and excavating a 1st century BC shipwreck off the coast of the Greek island of Antikythera. This wreck was first discovered in 1900 and excavated around that time, yielding a whole trove of statues and other ancient artifacts – including something called the Antikythera Mechanism, which is one of the first examples of what people consider to be an analog computer. It was a geared mechanism that was used for predicting celestial motion over a fairly extended period of time to predict where the sun was and the phases of the moon, etc. It’s a really unique artifact that hasn’t been documented previously. It’s a pretty fascinating site to be exploring.

The objective of our work there was to provide the archeologists with a baseline map before they started further excavation on the site. We first visited the site in September 2014 and conducted surveys over the main target area – as well as looking at other targets of interest that had been identified using acoustic multi-beam data collected in the previous year.

We returned in June 2015 and mapped a very extensive part of the coastline around the wreck site because some of the diving operations they’d conducted in 2014 identified other areas of interest. We delivered a map of their target excavation sites and they were able to use those maps in situ. They actually had copies of the maps on iPads, and were able to annotate these while diving underwater in depths of 50-60 meters. They were able to log all of their finds and situate themselves on the wreck site relative to the maps that we generated. It provided them with a baseline of what the site looked like before they started with the excavation operations.

They’ve just finished their field research and an exhibit has opened in Basel, Switzerland showcasing some of the finds and describing the site and some of the archeological work that’s been going on there. It’s been a really exciting project to be part of and has attracted a lot of interest. Our part of that was using AUV systems to give them baseline high-resolution 3D maps of the wreck site.

Ron Vanderkley: It kind of sounds like “Jacques Cousteau meets robot”

Stefan Williams: In fact Jacques Cousteau visited the site in 1976 and they did some excavation that was part of their Calypso program. They did a lot of underwater exploration and recovered a few artifacts, but this is really the first time that the site has been systematically mapped.
Ron Vanderkley: What is the future of AUVs? Where are the challenges?

Stefan Williams: AUVs have a big role to play in expanding our understanding particularly in deepwater environments. Our main focus has been in coastal environments linked to ecological and biological studies. I think there is a role to play also in understanding how changes in oceanography (for example temperature, salinity, and current profiles) are affecting the marine habitat. We have a big program as part of Australia’s Integrated Marine Observing System Program, running AUVs to look at how key ecological environments around the country are changing over time. We revisit these areas, and work closely with biologists and ecologists to document changes.

Another big role for these autonomous platforms is in understanding deepwater environments – getting them out into the abyssal plains to find out more about what’s down in the deep parts of the oceans, of which we know very little about.

One thing that really excites me is looking at some of the advancements that were made in long-term autonomous deployment. A lot of these autonomous underwater vehicle systems are really starting to come into the norm. There’s some fantastic work going on around the world in designing and building these systems both on manned surface vessels that are capable of very long range operation, and on underwater vehicles that are capable of thousands of kilometers of deployment.

That really changes the dynamics of how ocean science can be conducted. It reduces the requirement of having people on very expensive ships out at sea for long periods of time if you can send these robotic systems off to work autonomously. We’ve seen the precursors in oceanography, with autonomous float systems being used to help oceanographers get a better understanding of ocean circulation and physical oceanography on a global scale. The next revolution will be autonomous platforms that can do mapping and excavation work over long periods of time in deep water across the world. I think there’s real potential in that area.

Ron Vanderkley: What is Australia’s role in that? Are we able to create our own submersibles? Are we concentrating on the programming in that arena? Where do you see us?
Stefan Williams: We have covered both areas natively in Australia. Some of these vehicles are being purpose-designed for particular tasks. We’re currently designing our next generation of hydrolution survey vehicle for supporting the Marine Robotics program, and there are programs in Queensland and Tasmania looking at requirements for working under ice or in complex reef environments.

There is work here in Australia in building these sorts of systems, though there is not a very large commercial scene yet (a lot of the commercial platforms are being developed in the US and Europe). But there’s certainly a lot of research effort focused on designing vehicles that are fit for purpose, and I think we’re particularly well known for the mapping and the navigation work that we do. We have links with all the major oceanographic groups around the world. I think Australia as a whole is very well regarded in robotics and autonomous systems – both within the marine space, and also in ground and air vehicles – and is having a big impact in this area.

Ron Vanderkley: Finally, I’d like to ask you about employment prospects in Australia. If you were a student going to university looking at the options, where would you point them?

Stefan Williams: We have a lot of students come both through undergraduate and postgraduate teaching and research schemes, and the majority of them do find work both domestically and internationally. There’s certainly a lot of activity currently in the robotics space. Some of the big companies like Google, Apple and Uber have made well documented forays into autonomous driving systems, and they’re certainly employing a lot of PhD students, post docs and research fellows who have been employed in various labs around the world, so there’s a lot of potential there. We have former students and staff embedded in the majority of those companies.

Here in Australia we have a domestic market with a number of small startups that are really doing well in developing robotics systems and sensor payloads. The experience students get – and the type of skills that they acquire through their study of mechatronics and robotics – have application across a wide variety of domains, from traditional engineering work, to software, electronics and mechanical design work … all the way through finance and business studies. There’s a lot of interest in data analytics and machine learning, and a lot of the tools and techniques that we apply in robotics can be transferred across a variety of different domains.

One of my most recent students is now heading up a small Machine Learning group at one of the major banks. The skills and underpinning tools required here in our lab to classify marine imagery have direct application into other areas. When students ask me about what jobs are out there for mechatronics engineers, I tell them that there are always good jobs for good students. One of the keys to that is really being passionate about the things that you’re studying, and I think robotics excites a lot of people.

I teach an experimental robotics course – it’s an elective in the 4th year mechatronics programs here at Sydney University. The things that the students do in the course of the 13-week curriculum are astounding. They put a lot of time, effort, and thought into building these robotic systems. I think they feel very rewarded at the end, and the feedback I’ve gotten has always been very positive.
Ron Vanderkley: Thank you, Stefan, for your time. I hope to speak to you again soon.

Stefan Williams: It’s a real pleasure, Ron. Thanks for helping to promote the activities of robotics here in Australia and overseas.

This audio interview was transcribed and edited for clarity.

Talking Machines: Probabilistic programming, with Ben Vigoda

Talking Machines — Thu, 26 Nov 2015 17:36:47 +0000

Image: Gamalon.com

In episode twenty four we talk with Ben Vigoda about his work in probabilistic programming (everything from his thesis, to his new company). Ryan talks about Tensor Flow and Autograd for Torch, some open source tools that have been recently releases. Plus we talk a listener question about the biggestGamalon thing in machine learning this year.

This is the last episode in season one. We’ll be back in early January with season two!

We want to thank all our wonderful listeners for supporting the show, asking us questions, and making season two possible!

ep.195: ICRA 2015 Company Showcase, with Jens Hurley, Michael Ferguson, Simon DiMaio, François Boucher and Andrew Lewis

Audrow Nash — Sat, 14 Nov 2015 11:13:41 +0000

Transcript below

In this episode, Audrow Nash interviews several companies from the International Conferences on Robotics and Automation (ICRA) showcase. The companies span the following applications: mobile robots for military and commercial uses, warehouse solutions, robotic arms and manipulators, and robotic systems to assist surgery.

Companies are listed in the order of their interview with a picture of their booth at ICRA 2015. All photos were taken by Robohub and can be found here.

American Robot Company (AMBOT) specializes in advanced electronics, unmanned systems, autonomous vehicle solutions and intelligent platforms for research, commercial and military robotic applications.

Fetch Robotics provides warehouse solutions.

Intuitive Surgical designs and builds the da Vinci Surgical System for robotically assisted minimally invasive surgery.

Kinova designs and manufactures robotics platforms and components that are simple and safe under two business units: Assistive Robotics for people with disabilities and Service Robotics for humans and robots working in the same environments.

Applied Dexterity seeks to advance the field of robotically assisted surgery by creating a research surgical robot that can serve as a standard platform allowing researchers to share their software and improvements.

Interviewees

Jens Hurley is a Senior Roboticist at the American Robot Company.

Michael Ferguson is a Co-Founder and the Chief Technical Officer (CTO) of Fetch Robotics. Previously, Ferguson was the founder and CTO of Vanadium lab, developing low-cost education and hobby robot controllers, and a Software Engineer at Willow Garage.

Simon DiMaio is a Senior Manager of Research and Advanced Systems Development at Intuitive Surgical. Previously, DiMaio was an Instructor of Radiology and Postdoctoral Research Fellow at Harvard Medical School.

François Boucher is the Vice President of Business Development at Kinova Inc. in Boisbriand, Canada. He received his MBA and a Bachelor degree in Physics Engineering from Laval University. After his graduation, he worked as the General Manager of a technology transfer and investment company before joining Kinova in its early days to develop the service robotics market. In 2014 and 2015, Kinova was listed as one of the 50 most influential public & private companies in the global robotics industry by the Robotics Business Review and one of the fastest-growing company by Profit 500 in Canada.

Andrew Lewis is a Roboticist at Applied Dexterity. Previously, Lewis was a mechanical engineering graduate student in the BioRobotics lab at the University of Washington. His work in the BioRobotics Lab focused on the development of surgical robots, dynamically evaluated gravity compensation, and electromyography controlled iRobot Roombas. Lewis earned his BS in Robotics Engineering from Worcester Polytechnic Institute in 2011. His interests include mechanical design, systems development, and ethics in robotics engineering.

Links:

Transcript

Audrow Nash: Hi, can you introduce yourself.

Jens Hurley: My name is Jens Hurley. I’m a senior roboticist at AMBOT.

Audrow Nash: Can you tell me what you have set up at this booth?

Jens Hurley: Our booth is based on research mobility platforms. We’re doing a lot of ground robotics with a lot of autonomous function built into them.

Audrow Nash:This one right here, can you describe it a bit and then talk about its applications?

Jens Hurley: Absolutely, this guy is a ruggedized outdoor, high-speed, mobility platform. It weighs about 525 pounds. It can carry about 550 pounds, so it’s got a one-to-one payload ratio. It’s got four-wheel, steer independent suspension and four-wheel drive so it’s highly manipulative which makes it great for research applications. It also served in the military for a light and load application and medical evac. It has high-powered lithium ion batteries and a 8-10 hour runtime over difficult terrain.

Audrow Nash: How large is it and how tall?

Jens Hurley: Overall, I believe it’s about 52 inches long, about 42 inches wide and its ground clearance is about eight inches. The whole vehicle itself is about 19 inches from the ground.

Audrow Nash: It’s a large remote operated vehicle?

Jens Hurley: It can be. This particular one is set up for remote, a man in the loop. We also offer systems with autonomous functionality, like soldier following or vision based systems.

Audrow Nash: I see, and so it has these large carbon fibre capsules on its side. What do those do?

Jens Hurley: Those are actually our battery packs. They are quickly removable.

Audrow Nash: Why electric?

Jens Hurley: Silent operation, clean, quiet, overall a very nice function to have. A lot of applications in military require silent functionality.

Audrow Nash: What’s the max distance it could travel? Do you have an idea?

Jens Hurley: No, I couldn’t tell you the max distance. Runtime dependent about seven miles per hour for 10 hours, moving continuously.

Audrow Nash: 70 miles, okay. This vehicle looks like it has a very heavy suspension system.

Jens Hurley: It does.

Audrow Nash: Can you talk a bit about that?

Jens Hurley: Absolutely, it was developed to carry a high payload, heavy weight, under almost any conditions, so it’s got six inches of independent suspension at each corner which makes it contact with the ground. It’s very rugged, very durable.

Audrow Nash: How fast does it go?

Jens Hurley: This particular model has a top speed of about 40 miles per hour.

Audrow Nash: At those speeds why is good suspension valuable?

Jens Hurley: It’s key. If the payload’s too high – high centre of gravity – it’s just not a good thing to have at high speeds. The active suspension keeps the payload low, very stable, it’s physics.

Audrow Nash: Yes. Are there governments interested in this or are there other applications?

Jens Hurley: This particular robot here was developed for a military function, basically, to lighten the load of a soldier. The European space agency is using it for teleop, and universities such like John Hopkins and the Applied Physics Laboratory, are using it.

Audrow Nash: What do you mean by teleop?

Jens Hurley: It’s basically a man in the loop. So they’ll be operating from miles away to perform some sort of action.

Audrow Nash: Okay, so teleop and research applications. What about agriculture?

Jens Hurley: It’s something we’re looking into. There’s been a lot of interest here in agricultural purposes.

Audrow Nash: I see. What’s the purpose of the buttons?

Jens Hurley: It’s pretty standard that every robot comes with an emergency shut off: the red buttons. The rest of these are there for easy setup and control when you get it out of the package. A lot of the customers, when they get it up and going, move the onboard control system because they’re going to be controlling it from a distance.

Audrow Nash: I don’t really see how you put a payload on it, so describe it a little bit more. The wheels are a little bit taller than the body and the battery packs are a little higher than the body. Do you mount some sort of platform on it and then use the payload that way?

Jens Hurley: It depends on the application. A lot of payloads are being used just with manipulative arms and torsos and they do fit on the current payload space. If it’s something larger, like a palette size, we do offer mounting. There are hard mounting points located at each corner of the vehicle where we mount racks to elevate it up and off.

Audrow Nash: What about repair in the field?

Jens Hurley: Depending on what exactly has just broken down, it’s a very simple platform. Everything is open architecture, so it’s easy to get into, easy to repair. It’s a very straightforward, simple machine.

Audrow Nash: Are there standards in creating this? Do you use, for instance, one single size screw for accessing almost everything, or is there one tool to take apart most of it? Is there any modularity built in, so that you can quickly replace components?

Jens Hurley: Definitely. Most of the stuff is done in two metric sizes, M6 and M5, so it’s pretty much built together with the same hardware.

Audrow Nash: Thank you.

Jens Hurley: No problem.

Audrow Nash: Hi, can you introduce yourself.

Michael Ferguson: Yes, I’m Michael Ferguson. I’m the chief technology officer at Fetch Robotics.

Audrow Nash: Can you tell me about what you have set up at your booth?

Michael Ferguson: This week we are showing our new robot platform, Fetch. We’re letting people get up close and see the robot in person. Most people here at this show are particularly interested in the research edition of Fetch.

Audrow Nash: Can you tell me a bit about the robot and describe it.

Michael Ferguson: Sure. So, Fetch is a mobile manipulation platform, it has a mobile base that can drive around with a scanning laser in it to give it a view of the world and keep it localized. It has a torso that can rise up and down and attached to that we have a seven degrees of freedom arm. The gripper is a modularity point. Currently, we have a simple two-finger pincher on here.

Audrow Nash: What do you mean by modularity point, can you swap it?

Michael Ferguson: You can swap it, yeah. We have a standard specification on this, an ISO standard.

Audrow Nash: International Standards Organization?

Michael Ferguson: Yes. There are a number of vendors here at the show with really cool grippers that have, you know, sensors and all sorts of other stuff in them. And many of those will fit on our robot.

And then, lastly, the robot has a pan and tilt head, so that you can look in different directions, that’s carrying an RGB-D sensor. Colour and depth data are in 3D so you can see the whole world around you.

Audrow Nash: It’s a Microsoft Kinect or very similar?

Michael Ferguson: It’s a prime sense sensor, yeah, similar.

Audrow Nash: The robot has a wider base and that’s how it moves around. Then it has the centre part, that has the robotic arm and then above that is the vision, the depth sensor. What’s at the very base?

Michael Ferguson: It’s a scanning laser range finder.

Audrow Nash: Laser range finder? Is it only on a plane?

Michael Ferguson: It is on a plane, yeah. That’s actually manufactured by SIC.

Audrow Nash: Why seven degrees of freedom?

Michael Ferguson: Seven degrees of freedom give you really good dextrous manipulation. If you’re just picking things up off a table top, you probably don’t need a really dextrous arm, but if you want to reach into things, if you’ve got a complex cluttered environment, you really need the extra degrees of freedom there.

Audrow Nash: I see, and so it only has one arm and the arm is at the centre of the robot. Why this configuration? Is this so you can reach things?

Michael Ferguson: Yeah, there are two aspects to it. One is that two arms would make this robot significantly larger, heavier and bulkier. You want to navigate around human environments. We have skinny doors and things like that. The other is that very few applications really require two arms and, in many cases, if you did need a second arm, you just have a second robot come up and help and they can coordinate with each other. As far as the centre, really that’s for stability. There’s a very nice little tuck configuration for the arm that makes the platform extremely stable as the robot’s driving around – so that you can move quickly in the environment and not have to worry about your robot falling over.

Audrow Nash: So will this, primarily, be used for warehouse solutions to pick up products and put them into boxes to be shipped off somewhere? Is that what this has been designed for?

Michael Ferguson: We’re really designing to go into the logistics space, warehouse automation, pick and pack for items. There are videos on our website where you can see the robots doing those sorts of tasks.

Audrow Nash: If I’m not mistaken, the whole robot can rise up. Its spine kind of grows longer and that allows it to reach higher?

Michael Ferguson: Yes.

Audrow Nash: Okay, and how tall can it be?

Michael Ferguson: It gets up to about 1.7 meters or so. The exact specs are on our website. When you extend the arm all the way up you can still have some dextrous manipulability at about six foot.

Audrow Nash: You also mentioned that this could be a research platform. Can you tell me a bit about that?

Michael Ferguson: Yeah, we need a stable robot platform built on a operating system or ROS. We’re now making available our research edition, which has the robot hardware plus the open source side of navigation and some manipulation software, so that university labs can get a real jump-start. They don’t have to build robots by bringing pieces together. They have something that’s designed for mobile manipulation from the get go.

Audrow Nash: Thank you.

Michael Ferguson: Well you’re welcome.

Audrow Nash: Hi, can you introduce yourself.

Simon DiMaio: Sure, my name is Simon DiMaio. I work in our R&D group at Intuitive Surgical.

Audrow Nash: Now, can you tell me a bit about what you have set up at this booth?

Simon DiMaio: Sure, so this is called the Da Vinci Surgical system. It’s a system for minimally invasive surgery, for assisting surgeons to perform in a minimally invasive way. It does that by allowing the surgeons to control instruments inside the patient’s body from what we call a surgeon console. They can get very dextrous manipulation of surgical instruments while minimizing the invasiveness of the procedure for the patients. They fit these instruments through very small incisions in the patient’s abdomen, but still allow the surgeon to have control and dexterity.

To see the field in 3D, we have a stereo endoscope that allows the surgeon to see the inside, with very high quality HD images to control surgical instruments, like graspers and forceps and cutting instruments and electric quarter instruments, in a very natural way, almost as if they were inside the patient.

Audrow Nash: Now can you describe the Da Vinci a little bit.

Simon DiMaio: Absolutely, so there are really two main components in the Da Vinci. There’s the part that’s at the patient and this is where there are robotic manipulators for instruments that are inserted inside the patient. And that is a cart. It’s a mobile cart that carries four robotic arms. Usually, one of those arms carries the camera and the surgeon can control the pointing direction and the view that the camera sees. Then there are three additional arms that are usually used to manipulate instruments. Those can be for cutting, grasping, suturing and things like that.

Then, the other main piece is where the surgeon sits. They sit in an ergonomic console. You can think of it as kind of like a surgeon’s cockpit where they see the images from inside the patient. They see information from the system and they can interact and control by moving some really fancy joysticks, to control the movement of the instruments. They can interact with tissue from the other side of the room. Usually, the console is in the same room, connected to the rest of the system through a cable, but the surgeon controls the system from it.

Audrow Nash: The patient and cart, it kind of looks like a large hang man.

Simon DiMaio: Yeah.

Audrow Nash: It’s high, high above with the arms coming down.

Simon DiMaio: Correct.

Audrow Nash: It has four arms and each has a large tube that is inserted into the patient and that’s where the manipulators are and the camera.

Simon DiMaio: That’s right and that tube is called a trocar or a cannula and that’s the point of entry into the patient’s body. In order to create space for the surgeon to work in, the abdomen, or whichever cavity within the anatomy the surgeon is working on, would be inflated using carbon dioxide gas to create kind of a bubble. Those tubes that you referred to, the trocars or cannulas, have a seal on them, so that the carbon dioxide is trapped inside and maintained and so the instruments pass in and out of the body through that cannula, or trocar, through a seal.

Audrow Nash: I see, and so if they want different attachments, do they remove the inserted part from the arm and then swap in a new one?

Simon DiMaio: That’s correct. Yeah. They’re surgical articulated instruments that have wrists on them to mimic what the surgeon would want to do inside the body. Then there are a variety of end effectors or jaws that the instruments have. Some of them are for grasping tissue, like bowel, liver or kidney. Some of them are for cutting. Some of them are for suturing, so for handling needles. Those instruments are interchangeable on the system so they can be removed and replaced with different instruments depending on the task.

So, if the beginning of the procedure is a dissection, where you’re trying to expose tissues of interest, then you’ve got to remove some tissue and suture things back together. In each of those stages you would potentially use a different combination of instruments. So the system allows you to swap instruments during the procedure very quickly.

Audrow Nash: Is that the purpose of the fourth arm?

Simon DiMaio: The fourth arm could be managing retraction, for example, so it could be holding some tissue out of the way. You might be working underneath a liver or you might be working in an area where there is bowel and you might need to hold the bowel out of the way. It could be what’s called a suction irrigation instrument, that flushes the field with saline and then vacuums it out to manage blood and to keep the field dry and visibility good.
There are a variety of different things that you could use the fourth arm for. It could be used to manipulate an ultrasound probe. If the surgeon is looking at ultrasound images of the tissue and trying to see beyond visible surfaces in the endoscopic image, they might insert an ultrasound probe and manipulate that with the fourth arm. So it’s kind of an auxiliary arm that’s used for many different things.

Audrow Nash: I see.

Simon DiMaio: It can also be controlled from a second console so you can connect two consoles to the robot and have two surgeons.

Audrow Nash: When is that useful? Why?

Simon DiMaio: Training in one instance, if you have a proctor that’s training a mentor, a mentor that’s training a mentee, the proctor might be on one console and the trainee might be on the other. They can virtually swap instruments back and forth. So, a mentor might demonstrate a technique and then give control of the instruments to the trainee who can replicate those motions.

Another instance is if you have two experienced surgeons and they’re collaborating on a complex case, so that you can have all of the arms operating at the same time.

Audrow Nash: I see. How do you manage collisions between the arms? Does that happen?

Simon DiMaio: It does happen, the arm is working in very close proximity. Part of the design was to make the arms as skinny as possible so that they can work closer together. They’re pretty robust, so they can collide and they shouldn’t break.

Audrow Nash: Do they collide during surgery?

Simon DiMaio: They can collide during surgery and usually that happens when the surgeon’s trying to get the instruments very close together through a very narrow entryway. Some of the forces that occur during collisions are fed back into the console, so surgeons can feel some of them on the outside of the robot.

Audrow Nash: So there is some haptics.

Simon DiMaio: So there is some haptics. They don’t feel the very delicate interactions with tissue, but they can feel external collisions between the arms or perhaps with some rigid object. They get some feedback and they stop and investigate. There’s usually an assistant or nursing staff at the patient’s side that can warn them. But it happens quite often, and instruments work very closely together, it’s almost like if we were to stand side by side and our elbows were to knock against each other. It’s not a big deal, you keep going and that’s the nature of working in a confined space.

Audrow Nash: Now, can you tell me about the demo that you have set up at the booth?

Simon DiMaio: So we have a full Da Vinci system here and people are able to test-drive it. We’ve got a mannequin that has a simple plastic environment people can interact with. We’ve got some elastic bands so that they can get a sense of the intuitiveness of the motion of the arms. We also have a virtual reality-based simulator that’s used for training, basically a skills simulator that surgeons can use to learn how to tie knots and suture. They learn how to control the camera and the various instruments, and so are able to repeat the same skills and drills over and over again to develop muscle memory and familiarity with the system in a way that doesn’t require them to operate on patients or animals.

It’s a convenient way for people to use virtual reality models to hone their skills or to warm up before a procedure. So there are some surgeons that use the simulator like an athlete would do before a game. They might warm up and they might do some practice.

Audrow Nash: Interesting. Is there a certification procedure for doctors that are using the Da Vinci system in operations?

Simon DiMaio: There is a training path. I’m less familiar with that side of things but there is some training. I don’t believe that Intuitive directly certifies surgeons for using the system at their hospitals. That’s the responsibility of the hospital. But we provide some training, but we don’t train them on how to perform surgery, that’s something that they learn in med school, but we teach them how to use the tool and then hospitals decide how to certify.

Audrow Nash: What kind of feedback have you received from surgeons on the Da Vinci?

Simon DiMaio: Well, a variety of feedback. Obviously, there’s lots of interest in what they can do with it and what they can offer their patients. It’s a way of allowing surgeons to offer their patients an option for minimally invasive procedure where sometimes that’s not available. We get feedback related to the kinds of procedures that they can perform and the value of those procedures.

We also get feedback on the ergonomics of the system. For them, it’s being able to sit at a console in a fairly comfortable posture versus sometimes working in fairly awkward poses and configurations at the operating room table.
Obviously, there are lots of features that surgeons would like and so we get feedback on things that they’d like to see. It’s a work in progress, so we’re always interested in feedback; things that work well, things that they would like improved.

Audrow Nash: Thank you.

Simon DiMaio: You’re welcome.

Audrow Nash: Hi can you introduce yourself.

François Boucher: François Boucher, VP Business Development for Kinova Robotics.

Audrow Nash: Can you tell me what we have at the booth here?

François Boucher: Lighter, smooth, compact, efficient sets of service robotic arms. We have two different models, JACO and MICO with different ranges and size and weight.

Audrow Nash: First, what do you mean by service robotic arm?

François Boucher: Basically, it’s the opposite of industrial robotics. We’re not extremely fast and extremely precise, but we’re good at working around humans, in collaboration with them or with a human in the loop.

Audrow Nash: That means that they’re compliant to being touched, so if they bash into a human when it’s moving, it won’t hurt the human?

François Boucher: Yeah, exactly, but the arms are on top of all the features that we added: software and sensors. The arm itself weighs only five kilograms, including gripper control, so it won’t hurt anyone at that speed and that weight.

Audrow Nash: Very light, very cool. So how do you make them compliant? Is there a spring in them, like with Baxter?

François Boucher: No, it’s sensors in every actuator.

Audrow Nash: It generates a small electrical signal when you have force on the arm and it’s in the opposite direction?

François Boucher: Exactly.

Audrow Nash: How come you have a different approach? What’s the advantage to doing that over having like a spring in series with the actuator or pillow action?

François Boucher: Well, the fork gives us a safety feature but it also gives us a whole new range of applications, since the arm can now be speech controlled and not only just position controlled. With this whole new field of service robotics, that’s the kind of control that will be needed.

Audrow Nash: You mentioned two robots. What are they? Can you introduce me?

François Boucher: Yeah, it’s JACO and MICO. We started a company for assistive robotics, to mount robots on a wheelchair so that people can eat, drink, open doors, all by themselves. JACO was our first product five years ago.

Audrow Nash: How did you get its name?

François Boucher: It’s the name of the uncle of the founder of the company. He, himself, had muscular dystrophy. So, basically, you’re in a power wheelchair and you have limited motion in your arms. He was an inventor and he developed and built some sort of mechanical arm, with a cable from bicycle brakes and a motor from a windshield washer. It was our inspiration to start the company. His nickname was Jaco, so that’s why the robot’s named after him.

Audrow Nash: I see, and how did MICO get his name?

François Boucher: That’s an easy one. It’s a contraction of Mini and JACO so it turns out to be MICO.

Audrow Nash: Very cool. Can you tell me a bit about the grip and endefactors on these robotic arms?

François Boucher: Yeah, so for the assistive robotics purpose, we had to figure out how we could grip hundreds of different objects of different size and weight. We had to find a really versatile solution. We went with an under actuated gripper.

Audrow Nash: Tell me a bit about how it’s under actuated.

François Boucher: Basically, there’s one actuator per finger and, if you imagine, two tendons links with a spring.

Audrow Nash: I see. Okay.

François Boucher: So the great advantage with this gripper is that they are mechanically intelligent. We don’t have to worry much about which force we’re going to apply, how we’re going to grasp the object, since the flexible gripper adapts to its shape. So, with the precise force needed, we can grab an egg like we can grab a bottle of water.

Audrow Nash: You can grab an egg?

François Boucher: Yeah, exactly.

Audrow Nash: Surprising, that’s great. So, to grab an egg, do you have to tell it that it’s a soft object?

François Boucher: No, it’s really just a great mix of mechanical design and force imitation based on current. Sometimes people try to overdo things. We keep it simple and it works pretty well.

Audrow Nash: That’s interesting. Can you talk a bit more about that or is that IP?

François Boucher: It’s more experience than IP, I would say.

Audrow Nash: That’s kind of mind blowing for me. All right. So some of these don’t have any hands, robotic arms or graspers. What are you demonstrating there?

François Boucher: A lot of our arms are used in maintenance logistics, inspections or replacing a tool, a sensor or a camera. So the arm’s not only used for manipulation, but it can also be combined with a mobile mount platform to perform inspection and probing.

Audrow Nash: I see. Thank you.

Audrow Nash: Hi, could you introduce yourself.

Andrew Lewis: Sure, my name is Andrew Lewis and I’m a roboticist with Applied Dexterity and I’m here representing the Raven II and the Surgical Cockpit.

Audrow Nash: Tell me about the Raven II.

Andrew Lewis: The Raven II is an open source surgical robotics research platform. It was developed here at the University of Washington and then spun out into our start-up company, Applied Dexterity. We’ve helped to ship it around the world. Now there are about 18 or 19 labs around the world using Raven, or something similar.

Audrow Nash: Now, what stands out to me is open source surgical. Can you explain that a bit?

Andrew Lewis: Sure. So all the software is open source. It’s up on GitHub if you want to fork it and play around with it; throw it in your own version of Raven, if you’re that brave. The tooler face is also open source. We have a team, one of the original Raven teams that developed a Da Vinci adapter, and now we can use clinical Da Vinci tools on our robot and have access to their 150 plus tools without having to make them all ourselves.

Audrow Nash: Is most of the value you get from open sourcing this technology in low level parts of the software?

Andrew Lewis: Right. We’ve done all the low level parts and we’ve opened it all up so that researchers can look at anything they want. If they want to implement their own impedance control, or some other control techniques, they’re welcome to dive deep into the PID loops and what have you. You can just use some automation software, with some python scripts in ROS and, since all the source code is available to you, if something’s not working quite right you can change it yourself, or you shoot me an email and I’ll tell you how to change it, or change it for you.

Then, with the common platform, all of the schools or labs that have this robot can really lean on each other for advice on how to approach a problem.
If they’re having technical difficulties, they can check the forums and see that “oh yeah Harvard dealt with it a few years ago and they just unplugged this other thing.”

Audrow Nash: Is it largely research communities that are benefitting from this being open source?
Andrew Lewis: Yeah, so far we’ve distributed just to research labs to help us get in touch with hospitals about using this for research or training. But that would probably be a tighter, ready to roll, don’t-need-to-play-with-the-code version.

Audrow Nash: When we spoke about this a little bit ago, you mentioned that it’s not FDA approved.

Andrew Lewis: Right.

Audrow Nash: Can you talk a bit about what that means in term of its application for surgical, is it just a robot that is used in surgery first of all?

Andrew Lewis: The original version, the Raven I was used for Cholecystectomy, about a decade ago I guess.

Audrow Nash: I don’t know what that is.

Andrew Lewis: It’s gall bladder removal in a pig, and it was successful and I think that’s been the only real life surgery that’s been done with the robot. Harvard has started doing some work with it on time delay and tele-operation.
Since it’s not FDA approved, it means that, when a lab is looking to do some research with the platform that’s ready to go, they’re not paying for that FDA approval. The Da Vinci or another clinical systems is going run into millions of dollars because they’ve had to put time and a lot of effort into getting it approved.

Audrow Nash: That’s the advantage of it, the cost is kept down because it’s not FDA approved.

Andrew Lewis: Right.

Audrow Nash: I see. What’s kind of the goal do you have, where is the company heading?

Andrew Lewis: Our slogan is: “pushing the boundaries of surgical robotics.” The idea is to be able to make this platform available to research labs that are really pushing the boundaries and to support them wherever we can. Then we can make a lot of great connections in all these great labs.

And maybe try to license back some of the technologies that are coming out of the research community. And certainly learn a lot by just being around everyone, so we keep developing either surgical or tele-operated or autonomous systems, and gain huge advantages by being familiar with all these research topics.

Audrow Nash: Thank you.

Andrew Lewis: Sure.

Robots in Depth: Andra Keay on opportunities and challenges for startups

Per Sjöborg — Tue, 10 Nov 2015 14:30:38 +0000

Robots in Depth is a new video series featuring interviews with researchers, entrepreneurs, VC investors, and policy makers in robotics, hosted by Per Sjöborg. In this interview, Andra Keay, director of Silicon Valley Robotics, talks about opportunities and challenges for robotics startups and how developments in the agricultural sector can aid consumer robots.

Andra explains how Silicon Valley Robotics formed and what they do, including organising networking events such as the annual Robot Block Party.

She also shares her views on the areas of robotics where we will see significant developments in the near future, including progress in agricultural robotics that could feed into improvements in other industries, such as home robots.

You can support Robots in Depth on Patreon. Check out all the Robots in Depth videos here.

If you liked this article, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Bioinspired robotics #2: Materials, manufacturing & design, with Robert Wood

Wyss Institute — Mon, 09 Nov 2015 12:42:24 +0000

Source: Wyss Institute at Harvard University

In the Disruptive Podcast series, Terrence McNally speaks directly with Wyss Institute researchers, exploring what motivates them and how they envision our future as might be impacted by their disruptive technologies. In part 2 of the Disruptive: Bioinspired Robotics episode, Wyss Founding Core Faculty Member Robert Wood discusses new manufacturing techniques that are enabling popup and soft robots.

Wood is developing biologically inspired aerial and terrestrial microrobots, soft-bodied robots, and “printable” robots. His current research interests include new micro- and meso-scale manufacturing techniques, fluid mechanics of low Reynolds number flapping wings, control of sensor-limited and computation-limited systems, active soft materials, and morphable soft-bodied robots. He leads a team of over 40 researchers on the National Science Foundation (NSF) “RoboBees” project to develop coordinated colonies of autonomous robotic bees.

His group is also building agile ambulatory robots that are inspired by insects and centipedes. The long-term goal is to create a swarm of robotic insects capable of performing important tasks, such as search and rescue, hazardous environmental explorations, and pollination. Wood is collaborating with a diverse set of researchers at the Wyss who are exploring soft-bodied autonomous robots and soft devices for human-robot interaction and rehabilitation. One of these projects, called “Second Skin,” is a system in which sensing, actuation, and control mechanisms are embedded in soft devices that can be worn by patients with neuromuscular disorders to help them regain function. Wood is also working on novel manufacturing processes for “printable robots” with the goal of automating robot development and creating new methods for rapid prototyping complex electromechanical devices.

Talking Machines: Digital humanities with David Mimno

Talking Machines — Fri, 06 Nov 2015 11:00:58 +0000

David Mimno, assistant professor of information science (ISP).

In episode 23 we talk with David Mimno of Cornell University about his work in the digital humanities, and explore what machine learning can tell us about lady zombie ghosts and huge bodies of literature. Ryan introduces us to probabilistic programming and we take a listener question about knowledge transfer between math and machine learning.

https://soundcloud.com/talkingmachines/probabilistic-programming-and-digital-humanities

ep.194: Embodied Quadrotors, with Davide Scaramuzza

Audrow Nash — Sat, 31 Oct 2015 00:11:18 +0000

Transcript below.

In this interview, Audrow Nash speaks with Davide Scaramuzza, Assistant Professor of Robotics at the University of Zurich and leader of the Robotics and Perception Group, about autonomous unmanned vehicles (UAV) that navigate using only on-board systems—no GPS or motion capture systems.

Below are some videos of Scaramuzza’s research.

Davide Scaramuzza

Davide Scaramuzza (1980, Italian) is Assistant Professor of Robotics at the University of Zurich. He is founder and director of the Robotics and Perception Group, where he develops cutting-edge research on low-latency vision and visually-guided micro aerial vehicles. He received his PhD (2008) in Robotics and Computer Vision at ETH Zurich (with Roland Siegwart). He was Postdoc at both ETH Zurich and the University of Pennsylvania (with Vijay Kumar and Kostas Daniilidis). From 2009 to 2012, he led the European project “sFly”, which introduced the world’s first autonomous navigation of micro quadrotors in GPS-denied environments using vision as the main sensor modality. For his research contributions, he was awarded an ERC Starting Grant (2014), the IEEE Robotics and Automation Early Career Award (2014), a Google Research Award (2014). He coauthored the book “Introduction to Autonomous Mobile Robots” (MIT Press). He is author of the first open-source Omnidirectional Camera Calibration Toolbox for MATLAB, also used at NASA, Bosch, and Daimler. He is also author of the 1-point RANSAC algorithm, an effective and computationally efficient reduction of the standard 5-point RANSAC for visual odometry, when vehicle motion is non-holonomic. He is Associate Editor of the IEEE Transactions of Robotics and has numerous publications in top-ranked robotics and computer vision journals, such as PAMI, IJCV, T-RO, IJRR, JFR, AURO. His hobbies are piano and magic tricks.

Links:

Transcript

Audrow Nash: Hi, can you introduce yourself.

Davide Scaramuzza: Yeah. My name is Davide Scaramuzza and I am an Assistant Professor at the University of Zurich where I lead the robotics and perception group. My group is about three years old now. Before that I worked at ETH and the University of Pennsylvania. In fact, I got my PhD at ETH Zurich with Professor Roland Siegwart, and then I stayed there for another three years as a post-doc and led a European project called sFLY which was the first project to demonstrate autonomous navigation of vision controlled drones without GPS.

Then I moved to the University of Pennsylvania where I worked with Professors Kostas Daniilidis and Vijay Kumar. Then, in 2012, I got my position as Assistant Professor at the University of Zurich.

Audrow Nash: Now, your research, what is the goal, what are you working towards and what’s your motivation?

Davide Scaramuzza: I’m interested in developing autonomous machines, for both air and ground, that use mainly sensors for navigation and perception. I’m particularly interested in vision because I think vision is the most powerful sense for us humans, and for insects, in general. In fact, most of the brain cortex is dedicated to processing visual images. I’m very interested in exploiting image information for navigation, interpretation, reasoning, path planning and so on.

Audrow Nash: I see. Can you talk a bit about the drone research platform that you’re using with these vision systems?

Davide Scaramuzza: Yes. Since 2009 when I started doing this European project, most of my research has been formed around drones and vision control.

Audrow Nash: Are your drones quadcopters?

Davide Scaramuzza: Yes, we use quadrotors. We assemble them from off the shelf components. Actually, we’re very happy with the air drone but we only use the frame, the motors and the motor controllers from it. We replace the rest of the electronics and put in a PX4 autopilot. We run all our control perception planning algorithms on board on an android and send it to the PX4 autopilot, which we rewrote from scratch.

Audrow Nash: This is done entirely on board the quadrotor?

Davide Scaramuzza: This is all done entirely on board.

Audrow Nash: What are some applications of quadrotor drones, eventually?

Davide Scaramuzza: Well, now you can use quadrotors for search and rescue, law enforcement, room inspection, agriculture, even package delivery. Currently, my group is investing in room inspection for nuclear facilities, like CERN or Fukushima, and reactor buildings. We also have an interest in room inspection for bridges and search and rescue operations, after an earthquake for example. All the applications where GPS is not available, basically, because this allows us to focus on computer vision and visual control tools.

Audrow Nash: Turning to drones that are not using computer vision for navigation exclusively, what other modes of drones are there?

Davide Scaramuzza: If you don’t want to let your drone fly autonomously, the only option you have is a good experienced pilot and usually they use either line-of-site or goggles, so we wirelessly stream video. The problem is that, of course, it’s very difficult to control a drone using line-of-site, especially from far away. Even with goggles it’s difficult. Then, of course, at a certain distance the communication drops so there is nothing you can do.

When this happens you need to have activated the drone to be able to continue the exploration autonomously. Now, how do you localize a drone without GPS? The only way is to rely on the motor sensors; SLAM technology which stands for Simultaneous Localization and Mapping. You build a local map with the environment, then you match it with the global marker available from architectural sketches and the robot uses these algorithms to retrieve a position on the global map. Of course, you can use different sensors for SLAM, such as lasers. They’ve been very successful.

In the last 10 years we’ve seen a boom in visual SLAM using only one camera due to the fact that computer technology has progressed a lot, making it possible to run very sophisticated algorithms on a smaller computer like a Smartphone.

Audrow Nash: What are the problems of using vision on quadrotors?

Davide Scaramuzza: If you compare the performances of vision controlled drones with those remotely controlled, you will notice that vision controlled drones are still limited to controlled environments, which means controlled illumination and texture. Because a vision controlled robot needs texture in order to work, to perceive the environment. If there is no texture it will, basically, crash into the wall. One problem is that we are still restricted to a controlled environment.

Another problem is that vision algorithms are still too slow. We have an average latency, generally speaking, of 50 to 200 milliseconds.

Audrow Nash: Is that because of the sensors or the processing?

Davide Scaramuzza: It’s due to both actually. The processing, on average, takes about 50 to 200 milliseconds a frame, depending on the type of algorithms you’re running. Our semi-direct vision odometry algorithms take only 10 milliseconds on an android, but many vision, SLAM algorithms take much longer, like 30 to 50 milliseconds.

We’re also looking at different types of vision sensors, like event cameras that have an update rate of one million hertz, one mega hertz. They have as many pixels but they do not acquire the image all at the same time but asynchronously, in a similar way as the human eye works. The output is not a frame, it’s a sequence of asynchronous events at micro segment time resolution.

Audrow Nash: That moves into semi-direct visual odometry correct?

Davide Scaramuzza: Well, semi-direct visual odometry still uses high standard sensors.

Audrow Nash: It is similar in the process, is it not?

Davide Scaramuzza: What we’re doing with the event-based sensor is different from semi-direct visual odometry. It’s a different sensor so it’s separate. Before I explain SVO – semi-direct visual odometry – I have to tell you how standard visual odometry algorithms work:

We have two types of visual odometry algorithms. There are visual based or direct methods. Visual based approaches usually extract selling points across images and then they match them.

Audrow Nash: Notable features?

Davide Scaramuzza: Notable features like corners. Primitive features. No lines, small edges, equal edges for example. Then, the next step for visual based algorithms is to match corresponding core-ness across images. Then you apply a motion estimation algorithm, which is basically the core visual odometry. Usually these motion estimation algorithms work by minimizing the projection error between the observed notable points, and they project through the points that are available, from the previous frame for example.

The other approach is to use direct methods, or dense methods, that, instead of extracting several marked points from the images, work with all the pixels. The advantage is that, since you use all the pixels, you will have increased accuracy but you cannot minimize the projection error. Since they’re using photometric error minimization, they only work well if the motion baseline is very small between the two frames; you have to make sure that there is not much distance between consecutive frames.

What we invented is called SVO – semi-direct visual odometry – which, as the name says, leverages the advantages of official base approaches and direct approaches. How do we do that? Well, basically, we use direct methods for a small frame-to-frame motion estimation. Then, instead of future base methods for frame to key frame motion estimation when we’re moving, we skip frames that are too close by. When a certain frame is significantly far from a previous key frame, then we consider the new frame as a key frame and we run an adjustment in respect of the previous key frames.

Audrow Nash: What is a key frame?

Davide Scaramuzza: A key frame is basically a standard frame that is significantly far from the previous frame.

Audrow Nash: Okay. What are some of the projects that you’ve done using semi-direct visual odometry?

Davide Scaramuzza: Research wise, so far, SVO is only working on sparse image points. The next step was to densify the maps and we call these methods “dense methods.” Basically, we use every single pixel in the image; we try to track every single pixel and have a frame of an estimate of it and its uncertainty; a probabilistic filter for each pixel.
We made an app you can download from the iTunes store. It’s called 3D around and uses our remote and SVO technology to allow people to do dense reconstruction – of food, at the moment.

Audrow Nash: Interesting. What are some of the research projects that have come out of this?

Davide Scaramuzza: We are collaborating with industry, so we’re collaborating with sense FLY, which is now part of the parent group. They build drones. Initially, they became popular by making six wing airplanes. They’ve just made a new platform, a quadrotor platform, which is the first quadrotor with five vision sensors. Five cameras are looking in five different directions with small overlaps. The front looking camera has a tilt mechanism that allows it to rotate in any direction. Each of these five vision sensors is coupled with the sonar sensor for distance measurements. Basically, we are importing our technology to this platform.

We’re also collaborating with CERN in Geneva, where the famous particle accelerator is.

Audrow Nash: Going back to sense FLY for just a moment. Using your technology, what kind of things would that allow you to do?

Davide Scaramuzza: The main advantage of using our technology is for inspection operations. It will allow anyone with little or no experience to fly a quadrotor and perform simple inspection operations, like lock to a wall, lock to the surface or get a closer view of an object. The quadrotor will automatically, autonomously approach an object in order to get the best image. There are other things that I am not allowed to say because of confidentiality.

Audrow Nash: You were saying about CERN?

Davide Scaramuzza: We have a project where they want to replicate the conditions of the Big Bang. Every time they do an experiment, they produce a lot of particles and a lot of radioactivity, so you can’t send technicians down there during this time and even for a short while after. Often, they have small accidents and have to very quickly find the cause. We’re working on, one day, replacing the technicians with drones.

Currently, we are interested in automatic inspection of the LHC tunnels, as well as the secondary beam areas of CERN, which is basically the building where all the pipes emerge from the main tunnel and are used to run different experiments.

Audrow Nash: I see. You partner with a lot of companies with this research. Can you talk a bit about pairing with industry? Does your experience working with different companies benefit you and them?

Davide Scaramuzza: I’m very interested in helping companies because we do research to serve the community, which ultimately serves humanity. Many researchers complain that we should focus on research and not bother about the applications, which is the duty of the company. I think that you need applications in order to understand what are the right research questions to ask. Actually, what I like a lot about working in collaboration with companies is that they make you think about robustness; achieving robustness through demonstration.

This robustness basically means that the system works. Ideally, you want it to work 100%, while, in research, you’re happy if it functions at 10%, then you can already prove a concept.

Now robustness actually opens up interesting research questions, it makes you think about what causes the system to fail. What can we do to avoid, for example, the system failing with light? You may want to control the environment using lights on board, or changing the camera parameters.

Companies also benefit because they can have PhD students or post-docs who are working part time on the project.

Audrow Nash: What advice do you have for those beginning their research career?

Davide Scaramuzza: My advice is always to read a lot of papers, work simultaneously, code a lot. I think it’s very important to do both; not just work in theory, but also to understand what you’re working with. Usually, I ask my PhD students to do some hands on experience on drone control and computer vision and, at the same time, start reading papers, a lot of papers. You have to read a lot in order not to reinvent the wheel, reading is very important, and hands on experience.

Audrow Nash: Thank you.

Davide Scaramuzza: Thank you. You’re welcome.

Robots in Depth: Michael Rubenstein on the challenge of building 1,024 Kilobots

Per Sjöborg — Fri, 30 Oct 2015 20:36:25 +0000

Robots in Depth is a new video series featuring interviews with researchers, entrepreneurs, VC investors, and policy makers in robotics, hosted by Per Sjöborg. In this interview, Michael Rubenstein describes how he has taken his robotics research from theory into practice by building cheap and small robots — 1024 of them to be exact.

Michael first talks about his PhD research in algorithms for modular self-reconfigurable robots at the University of Southern California, supervised by Wei-Min Shen.

He then shares some of the challenges involved in building 1,024 robots, and how what you learn from actually building robots is different from what you learn from simulating them. This project helped him fine-tune the algorithms from his earlier research.

By working on Kilobot, Michael also learned how to make inexpensive robots that fit the educational market well, and has created robots that can be programmed by school children at robotics summer camps.

The Kilobot work was done in the Self-Organizing Systems Research Group at Harvard University. Michael is now faculty at Northwestern University.

You can support Robots in Depth on Patreon. Check out all the Robots in Depth videos here.

If you liked this article, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Bioinspired robotics #1: Swarm collectives, with Radhika Nagpal

Wyss Institute — Wed, 28 Oct 2015 15:24:28 +0000

Source: Wyss Institute at Harvard University

Nagpal is developing programming paradigms that enable new types of robotic systems to mimic the collective behaviors of living creatures. Inspired by bee swarms and termite colonies, she is applying nature’s approach to decentralized cooperation to engineer new collective systems that meet real-world challenges. Biological systems, from embryos to social insects, depend upon the coordinated behavior of large numbers of individual agents to achieve highly complex outcomes — termites build skyscrapers and embryonic cells develop into healthy babies. Nagpal is building new types of distributed systems — from multi-modular robots and robot swarms, to vast sensor networks — that have similar capabilities.

If you liked this article, you may also be interested in:

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Healthcare robotics for therapy, with Ayanna Howard

International Journal of Advanced Robotic Systems (IJARS) — Tue, 27 Oct 2015 22:13:18 +0000

In this short video lecture, Ayanna M. Howard, Motorola Foundation Professor and Associate Director of Research at the Institute for Robotics and Intelligent Machines (Georgia Institute of Technology), introduces us to the world of healthcare robotics for therapy purposes.

Howard focuses on how to provide highly functional robotics that can be used safely both at home and at the hospital to assist a variety of individuals in terms of age, disabilities, and specific treatments. She also highlights the benefits, characteristics and current developments of robotic systems employed in assisting individuals to improve their movement coordination or function, for example in children with CP who not only need therapy assistance but also a play component to motivate and engage.

To find out more about the challenges of robots in therapy, play therapy implementation and robot monitoring/manipulation of the environment, check out this latest lecture.

Source: Georgia Institute
of Technology

Ayanna Howard received her B.S. in Electrical Engineering from Brown University, her M.S.E.E. from the University of Southern California, and her Ph.D. in Electrical Engineering from the University of Southern California, Los Angeles in 1999. Her area of research is centered around the concept of humanized intelligence, the process of embedding human cognitive capability into the control path of autonomous systems. This work, which addresses issues of autonomous control as well as aspects of interaction with humans and the surrounding environment, has resulted in over 100 peer-reviewed publications in a number of projects from scientific rover navigation in glacier environments to assistive robots for the home. To date, her unique accomplishments have been documented in over 12 featured articles – including being named as one of the world’s top young innovators of 2003 by the prestigious MIT Technology Review journal and in TIME magazine’s “Rise of the Machines” article in 2004.

From 1993-2005, Howard was at NASA’s Jet Propulsion Laboratory, California Institute of Technology, where she led research efforts on various robotic projects utilizing vision, fuzzy logic, and neural network methodologies. In 2005, she joined the School of Electrical and Computer Engineering at Georgia Tech, where she is a member of the systems and controls technical interest group and the founder of the Human-Automation Systems (HumAnS) Laboratory. Howard is also a member of the Center for Robotics and Intelligent Machines at Georgia Tech and currently serves as the robotics Ph.D. program coordinator.

Howard A.M. IJARS Video Series: Healthcare Robotics for Therapy [online video]. International Journal of Advanced Robotic Systems, 2015, 12:V8. DOI: 10.5772/61754

Crowdsourcing in machine learning, with Adam Kalai of Microsoft Research

Talking Machines — Mon, 26 Oct 2015 18:02:08 +0000

In episode twenty two we talk with Adam Kalai of Microsoft Research New England about his work using crowdsourcing in Machine Learning, we take a look at the workshops being presented at NIPS this year, and we take a listener question about changing the number of features your data has.

talk – Robohub

Dan Mantz: Growing Interest in STEM, Soft Skills, and Labor Shortages | Sense Think Act Podcast #21

Maria Telleria: Construction Robots and Working with Unions | Sense Think Act Podcast #20

Dan O’Mara: Turning Robotics Education on its Head | Sense Think Act Podcast #19

Afreez Gan: Open Source Robot Dog, Kickstarter, and Home Robots | Sense Think Act Podcast #18

Dayo McIntosh: A Robot Mixologist to Spark Wellness Talks | Sense Think Act Podcast #17

Helen Greiner: Solar Powered Robotic Weeding | Sense Think Act Podcast #16

Christian Fritz: Full-stack Robotics and Growing an App Marketplace | Sense Think Act Podcast #15

Tim Chung: DARPA Subterranean Challenge | Sense Think Act Podcast #14

Steve Macenski: Open Source Robot Navigation | Sense Think Act Podcast #13

Matt Robinson: Accelerating Industrial Workflows with Open Source | Sense Think Act Podcast #12

CBQ: Commercial-grade Autonomous Mowers, Safety, and Dogfooding | Sense Think Act Podcast #11

Brett Aldrich: State Machines for Complex Robot Behavior | Sense Think Act Podcast #10

Youssef Benmokhtar: Digitization of Touch and Meta AI Partnership | Sense Think Act Podcast #9

Jason Richards: Robotic Food Delivery, Crowdfunding, and Working with Lawmakers | Sense Think Act Podcast #7

Tobias Holmes: Agriculture Robots, Herbicide Resistance, and Education | Sense Think Act Podcast #6

Sense Think Act Pocast: Erik Schluntz

Sense Think Act Podcast: Melonee Wise

Links

Sense Think Act Podcast: Adrian Macneil

Sense Think Act Podcast: Dave Coleman

Sense Think Act Podcast: Ryan Gariepy

Robots that feel by seeing

Design

Data Processing

Applications

How to build a robotics startup: the product idea

Related links

Subscribe to the podcast using any of the following methods

Or watch the video

IEEE RAS Soft Robotics Podcast with Ali Khademhosseini: Biomaterials, soft robotics and the Terasaki Institute

IEEE RAS Soft Robotics Podcast with Hod Lipson: Can we design self-aware robots?

Talking Machines: The long view and learning in person, with John Quinn

Talking Machines: Machine Learning in the Field and Bayesian Baked Goods, with Ernest Mwebaze

Talking Machines: Data Science Africa, with Dina Machuve

Talking Machines: Getting a start in ML and applied AI at Facebook, with Joaquin Quiñonero Candela

Talking Machines: Bias variance dilemma for humans and the arm farm, with Jeff Dean

Talking Machines: Overfitting and asking ecological questions, with Professor (Emeritus) Tom Dietterich

Talking machines: Graphons and “inferencing”

Hosts of Talking Machines: Neil Lawrence and Ryan Adams

Brian Cox presents Science Matters: Machine learning and artificial intelligence

Talking Machines: ANGLICAN and Probabilistic Programming

Talking Machines: Restricted Boltzmann Machines, with Eric Lander

Talking Machines: Generative art and Hamiltonian Monte Carlo, with Doug Eck

Talking Machines: Machine learning and the Flint water crisis, with Jake Abernethy

Talking Machines: Automatic Translation and t-SNE, with Hal Daume

Talking Machines: Fantasizing cats and data numbers, with Iain Murray

ep.210: ICRA 2016 Exhibition (Part 1 of 2), with Scania, PAL Robotics, Husqvarna and AnyBody Technology

Talking Machines: Spark and ICML, with Sinead Williamson

ep.209: INNOROBO 2015 Showcase, with RB 3D, BALYO, Kawada Robotics, Partnering Robotics, and IRT Jules Verne

Talking Machines: Computational learning theory and machine learning for understanding cells, with Aviv Regev

ep.208: Ladybird, with James Underwood

Talking Machines: Sparse Coding and MADBITS, with Clement Farabet

ep.207: Evolutionary Approaches for Flying Robots, with Guido De Croon

Talking Machines: Remembering David MacKay

ep.206: Mecha Monsters, with Silas Adekunle

Talking Machines: Machine learning and society, with Neil Lawrence

ep.205: Hadrian Bricklaying Robot, with Mark Pivac

Talking Machines: Software and statistics for machine learning, with Rob Tibshirani

ep.204: Satellite Assembly in Space, with John Lymer

ep.203: Agilic and PiBot, with Harry Gee

Talking Machines: AI safety and the legacy of Bletchley Park, with Nick Patterson

ep.202: Automation and Employment, with Michael Osborne

ep.201: The Airbus Shopfloor Challenge, with Curtis Carson

Turning the lens on robotics reporting: An interview with New York Times technology reporter John Markoff

ep.200: 200th Episode Special, with Rodney Brooks

Flying robots, with Dario Floreano (Part two)

ep.199: Microrobots for Harvesting Crystals, with Simone Schürle

Interview with IJARS, with Howie Choset and Seth Hutchinson

ep.198: Construction Drilling, with Konrad Fagertun

ep.197: Multi-Agent Systems and Human-Swarm Interaction, with Magnus Egerstedt

Bioinspired robotics #3: Wearables, with Conor Walsh

ep.196: Marine Robotics Systems, with Stefan Williams

Talking Machines: Probabilistic programming, with Ben Vigoda

ep.195: ICRA 2015 Company Showcase, with Jens Hurley, Michael Ferguson, Simon DiMaio, François Boucher and Andrew Lewis

Robots in Depth: Andra Keay on opportunities and challenges for startups

Bioinspired robotics #2: Materials, manufacturing & design, with Robert Wood

Talking Machines: Digital humanities with David Mimno

ep.194: Embodied Quadrotors, with Davide Scaramuzza

Robots in Depth: Michael Rubenstein on the challenge of building 1,024 Kilobots