Nico, Emil, and Moritz founded ReRun with the mission of making powerful visualization tools free and easily accessible for roboticists. Nico and Emil talk about how these powerful tools help debug the complex problem scopes faced by roboticists. Tune in for more.

Nikolaus West
Co-Founder and CEO
Niko is a second-time founder and software engineer with a computer vision background from Stanford. He’s fanatic about bringing great computer vision and robotics products to the physical world.

Emil Ernerfeldt
Co-Founder and CTO
Emil fell in love with coding over 20 years ago and hasn’t looked back since. He’s the creator of egui, an easy-to-use immediate mode GUI in Rust, that we’re using to build Rerun. He brings a strong perspective from the gaming industry, with a focus on great and blazing fast tools.

Links

• ReRun
• Download mp3
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

The ROS Awards are the Oscars of the ROS world! The intention of these awards is to express recognition for contributions to the ROS community and the development of the ROS-based robot industry, and to help those contributions gain awareness.

Conditions

• Selection of the winners is made by anonymous online voting over a period of 2 weeks
• Anybody in the ROS community can vote through the voting enabled website
• Organizers of the awards provide an initial list of 10 possible projects for each category but the list can be increased at any time by anybody during the voting period
• Since the Awards are organized by The Construct none of its products or developers can be voted
• Winners are announced at the ROS Developers Day yearly conference
• New on 2022 edition: Winners of previous editions cannot win again, in order to not concentrate the focus on the same projects all the time. Remember, with these awards, we want to help spread all ROS projects!

Voting

1. Every person can only vote once in each category
2. You cannot change your answers once you have submitted your vote
3. Voting is closed 3 days before the conference, and a list of the finalists per each category is announced in the same week
4. Voters cannot use flaws in the system to influence voting. Any detection of trying to trick the system will disqualify the votes. You can, though, promote your favorite among your networks so others vote for it.

Measures have been taken to prevent as much as possible batch voting from a single person.

Categories

Best ROS Software

The Best ROS Software category comprises any software that runs with ROS. It can be a package published on the ROS.org repo of just a software that uses ROS libraries to produce an input. Open Source and closed source are both valid.

Finalists

1. Ignition Gazebo, by Open Robotics
2. Groot Behavior Tree, by Davide Faconti
3. Webots, by Cyberbotics
4. SMACC2, by Brett Aldrich
5. ros2_control, by several ROS developers
6. PlotJuggler, by Davide Faconti

Winner: Webots, by Cyberbotics

Learn more about the winner in this video:

Best ROS-Based Robot

The Best ROS-Based Robot category includes any robot that runs ROS inside it. They can be robotics products, robots for research, or robots for education. In all cases, they must be running ROS inside.

Finalists

1. Panda robot arm, by Franka Emika
2. TIAGo, by Pal Robotics
3. UR robot arm, by Universal Robots
4. Turtlebot 4, by Clearpath
5. Nanosaur, by Raffaello Bonghi
6. Leo Rover, by Leo Rover

Winner: Nanosaur, by Raffaello Bonghi

Learn more about the winner in this video:

Best ROS Developer

Developers are the ones that create all the ROS software that we love. The Best ROS Developer category allows you to vote for any developer who has contributed to ROS development in one sense or another.

Finalists

1. Francisco Martín
2. Davide Faconti
3. Raffaello Bonghi
4. Brett Aldrich
5. Victor Mayoral Vilches
6. Pradheep Krishna

Winner: Francisco Martín

Learn more about the winner in this video:

Insights from the 2022 Edition

1. This year, the third time we organise the awards, we have increased the total number of votes by 500% So we can say that the winners are a good representation of the feelings of the community.
2. Still this year, the winners of previous editions received many votes. Fortunately, we applied the new rule of not allowing to win previous winners, to provide space for other ROS projects have the focus on the community, and hence help to create a rich ROS ecosystem.

Conclusions

The ROS Awards started in 2020 with a first edition where the winners were some of the best and well-known projects in the ROS world. In this third edition, we have massively increased the number of votes from the previous edition. We expect this award will continue to contribute to the spreading of good ROS projects.

See you again at ROS Awards 2023!

Amit Goel, Director of Product Management for Autonomous Machines at NVIDIA, discusses the new collaboration between Open Robotics and NVIDIA. The collaboration will improve the way ROS and NVIDIA’s line of products such as Isaac SIM and the Jetson line of embedded boards operate together.

NVIDIA’s Isaac SIM lets developers build robust and scalable simulations. Dramatically reducing the costs of capturing real-world data and speeding up development time.

Their Jetson line of embedded boards is core to many robotics architectures, leveraging hardware-optimized chips for machine learning, computer vision, video processing, and more.

The improvements to ROS will allow robotics companies to better utilize the available computational power, while still developing on the robotics-centric platform familiar to many.

For the developers out there, check out the ISAAC ROS Github repository to implement this yourself.

Amit Goel

Amit Goel is Director of Product Management for Autonomous Machines at NVIDIA, where he leads the product development of NVIDIA Jetson, the most advanced platform for AI computing at the edge.

Amit has more than 15 years of experience in the technology industry working in both software and hardware design roles. Prior to joining NVIDIA in 2011, he worked as a senior software engineer at Synopsys, where he developed algorithms for statistical performance modeling of digital designs.

Amit holds a Bachelor of Engineering in electronics and communication from Delhi College of Engineering, a Master of Science in electrical engineering from Arizona State University, and an MBA from the University of California at Berkeley.

Links

• Download mp3 (29.8 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

The ability for humans to generalize their knowledge and experiences to new situations is remarkable, yet poorly understood. For example, imagine a human driver that has only ever driven around their city in clear weather. Even though they never encountered true diversity in driving conditions, they have acquired the fundamental skill of driving, and can adapt reasonably fast to driving in neighboring cities, in rainy or windy weather, or even driving a different car, without much practice nor additional driver’s lessons. While humans excel at adaptation, building intelligent systems with common-sense knowledge and the ability to quickly adapt to new situations is a long-standing problem in artificial intelligence.

A robot trained to perform a given task in a lab environment may not generalize
to other environments, e.g. an environment with moving disco lights, even
though the task itself remains the same.

In recent years, learning both perception and behavioral policies in an end-to-end framework by deep Reinforcement Learning (RL) has been widely successful, and has achieved impressive results such as superhuman performance on Atari games played directly from screen pixels. Although impressive, it has become commonly understood that such policies fail to generalize to even subtle changes in the environment – changes that humans are easily able to adapt to. For this reason, RL has shown limited success beyond the game or environment in which it was originally trained, which presents a significant challenge in deployment of policies trained by RL in our diverse and unstructured real world.

Generalization by Randomization

In applications of RL, practitioners have sought to improve the generalization ability of policies by introducing randomization into the training environment (e.g. a simulation), also known as domain randomization. By randomizing elements of the training environment that are also expected to vary at test-time, it is possible to learn policies that are invariant to certain factors of variation. For autonomous driving, we may for example want our policy to be robust to changes in lighting, weather, and road conditions, as well as car models, nearby buildings, different city layouts, and so forth. While the randomization quickly evolves into an elaborate engineering challenge as more and more factors of variation are considered, the learning problem itself also becomes harder, greatly decreasing the sample efficiency of learning algorithms. It is therefore natural to ask: rather than learning a policy robust to all conceivable environmental changes, can we instead adapt a pre-trained policy to the new environment through interaction?

Left: training in a fixed environment. Right: training with
domain randomization.

Policy Adaptation

A naïve way to adapt a policy to new environments is by fine-tuning parameters using a reward signal. In real-world deployments, however, obtaining a reward signal often requires human feedback or careful engineering, neither of which are scalable solutions.

In recent work from our lab, we show that it is possible to adapt a pre-trained policy to unseen environments, without any reward signal or human supervision. A key insight is that, in the context of many deployments of RL, the fundamental goal of the task remains the same, even though there may be a mismatch in both visuals and underlying dynamics compared to the training environment, e.g. a simulation. When training a policy in simulation and deploying it in the real world (sim2real), there are often differences in dynamics due to imperfections in the simulation, and visual inputs captured by a camera are likely to differ from renderings of the simulation. Hence, the source of these errors often lie in an imperfect world understanding rather than misspecification of the task itself, and an agent’s interactions with a new environment can therefore provide us with valuable information about the disparity between its world understanding and reality.

Illustration of our framework for adaptation. Left: training before
deployment. The RL objective is optimized together with a self-supervised
objective. Right: adaptation during deployment. We optimize only the
self-supervised objective, using observations collected through interaction
with the environment.

To take advantage of this information we turn to the literature of self-supervised learning. We propose PAD, a general framework for adaptation of policies during deployment, by using self-supervision as a proxy for the absent reward signal. A given policy network $\pi$ parameterized by a collection of parameters $\theta$ is split sequentially into an encoder $\pi_{e}$ and a policy head $\pi_{a}$ such that $a_{t} = \pi(s_{t}; \theta) = \pi_{a} (\pi_{e}(s_{t}; \theta_{e}) ;\theta_{a})$ for a state $s_{t}$ and action $a_{t}$ at time $t$. We then let $\pi_{s}$ be a self-supervised task head and similarly let $\pi_{s}$ share the encoder $\pi_{e}$ with the policy head. During training, we optimize a self-supervised objective jointly together with the RL task, where the two tasks share part of a neural network. During deployment, we can no longer assume access to a reward signal and are unable to optimize the RL objective. However, we can still continue to optimize the self-supervised objective using observations collected through interaction with the new environment. At every step in the new environment, we update the policy through self-supervision, using only the most recently collected observation:

$$s_t \sim p(s_t | a_{t-1}, s_{t-1}) \\
\theta_{e}(t) = \theta_{e}(t-1) – \nabla_{\theta_{e}}L(s_{t}; \theta_{s}(t-1), \theta_{e}(t-1))$$

where L is a self-supervised objective. Assuming that gradients of the self-supervised objective are sufficiently correlated with those of the RL objective, any adaptation in the self-supervised task may also influence and correct errors in the perception and decision-making of the policy.

In practice, we use an inverse dynamics model $a_{t} = \pi_{s}( \pi_e(s_{t}), \pi_e(s_{t+1}))$, predicting the action taken in between two consecutive observations. Because an inverse dynamics model connects observations directly to actions, the policy can be adjusted for disparities both in visuals and dynamics (e.g. lighting conditions or friction) between training and test environments, solely through interaction with the new environment.

Adapting policies to the real world

We demonstrate the effectiveness of self-supervised policy adaptation (PAD) by training policies for robotic manipulation tasks in simulation and adapting them to the real world during deployment on a physical robot, taking observations directly from an uncalibrated camera. We evaluate generalization to a real robot environment that resembles the simulation, as well as two more challenging settings: a table cloth with increased friction, and continuously moving disco lights. In the demonstration below, we consider a Soft Actor-Critic (SAC) agent trained with an Inverse Dynamics Model (IDM), with and without the PAD adaptation mechanism.

Transferring a policy from simulation to the real world. SAC+IDM is a
Soft Actor-Critic (SAC) policy trained with an Inverse Dynamics Model (IDM),
and SAC+IDM (PAD) is the same policy but with the addition of policy
adaptation during deployment on the robot.

PAD adapts to changes in both visuals and dynamics, and nearly recovers the original success rate of the simulated environment. Policy adaptation is especially effective when the test environment differs from the training environment in multiple ways, e.g. where both visuals and physical properties such as object dimensionality and friction differ. Because it is often difficult to formally specify the elements that vary between a simulation and the real world, policy adaptation may be a promising alternative to domain randomization techniques in such settings.

Benchmarking generalization

Simulations provide a good platform for more comprehensive evaluation of RL algorithms. Together with PAD, we release DMControl Generalization Benchmark, a new benchmark for generalization in RL based on the DeepMind Control Suite, a popular benchmark for continuous control from images. In the DMControl Generalization Benchmark, agents are trained in a fixed environment and deployed in new environments with e.g. randomized colors or continuously changing video backgrounds. We consider an SAC agent trained with an IDM, with and without adaptation, and compare to CURL, a contrastive method discussed in a previous post. We compare the generalization ability of methods in the visualization below, and generally find that PAD can adapt even in non-stationary environments, a challenging problem setting where non-adaptive methods tend to fail. While CURL is found to generalize no better than the non-adaptive SAC trained with an IDM, agents can still benefit from the training signal that CURL provides during the training phase. Algorithms that learn both during training and deployment, and from multiple training signals, may therefore be preferred.

Generalization to an environment with video background. CURL is a
contrastive method, SAC+IDM is a Soft Actor-Critic (SAC) policy trained
with an Inverse Dynamics Model (IDM), and SAC+IDM (PAD) is the same
policy but with the addition of policy adaptation during deployment.

Summary

Previous work addresses the problem of generalization in RL by randomization, which requires anticipation of environmental changes and is known to not scale well. We formulate an alternative problem setting in vision-based RL: can we instead adapt a pre-trained policy to unseen environments, without any rewards or human feedback? We find that adapting policies through a self-supervised objective – solely from interactions in the new environment – is a promising alternative to domain randomization when the target environment is truly unknown. In the future, we ultimately envision agents that continuously learn and adapt to their surroundings, and are capable of learning both from explicit human feedback and through unsupervised interaction with the environment.

This post is based on the following paper:

• Self-Supervised Policy Adaptation during Deployment
  Nicklas Hansen, Rishabh Jangir, Yu Sun, Guillem Alenyá, Pieter Abbeel, Alexei A. Efros, Lerrel Pinto, Xiaolong Wang
  Ninth International Conference on Learning Representations (ICLR), 2021
  arXiv, Project Website, Code

My coding project is to start building an ethical black box (EBB), or to be more accurate, a module that will allow a software EBB to be incorporated into a robot. Conceptually the EBB is very simple, it is a data logger – the robot equivalent of an aircraft Flight Data Recorder, or an automotive Event Data Recorder. Nearly five years ago I made the case, with Marina Jirotka, that all robots (and AIs) should be fitted with an EBB as standard. Our argument is very simple: without an EBB, it will be more or less impossible to investigate robot accidents, or near-misses, and in a recent paper on Robot Accident Investigation we argue that with the increasing use of social robots accidents are inevitable and will need to be investigated.

Developing and demonstrating the EBB is a foundational part of our 5-year EPSRC funded project RoboTIPS, so it’s great to be doing some hands-on practical research. Something I’ve not done for awhile.

Here is a block diagram showing the EBB and its relationship with a robot controller.

As shown here the data flows from the robot controller to the EBB are strictly one way. The EBB cannot and must not interfere with the operation of the robot. Coding an EBB for a particular robot would be straightforward, but I have set myself a tougher goal: a generic EBB module (i.e. library of functions) that would – with some inevitable customisation – apply to any robot. And I set myself the additional challenge of coding in Python, making use of skills learned from the excellent online Codecademy Python 2 course.

There are two elements of the EBB that must be customised for a particular robot. The first is the data structure used to fetch and save the sensor, actuator and decision data in the diagram above. Here is an example from my first stab at an EBB framework, using the Python dictionary structure:

Whether a dictionary is the best way of doing this I’m not 100% sure, being new to Python (any thoughts from experienced Pythonistas welcome).

The idea is that all robot EBBs will need to define a data structure like this. All must contain the first field “robot”, which names the robot’s type, its version number and serial number. Then the following fields must use keywords from a standard menu, as needed. As shown in this example each keyword is followed by a list of placeholder values – in which the number of values in the list reflects the specification of the actual robot. The ePuck robot, for instance, has 2 motors and 8 infra-red sensors.

The final field in the data structure is “decisionCode”. The values stored in this field would be both robot and applications specific; for the ePuck robot these might be 1 = ‘stop’, 2 = ‘turn left’, 3 = ‘turn right’ and so on. We could add another value for a parameter, so the robot might decide for instance to turn left 40 degrees, so “decisionCode” : [2,40]. We could also add a ‘reason’ field, which would save the high-level reason for the decision, as in “decisionCode” : [2,40,”avoid obstacle right”] noting that the decision field could be a string as shown here, or a numeric code.

As I hope I have shown here the design of this data structure and its fields is at the heart of the EBB.

The second element of the EBB library that must be written for the particular robot and application, is the function which fetches data from the robot

How this function is implemented will vary hugely between robots and robot applications. For our Linux enhanced ePucks with WiFi connections this is likely to be via a TCP/IP client-server, with the server running on the robot, sending data following a request from the client getRobotData(ePuckspec).   For simpler setups in which the EBB module is folded into the robot controller then accessing the required data within getRobotData() should be very straightforward.

The generic part of the EBB module will define the class EBB, with methods for both initialising the EBB and saving a new data record to the EBB. I will cover that in another blog post.

Before closing let me add that it is our intention to publish the specification of the EBB, together with the model EBB code, once it had been fully tested, as open source.

Any comments or feedback would be much appreciated.

Link to the original post here.

Imagine for a moment that a road is used only for a single car and driver. Everything is smooth and wonderful. Then you wake up from that utopian dream and remember that our road networks have multiple cars of varying sizes, from different manufacturers, each with a driver with unique behaviors behind the wheel. We quickly realize that traffic conventions and rules are in place to avoid complete and utter chaos. We believe with increasing robotic use cases in the public domain as we all do see, a similar parallel reality needs to be realized and we propose that RoMi-H, an open-source robot and infrastructure framework that simplifies cross fleet robot collaboration, is the way to achieve this coming reality!

Even before the onset of COVID-19, the number of robots and automation technologies introduced and tested in the healthcare industry has been skyrocketing. Service robots perform an ever increasing and diverse set of tasks; taking on smaller and more sensitive deliveries in some cases and relying more heavily on shared infrastructure such as elevators (lifts), doors, and passageways. No single robotics or automation provider can supply the breadth of solutions required in a modern healthcare facility and no facility can afford to operate siloed systems requiring dedicated infrastructure and operating unique user interfaces. Therein lies the challenge.

ROMI-H allows robots from different vendors to interact with each other as well physical assets in the hospital like elevators. The robots can even avoid gurneys and people.

First announced in July 2018, Robotic Middleware for Healthcare (RoMi-H) is a unique open-source system built on ROS 2 and simulated using the Gazebo simulator. It allows for uniform communication and monitoring across robot platforms, sensors, and enterprise information systems. A brief explanation of this initiative can be found in WIRED Magazine: As Robots Fill the Workplace, They Must Learn to Get Along. In tomorrow’s reality, interoperability must be front and center for every developer, manufacturer, systems integrator, and end-user.

As we have written before:

We need food-delivery robots from one vendor to communicate with drug-delivery robots from another vendor. We need a unified approach to command and control for all the robots in a facility. We need a reliable way to develop and test multi-vendor systems in software simulation prior to deployment. And for it to succeed we need this critical interoperability infrastructure to be open source.

Under the leadership of Singapore’s Centre for Healthcare Assistive and Robotics Technology (CHART) and with collaborators such as IHiS, Hope Technik, GovTech and other solution providers, Open Robotics has been working since 2018 to develop an open-source software solution. Its goal is to realize the potential of a vendor agnostic and interoperable communication system for heterogeneous robots, sensors, and information systems in the healthcare space. To accelerate its development we are encouraging contributions to the open-source codebase to accelerate the development of a robust and sustainable system.

RVIZ Shows each robot’s planned path as well as keep out areas for other robots with ROMI-H

In order to understand the underlying mechanics of RoMi-H, we encourage you to take a look at Programming Multiple Robots with ROS 2. It is being continuously updated and will provide you with a thorough explanation of ROS 2 — upon which RoMi-H is built — and the core Robotics Middleware Framework (RMF) that serves to power RoMi-H. The book also features a tutorial on how one might build a web application that can interface with RoMi-H to create useful applications for robot operators or user-facing tools for the robotics industry.

RoMi-H is able to apply the same software across the different robotic systems while ROS 2 manages the communication and data routing from machine to machine; allowing for real-time, dependable and high-performance data exchanges via a publish-subscribe pattern. Publishers group their messages into different classes and subscribers receive information from the classes of messages they have indicated an interest in. This allows RMF to provide a common platform for integrating heterogeneous robotic systems.

We see the RoMi-H project as a significant step, encouraging an open and integrated approach to robotics development and digitising healthcare. We are looking forward to receiving feedback and contributions from interested parties.

RMF can take a simple map and translate it into a Gazebo simulation. The entire system is powered by ROS 2.

Learn More

A public webinar that introduced RMF and featured a live demonstration took place on 18 August 2020 in the CHART Lab. The presentations and recordings can be viewed here. At the same time, if you are interested in finding out more and viewing the source code, do check out the ROS 2 book and the following repositories:

ROS 2 Book:

• https://osrf.github.io/ros2multirobotbook

Github Repositories:

• RMF: https://github.com/osrf/rmf_core
• Traffic Editor: https://github.com/osrf/traffic_editor
• Free Fleet Management: https://github.com/osrf/free_fleet
• Demo Worlds: https://github.com/osrf/rmf_demos

We would like to acknowledge the Singapore government for their vision and support to start this ambitious research and development project, “Development of Standardised Robotics Middleware Framework – RMF detailed design and common services, large-scale virtual test farm infrastructure, and simulation modelling”. The project is supported by the Ministry of Health (MOH) and the National Robotics Program (NRP).

Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the NR2PO, MOH or other parties.

James Bruton playing his barcode synths

As every other Saturday, I’m bringing you another cool open-source project from James Bruton. Today, how about becoming an experimental musician with your own barcode scanner synthesizer?

I introduced James Bruton in the first post of this focus series on him, where I showed you the Boston Dynamics-inspired open robot dogs projects that consolidated him as one of the top maker on YouTube. As a sort of musician, the barcode synth project I’ve picked for this second post grabbed my attention among the countless videos he’s got on his channel.

To be more specific, the barcode synth consists of two projects. A bit more than a year ago, James showed how to build a four-neck guitar synth with the frets (the place where you put your fingers to play a note on the guitar) being barcodes instead of strings. To play this guitar, you only need a barcode reader connected to an Arduino that converts the data read from the barcodes into a number that represents a MIDI note – which is a digital representation of a musical note based on the MIDI standard. You can then plug the Arduino into a synth or your computer (if you love virtual instruments as much as I do!) to transform the MIDI output into actual sound. Extra features of this guitar included pitch bending and octave shifting buttons. You can access the open-source code of this type of guitar here, and enjoy James’ explanation (and performance) in the following video:

A couple of months ago, James made an improved version of the previous guitar synth. Instead of using the number given by the barcode, for this improved synth he hacked the barcode reader to interpret the barcodes as images so that the output is the raw square wave that it sees. With he help of a Teensy microcontroller to do the digital signal processing and a Raspberry Pi to display barcode images on a screen fitted to a 3D-printed guitar, he could produce a richer range of sounds compared to the previous version. If you want to build you own barcode synth, check out the open-source files and his video (you’ll be impressed to find out how a zebra sounds like!):

Make tech, make music, and stay tuned!

James Bruton with openDog V2

What if you could ride your own giant LEGO electric skateboard, make a synthesizer that you can play with a barcode reader, or build a strong robot dog based on the Boston Dynamics dog robot? Today sees the start of a new series of videos that focuses on James Bruton’s open source robot projects.

James Bruton is a former toy designer, current YouTube maker and general robotics, electrical and mechanical engineer. He has a reputation for building robot dogs and building Iron Man inspired cosplays. He uses 3D printing, CNC and sometimes welding to build all sorts of robotics related creations. Highlights include building Mark Rober’s auto-strike bowling ball and working with Colin Furze to build a life-sized Iron Man Hulkbuster for an official eBay and Marvel promo. He also built a life-sized Bumblebee Transformer for Paramount to promote the release of the Bumblebee movie.

I discovered James’ impressive work in this episode of Ricardo Tellez’s ROS Developers Podcast on The Construct, which I highly recommend. Whether you enjoy getting your hands dirty with CAD files, 3D-printed parts, arduinos, motors and code, or you like learning about the full research & development (R&D) process of a robotics project, you will have loads of hours of fun following this series.

Today I brought one of James’ coolest and most successful open source projects: openDog and its different versions. In James’ own words, “if you want your very own four-legged friend to play fetch with and go on long walks then this is the perfect project for you.” You can access all the CAD files and code here. And without further ado, here’s the full YouTube playlist of the first version of openDog:

James also released another series of videos developing an affordable version of openDog: Mini Robot Dog. This robot is half the size of openDog and its mechanical components and 3D-printed parts are much more cheaper than the former robot without sacrificing compliance. You can see the full development in the playlist below, and access the open source files of version 1 and version 2.

Based on the insight gained through the R&D of openDog, Mini Robot Dog and these test dogs, James built the ultimate robot dog: openDog V2. For this improved version of openDog, he used brushless motors which can be back-driven to increase compliance. And by adding an Inertial Measurements Unit, he improved the balance of the robot. CAD files and code are available here. If you want to find out whether the robot is able to walk, check out the openDog V2 video series:

If you like James Bruton’s project, you can check out his website for more resources, updates and support options. See you in the next post of our focus series!

In this episode, Audrow Nash speaks with Dave Coleman, CEO of PickNik Robotics, about the open source robotics manipulation platform called MoveIt. Coleman talks about MoveIt’s story, from inception and the early days to development and maintenance, as well as how MoveIt relates to the Robot Operating System (ROS) and their move to support ROS-2. He also speaks about MoveIt’s implementation, including global versus local planners and what that means. Coleman concludes by talking about World MoveIt Day and how those interested can begin learning MoveIt and contributing.

Here is a 2017 montage of how MoveIt has been used:

Dave Coleman

Dave Coleman completed his PhD in Computer Science at CU Boulder focusing on motion planning and his B.S. at Geogia Tech in Mechanical Engineering. Coleman has 12 years experience working in the field of robotics automation and is a leader in the open source MoveIt and ROS communities. His insights into robot-agnostic platforms for different morphologies, theoretical approaches, and different end-user requirements give him a well-rounded understanding for powerful robotic software. He has worked and consulted for all types of robotics companies including Google Robotics, Open Robotics, and Willow Garage.

Links

• Download mp3 (17.8 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Lilly Clark interviews Aleksandr Kapitonov, “robot economics” academic society professor at Airalab, on his work for Robonomics Platform, an Ethereum network infrastructure for integrating robots and cyber-physical systems directly into the economy. Kapitonov discusses the advantages of using blockchain, use cases including a fully autonomous vending machine, and the Robonomics technology stack.

Below are two videos showing the Robonomics Platform in action via a fully autonomous robot artist and drones for environmental monitoring.

Aleksandr Kapitonov

Aleksandr Kapitonov is a “robot economics” academic society progressor at Airalab (the team behind Robonomics Platform), an assistant professor of Control Systems and Robotics at ITMO University, and regional coordinator of the Erasmus+ IOT-OPEN.EU project for researching and developing IoT education practices. His research focuses on navigation, computer vision, control of mobile robots and communication for multi-agents systems.

Links

• Download mp3 (17.7 MB)
• Robot Operating System introduction course
• Blockchain course from Linux Foundation
• Robonomics winter school
• Kapitonov’s Plenary speech at Blockchain for Robots Symposium

In this episode, Audrow Nash speaks with Ian Bernstein, Founder and Head of Product at Misty Robotics, about a robotics platform designed for developers called Misty II.  Bernstein discusses the motivation behind making a robotics platform for developers (relating it to personal computers), Misty II’s hardware extensibility and software “skills,” and the future direction of Misty Robotics.

A video introducing the Misty II platform:

Ian Bernstein

Ian Bernstein is Founder and Head of Product at Misty Robotics, a spin-off company from Sphero, Inc. focused on building personal robots for the home and office. In this role, Bernstein leads Misty Robotics’ product development and design. Prior to Misty Robotics, Bernstein served as Founder and Chief Technology Officer at Sphero, Inc. that has shipped more than 3 million robots to date. Bernstein joined TechStars in 2010 with Sphero co-founder Adam Wilson and created Sphero, the original app-enabled robotic ball. Bernstein has had a lifelong passion for robotics and creation and has been building robots since he was 12. Ian holds a BS in Electrical and Electronics Engineering from Colorado State University.

Links

• Download mp3 (13.8 MB)
• Misty Robotics’ homepage
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Audrow Nash and Christina Brester conduct interviews at the 2016 International Association of Science Parks and Areas of Innovation conference in Moscow, Russia. They speak with Roman Luchin, CEO of CyberTech Labs., about a robotics development platform called Trik. Trik is intended to be an intermediate step when learning about robotics between Lego Mindstorms and programming on an embedded platform. Trik allows users to program with a graphical interface by ordering blocks. These blocks contain code in several common programming languages (python, F#, Pascal, etc.) and the code can be modified directly.

This is the second of three interviews from the conference.

Roman Luchin

Roman Luchin is Founder of TRIK and CEO of CyberTech Labs in Russia. He started as a researcher in a laboratory of theoretical cybernetics in Saint Petersburg State University, where he developed multi-agent control systems. Faced with two challenges in Russian robotics, young professionals and outdated approaches, he decided to devote himself to the improvement of STEM education through robotics and the creation of new educational tools. As a result, he developed a unified cybernetic platform for education, research, and robotics projects called TRIK. There is a continuously growing community around TRIK, including universities, researchers, prototyping labs, and creative teams working to provide everyone with the opportunity to create modern intelligent robots that can “see”, “hear” and collaborate.

Links

• Download mp3 (16.5 MB)
• IASP 2016 conference website
• Skolkovo’s robotics page
• Trik’s homepage
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

What do you get when you put together wood and rope? Well according to Plymouth University’s Professor Guido Bugmann: a low-cost, open source, 2 meter tall robot! All buildable for under £2000. The Cheap Arm Project (CHAP) began as an MSc project aimed at developing an affordable mobile robot arm system that could be used by wheelchair users to access daily objects at inaccessible heights or weights (the extreme case being 2 litre bottle).

Now the project has turned open source in the hope of continuing development and gathering more feedback. With the entire hardware and software details available online, with detailed instructions, the team hopes that community members will attempt to construct their own CHAP robot.

While building an entire robot may conventionally seem a complex and expensive undertaking, CHAP stands out with its use of 3D printed parts and wood. In fact, the “under £2000” price tag of required parts is primarily made up of the use six Dynamixel motors, each costing up to £200 and used by the project due to having numerous available during development. With this in mind, future work will look at the use of lower cost motors.

Rather than a robot ‘brain’, CHAP is powered by a human one. To accomplish this, CHAP’s Raspberry Pi was used to host a local web page to enable tele-operation from any connected device, all accessible through a WiFi hotspot also hosted by the Pi. This was an important design choice as it removed the need for platform specific apps and associated costs, simultaneously simplifying deployment and testing.

This was done deliberately to keep the design simple, and to allow users the freedom to interact with their world. Early user-tests discovered that some level of intelligence would be needed to simplify controls. With currently three joints, and three axis available; some steps are needed to automate object interaction (pull requests welcome!).

The next step for CHAP will be refinement of hardware and software; however the very first will be building a community! With full design documents available it is hoped that with support these can be improved and simplified, while simultaneously improving the effectiveness of the software.

Link to the github site can be found here: https://mobile-chap.github.io/Web/

The NAIST OpenHand M2S was developed by a team of students as part of the school’s annual CICP project (read the blog post about it here), in which students can propose and organize their own research projects. Based on the Yale OpenHand M2, the NAIST OpenHand M2S was developed for textile manipulation, sensitive grasping as well as pinpoint pushing with high loads. All of the parts (except motors, sensors etc.) can be downloaded from here and 3D printed.

For grasping and manipulation, the hand is equipped with two 3D force sensors that act as fingertips. Using these, the hand can gently grasp textiles and let them glide through its fingers, pull them taut when tension is required, and even recognize different materials. By rubbing its fingertips together, a force signal is generated which allows the hand to sense if it successfully grasped a textile, and which kind of material it is.

Lastly, the rigid finger of the hand allows it to push with high loads and tuck into small openings. This can be used for tasks like making a bed, but also during the manufacturing of a car or airplane seat, which is part of the team’s research. Future versions of the hand will be able to pick up thin objects from a table easily without moving the robot.

You can read the conference paper here. Download the 3D files here, to use the gripper in your own research.

If you liked this article, you may also enjoy these:

• Robotics, maths, python: A fledgling computer scientist’s guide to inverse kinematics
• Python programming your NAO robot
• Soft robotic gripper can pick up and identify wide array of objects
• Sticky business: Five adhesives tested for 3D printing
• 3-D printing hydraulically-powered robots, no assembly required

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Tutorial 1: Speech

Helpful hints:

Follow these steps to create a box that we could put our python code into, and have our robot speak: “Hello”. “Hello I am a robot”.

1. Right click the main section box and go to “add new box”
2. Create a name for the box
3. Input main image
4. Select the type of box

Now we’re having a look at the script. Go into the scripts and click ok. And there we have our NAO little box. If we click into it, we have the python code. What we are going to do is change some of the codes to get our robot to speak.

1. Double click the box
2. Select the piece of code you need to alter to get our robot to speak(“def onInput_onStart(self)”)
3. Take out the pass section
4. Type in “ttsproxy = ALProxy(“ALTextToSpeech”)”. (Make sure you get the comment in there).
5. Underneath that, we are going to put “ttsproxy.say” and add the comment “Hello, I am a robot”.

The first line could create an object that gives us access to the robot’s text to speech capabilities. This object is assigned to the variable ttsProxy. We can access the object later through the name ttsProxy. I am going to mount the top line, and the alspeech is there. The second line calls a different function site. This function takes an argument “Hello, I am a robot” and then he uses a string or sequence of characters between two double quoted marks. The robot will speak the string that will pass to the say message aloud.  There are your simple first steps into python and choreograph.

You can now, hopefully, get your NAO robot to talk by using your python programming skills. Now have a play with it’s behavior to see if you can get your NAO to say a longer sentence, or maybe even sing a song. Have a play, and use your imagination!

Tutorial 2: Walking

Helpful hints:

No we’re going get your robot to walk using Python. The NAO can be program to walk to any point using the in-build Python Programming Language instead of using the drag and drop boxes in the choregraphe. Python is more expressive and allows you to do things like computing trigonometric calculations on the robot.

1. Right click the main section box and go to “add new box”
2. Create a name for the box
3. Input main image
4. Select the type of box

First, we want our robot to stand up. Drag a stand up icon into the box. I am actually using the NAO key so the 3D robot in the robot view, and this because we need to put the robot on connecting so we can see what’s going on quite clearly. Oversee when you are using this on the real robot and never forget to trigger on the motors and connect everything up properly.

Open the Code Box

1. Go to “def onInput_onStart(self):”
2. Write “motionProxy = ALProxy(“ALMotion”)”
3. On the next line write “walkTo(0.2,0.0,0.0)”
4. End it with “self.onStopped()”

The first line is a proxy called ALmotion, which what is we created here, which allows us to call the motion functions up. The second line is the walkTo, which moves the robot to a specific distance. Now, look on the decimal places: see that .2 is X and Y and the radius, so he’s going to move .2 forward. Then if I start playing with the other numbers here, he’ll move to the Y radius and then to the other turning point as well. The final bit is the self.on. To stop this, we will call and tell the choregraphe to stop the procedure when it is at the end and just simple loop and loop and loop.

If we are good, we should see this now on the 3D NAO. So if I hit play, we can see it walking along. Well done guys, you can now get your NAO Robot to walk by using
your Python programming skills. Tread carefully!

If you liked this article, you may also enjoy these:

• Robotics, maths, python: A fledgling computer scientist’s guide to inverse kinematics
• Sticky business: Five adhesives tested for 3D printing
• Meet RoboGabby: She’s on a mission to teach more girls about robotics
• Adjust how you learn and quickly pick up robotics programming
• 3-D printing hydraulically-powered robots, no assembly required

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

This year marks the occasion of ROS turning 9 years old! Over the years, ROS has grown into a strong world-wide community. It’s a community with a large variety of interests: from academic researchers to robotic product developers, as well as the many robot users. Academic use of ROS continues to grow. Citations of the first ROS paper “ROS: An Open-Source Robot Operating System” has grown to 2,871.

To get a better sense of what’s happening in the ROS community, if you have not already done so, I highly recommend reviewing the ROSCon 2016 program. You can also find all the video recordings in this gallery. ROSCon 2016 was another great event bringing ROS community members together to share how they’re using ROS to solve their challenges. As the goal of ROSCon is to share information between the entire community, we record the talks and make them available online. We’ve sold out our venues the last two years and are looking forward to another ROSCon next fall!

Part of understanding our growing community is to try to measure it. For the last 6 years we’ve been generating metrics reports. These reports can give a sense of aggregate concerning what’s happening in the ROS community. Our most recent report is from July 2016. David Lu has put together plots of several of the metrics across the last 6 years which can be quite informative.

This year we wanted to dig a little deeper into the code metrics, so we downloaded the source of all of packages listed in the Indigo Igloo rosdistro and ran some analysis.

• The total line count is over 14 million lines of code
• There have been 2477 authors
• And 181509 commits
• Averaging 73.3 commits per author

You can see the commits as a function of month in this graph.

Our committers are active around the world as evidenced by the commits coming in at all hours of the day.

And the git commits record 24 different time zones (out of 39 possible).

Analyzing the repository for significant lines of code using [SLOCCount](http://www.dwheeler.com/sloccount/) shows:

• 4,077,199 significant lines of code.
• This represents an estimated 1,236 person-years of development.
• For a sense of scale, that is an average of 137 developers contributing full time over the last 9 years!

For those of you curious about the breakdown by language lines of code, it is as follows:

• cpp: 2608592 (63.98%)
• python: 553332 (13.57%)
• ansic: 297629 (7.30%)
• xml: 280615 (6.88%)
• lisp: 149439 (3.67%)
• java: 135343 (3.32%)
• ruby: 26484 (0.65%)
• sh: 21120 (0.52%)

This only represents the packages publicly released into the Indigo rosdistro index.

Note that the tools only worked on Git repos, so code from other source control systems was excluded. There are also a few projects which predate ROS but have ported to use ROS and their history is included.

We’re looking forward to continuing growth through 2017 leading up to the 10-year anniversary of ROS. With the Beta 1 version of ROS 2.0 out, there will be space for new development. We’re looking forward to our next release, Lunar Loggerhead, to coincide with Ubuntu’s next release, Zesty Zapus. With both of these, the ROS community can continue to rely on the many libraries, tools, and capabilities they have come to know and enjoy, as well as begin to experiment with the new features in ROS 2.0

Another exciting project to watch is the upcoming TurtleBot 3! The TurtleBot and TurtleBot 2 have been great platforms for learning and prototyping. However by packing that same capability into a smaller platform with more punch, we look forward to it providing another avenue to grow the ROS community.

We write these anniversary posts to help give you a sense of how ROS has been doing over the past year, but we’d certainly encourage you to find out for yourself. Get involved. Write or edit a wiki page. Answer a question on ROS Answers. Come to ROSCon. And, when you’re ready, think about helping to maintain ROS itself, or even contributing a brand new ROS package.

OSRF is doing great, but the long-term success of ROS depends on every member of the incredibly awesome ROS community. If you’re already an active part of the ROS community, we can’t thank you enough; and if you’re not, think about how you can help ROS grow and thrive for the next nine years, and beyond.

If you enjoyed this article, you may also also enjoy:

• Introducing H-ROS: the Hardware Robot Operating System
• #IROS2016 in pictures
• A practical look at latency in robotics: The importance of metrics and operating systems

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Robotics is becoming more accessible for many people, but the complexities of legged robots mean they remain beyond the reach of most consumers. The complex mechanics, electronics and code algorithms make these robots much more difficult to simply get working. Four legged robots are especially difficult, they might seem very stable but simply lifting a leg up can cause the robot to fall.

QuadBot is an open-source, 3D printed robot that aims to change this.

QuadBot is suitable for makers of any skill level, it works straight out the box, and can be programmed using scratch based blocks. And when you’re ready to explore robotics further, every aspect of QuadBot can be changed, from the mechanical design to individual lines of Arduino code. Once you’ve mastered walking, you can add sensors to the board, and make QuadBot avoid obstacles, follow light and even dance to music!

QuadBot was released on Kickstarter and comes either as a bare bones (3D Print it yourself) kit, or as a full kit. Within the first week, QuadBot reached 100% of the £12,500 goal, and is nearly at £20,000 at the time of writing!

QuadBot is the creation of Jack Scott-Reeve and Josh Elijah, two graduates of Engineering who felt that Robotics Education needed a change. Now that it is funded (and with 5 weeks still left on the campaign), they can focus on improving the product for our early backers. This means adding goals like extra colours, sensor packs and improved learning resources. However, among many Kickstarter campaigns there’s a tendency to overstretch and under deliver, so Elijah and Reeve are letting the product sell on its base features rather than promising massive additions to the design.

Nevertheless, QuadBot has massive future potential for expansion and increased capability, whether using modular design to create QuadBot sensor modules or integrating a Raspberry Pi onto the board to allow Python coding and vision processing.

One emerging market for this technology is robotics in schools. Teaching programming is becoming more common in UK schools, as are products to augment learning like the BBC Microbit. QuadBot could potentially be integrated with existing products like Microbit and the Raspberry Pi, to bring practical robotics into schools. With large distributors like RS Components interested, we could soon see QuadBot being distributed internationally, and with a curriculum to go along with it.

At present it’s hard to say in what direction Elijah and Reeve will go with QuadBot but, with a fully funded Kickstarter already, it’s likely we’ll see many new iterations and designs in the near future!

Click here to visit the QuadBot website.

If you enjoyed this article, you might also be interested in:

• Design, simulate and build a custom drone
• A new quadrupedal robot, the ANYmal
• Make your own Dtto Modular Robot – Assembly instructions (Part 1 of 3)
• mROBerTO: The modular millirobot for swarm behavior studies

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Eight years ago, Morgan Quigley, Eric Berger and Andrew Ng published a paper that was not about ROS. It was about STAIR, the STanford Artificial Intelligence Robot, which used a library called Switchyard to pass messages between software modules to perform complex manipulation tasks like stapler grasping. Switchyard was a purpose-built framework that was designed to be modular and robot-independent, and it was such a good idea that in 2009, “ROS: An Open-Source Robot Operating System” was presented at the IEEE International Conference on Robotics and Automation (ICRA) in Japan. As of this month, the paper introducing ROS has been cited 2,020 times, an increase of more than 50% over last year.

The popularity of one single paper is only a minor indicator of the popularity of the robot operating system that it introduced. At eight years old, ROS is growing faster than ever, and helping the robotics community to grow along with it. We’re especially excited to see how brand new startups have been taking advantage of the open source nature of ROS to help them develop useful, reliable robots that are creating entirely new markets. In 2015 alone, more than $150 million in VC funding (that we know of) was invested in businesses that use ROS.

Large, established companies have been taking more and more notice of ROS as well. At ROSCon this year, Fetch Robotics was joined as a platinum sponsor by Ubuntu, and a record number of gold sponsors included NVIDIA, Bosch, and Qualcomm and attendees from companies such as BMW, DJI, Intel and more. ROSCon 2015 was by far the largest conference we’ve ever had: it sold out weeks in advance. Clearly next year we’re going to have to find a much bigger venue to make room for more attendees, more speakers, and more exhibitors.

Taking a look at how much our community has grown this year, it’s easy to see why ROSCon has become so popular: it’s a reflection of the enthusiasm and engagement of the ROS user base. In May 2015 alone, nearly nine million ROS packages were downloaded from over 70,000 unique IP addresses, and these numbers don’t even count mirrors. This suggests that ROS probably has hundreds of thousands of active users. We also have a very robust developer community: 1,840 people have contributed to ROS’ 10 million lines of code, averaging 20 commits per day. The ROS wiki has gotten 10% bigger since last year, and there are over 11,000 users on ROS Answers, a 32% increase over last year, with a total of more than 5,000 questions answered. It’s numbers like these that make us so confident in the long term future of ROS.

Counting ROS

Because of the nature of the ROS license, we actually don’t know how many users, robots, and developers there are utilizing ROS. Many of the numbers that we are citing throughout are likely to be much larger. For example, we specifically know of approximately 80 types of robots using ROS, but almost every day we hear about new ones. And not every company using ROS discloses so publicly, so our estimates on venture capital investment can be better characterized as lower bounds than estimates.

If you’re not part of the ROS community yet, there’s never been a better time to get involved. Even if you don’t have experience with robots or programming, there’s a wide variety of low-cost robots and helpful online tutorials that can get you started, and we’re also delighted to announce (just in time for the holidays!) that O’Reilly Media has published “Programming Robots with ROS: A Practical Introduction to the Robot Operating System,” by Morgan Quigley, Brian Gerkey, and Bill Smart, which will take you from zero to ROS expert in just 448 pages.

Learning ROS will allow you to do all kinds of cool stuff with more than 80 robotic platforms. You can choose from the capable, affordable TurtleBot, one of the many sophisticated humanoid robots that competed in the DARPA Robotics Challenge, or even NASA’s Robonaut, currently undergoing testing on the International Space Station. Robots powered by ROS are everywhere, and here are just a few of them:

For full frame clips see the long version of the montage.

Of course, we have no idea how many robots are actually running ROS, or how many people or companies are using it, because ROS is open source and completely free. We’re often surprised to learn that cutting edge robots that we’re already familiar with are powered by ROS, as when BMW announced at ROSCon that they’ve been using ROS in their autonomous cars for the past few years. We weren’t at all surprised to hear why BMW chose ROS for its autonomous driving research though: they appreciate its popularity, its stability and reliability based on a large user base, the fact that it makes it easy to collaborate, and its open source nature.

As the ROS community has grown, various special interest groups have organized to promote ROS for specific application domains. The ROS Industrial Consortium is one such group. ROS-I is a software library that builds on ROS and leverages its power and flexibility to control manufacturing automation equipment including industrial robot arms. It is supported by the 36-member organizations comprised of companies such as 3M, ABB, Boeing, Ford, Boeing, Siemens and more. Representatives from Boeing, Caterpillar, Yaskawa and more speak on behalf of ROS and ROS-I in this video.

2016 is poised to be the biggest year ever for ROS, and we’d like to highlight two things that are worth getting particularly excited about. The first is ROS 2.0, which we’ve been developing for the past few years. ROS 2 will support the growth of the ROS community by making it much easier to work with small embedded systems, teams of multiple robots, and robots that require real-time control. We’d also like to make sure you’re familiar with Robotics Fast Track (RFT), which is  a program that we’re working on with DARPA. It’s an easy way for you to get government funding for your awesome robotics ideas without having to give up any of your IP, and absolutely anyone can apply.

We write these anniversary posts to help give you a sense of how ROS has a whole has been doing over the past year, but we’d certainly encourage you to find out for yourself, by getting involved. Write or edit a Wiki page. Answer a question on ROS Answers. Come to ROSCon. And, when you’re ready, think about helping to maintain ROS itself, or even contributing a brand new ROS package. OSRF is doing great, but the long-term success of ROS depends on all of the incredibly awesome ROS users themselves. If you’re already an active part of the ROS community, we can’t thank you enough, and if you’re not, think about it: you can help ROS grow and thrive for eight more years, and beyond.

Foote explains the benefits of open source and how it helps make research and development more efficient when both researchers and start-ups can focus on their added value, rather than starting from scratch. This is an advantage for the whole robotics community, enabling it to move forward much faster. 

Foote also describes how developers can contribute directly to further developing the ROS standard, for the benefit of the robotics community. 

You can support Robots in Depth on Patreon. Check out all the Robots in Depth videos here.

Launched in 2009 by a group of Waterloo engineering students, Clearpath‘s unmistakeable bright yellow and black robots have become synonymous with unmanned vehicle research in university research labs around the world. Now, as the field of robotics matures, Clearpath is forging into industrial applications, too. We caught up with CTO Ryan Gariepy at the 2015 Field and Service Robotics conference (FSR), to talk about their roots in research, the role of ROS and open source in their business model, and the challenges and opportunities of launching a robotics startup in Canada.

[HS] You got your start in the business of robotics while you were still a student at Waterloo. What made you decide to launch a robotics company at that time? Did you have a specific project or technology you were developing that made you think, “Hey, this could actually be useful out there!” …? Or was it that you saw the potential in the field more generally?

[RG] We started Clearpath right out of university in 2009 as we finished our undergraduate degrees – I did my Masters at the same time as we were getting off the ground, which is generally regarded to be a bad idea in terms of stress levels. It’s hard to believe that we just celebrated our sixth anniversary this year.

Starting the company was less about the technology and more about the potential … we just looked at everything around us and said, “Why do we have to shovel snow? This can be automated. Why do we have to do any work that does not require human creativity?” There are a lot of dangerous jobs out there that most people don’t realize are performed on a daily basis by tens of thousands of people in order to keep society going, like mining, farming, construction, stocking warehouses. And yet we need fuel and food and buildings, and we want our same-day deliveries from the Internet. There is a demand for this kind of work to take place.

Around the time we were starting out, Google Maps was getting off the ground. The concept of having information available at your fingertips was also becoming compelling and valuable to people, and the best way to get information about the world on a constant basis is to involve robotics or automation in some form. At the same time, sensors and the cost of sensing were improving due to the rapid uptake in smart phones; we were seeing advances in laptops, smartphones, and electric and hybrid vehicles driving down costs and improving battery density.

People now accept that robotics, automation, and intelligent systems can actually solve some of their problems.

All of this was coming together and we felt that autonomous robotics was possible at a relatively high reliability and low cost, especially when compared with what had been done before.

What would you say have been the biggest changes that you’ve seen in the field of robotics since then?

I think one of the biggest changes is simply that people now accept that robotics, automation, and intelligent systems can actually solve some of their problems.

This recognition means that there are a lot of opportunities out there, both in terms of hardware and software. In some industries – ones that have been around for a long time – they already know how to build solid hardware that could be made robotic, but they need the right parts and the right software to automate it. Then there are other industries that need full solutions. There are a lot of pieces starting to come together now.

Your early customer base was made up of mainly universities and research labs. Do you find yourself getting pulled into a different direction now that you’re expanding more into industry?

The research market is definitely maturing and our product development cycle within that side of the business is very well understood. We’ve released two completely different platforms in the last eight months alone.

Now we are in a position where we can leverage the experience we gained from the R&D market to address challenges in industry. We are actively pushing the development of full robotic solutions (including software), and will soon be releasing a new platform along these lines.

How do you manage the problem of building in more than one direction at the same time?

One of our biggest challenges is that there is no one single clear challenge, especially at the scale we’re at right now. At this stage there are many things that need to be focused on: sales, operations, quality assurance, new features, understanding where the market’s going, product management, marketing … so the challenge is more about managing the priorities of any given department.

How do I provide guidance to the team so that they can spend their time on the right tasks? How do we remain dynamic as a startup? How do we remain flexible enough to change direction quickly if we need to, but still be stable enough to stick to our core values? These are the kinds of questions we are dealing with.

Speaking of core values, Clearpath took a chance from the beginning by focusing on open source and ROS. Does open source make for a good business model?

Why open source? No matter how large a company we become, there will always be more smart people who can build and program robots outside our company than within it.

One of the reasons we looked at open source was the philosophy that no matter how large a company we become, there will always be more smart people who can build and program robots outside our company than within it. So it’s only logical that we work with whatever the community ends up creating around open source robotics, both to reinforce the community, and to hire from it.

That said, from a business model perspective it is more complicated. There are many companies that succeed with open source, but there is still no clear-cut model for building an open source company – and that tends to make people nervous. And yet by now we’ve seen several companies build on open source, including Google and Facebook, and it’s the same thing as building a startup on a LAMP stack: yes, you use open source and you may push back some code to open source, but you still have value you’re bringing on your own.

Earlier on we sold only hardware, which is easier to protect. Now that we’re getting into software sales, open source creates additional considerations for us, but it still hasn’t been an issue. Building on open source makes us more effective, and it certainly helps us engage much more with the robotics community.

Now that you’ve done all the legwork of developing the product and proving the market, do you have any concerns that another company with more resources could just come along and say, “We know who their buyers are, and we know what their markets are, and we can imitate their products …”?

Clearpath is sponsoring the 2015 Robot Launch “Open Source Robotics” Award, offering 10 hours of ROS consulting services to the winner.

ROS is open source and we contribute a lot back to it. We test a lot, we support people, we contribute drivers, and we contribute utilities back. However, the work that we do on autonomy and in controls –planning, mapping localization, interfacing, vehicle management – is not open source. We may open source parts of it in the future as we advance our state of the art, but the ‘secret sauce’ is still closed, even though it’s built on ROS middleware.

Could it make life easier for our competitors in some cases? Yes, but by the same token nobody makes money on having a better robot-logging framework, for example, and everybody saves money by having a good free robot-logging framework. If a better logging framework is your only competitive advantage, it’s not really an advantage, is it? Personally, I’d rather not spend money on a logging framework, and spend more to compete on the actual value that we bring to our clients.

You mentioned snow shoveling as one of your inspirations, which is a stereotypically Canadian problem. Did the Canadian climate and landscape figure into your business plan from the beginning? Is that why you wound up in field and service robotics, as opposed to some other application area?

Of course our robots can survive the winter – we’re Canadian!

One of our competitive advantages is the terrible weather we have in Canada. We have harsh winters, we have a lot of forestry, a lot of primary resources to mine and harvest, and we have a lot of space that is sparsely populated, which makes it easier to do testing. We also happen to have a very well developed technology ecosystem here. When you put those things together, it’s natural to lean towards field robotics.

When we started there were very few platforms out there that could actually do field robotics, and so we set that as our first niche.

How is the Canadian business landscape for robotics?

Many of our first customers were Canadian universities, some of which are leading the way in robotics integration and research. Meanwhile, Industry Canada is looking at how robotics technology can be adapted to existing manufacturing clusters in Southwestern Ontario, like the auto sector, which took a hit when the North American auto industry had a downturn. Canada also has strengths such as communications and optics, which naturally extend into robotics. It also helps when we can do business with Canadian suppliers; we get our GPS’ from NovAtel, which is based in Calgary, for example.

That said, as a Canadian company you’re forced to go multinational almost instantly because there isn’t enough of a robotics market in Canada to stay Canadian for long. We recently announced that we’ll be expanding office space into Silicon Valley and a primary reason for that is because of the opportunity that is available in the US market. Regardless, the Waterloo Region will continue to be Clearpath’s ‘home base’ for the foreseeable future.

People now refer to Waterloo as ‘Silicon Valley North’. What’s changed in the incubation space where Clearpath grew up since you launched?

It used to be that investors would demand that you move to the area where their office was, which was typically Boston or New York City or Silicon Valley, but that’s not always the case anymore.

It used to be that investors and venture capitalists would demand that you move to the area where their office was, which was typically Boston or New York City or Silicon Valley, but that’s not always the case anymore. Investors are starting to recognize that there are also benefits to being in other locations, especially for startups. Waterloo has a very dynamic tech community. It’s also a quarter of the price to buy a house here than in some of those larger US cities, and the time it takes me to get to work is 10 minutes, which makes it pretty livable and affordable compared to Silicon Valley. We found that VCs are now willing to accept that good ideas and businesses can grow and thrive in other areas as long as these areas have the right attitude – and Waterloo certainly has that.

Take the displacement of BlackBerry employees for instance – quite a few very smart people who were working at Blackberry have moved on to bring their skills and experience to other industries. We’ve hired a number of them, and a number of them have gone off to launch their own startups.

On top that is the continual increase in the brand of the Waterloo Region, and specifically the University of Waterloo (UW), as the place to go for getting a great education and practical experience that can lead to entrepreneurship. YCombinator’s Paul Graham has noted that UW has a fantastic record for turning out very strong companies, specifically hardware companies.

The other change worth mentioning is that Clearpath isn’t alone. The first group of startups that came out of UW includes Pebble, Vidyard, and more recently Thalmic Labs. These are all people who have graduated from UW’s engineering program within a few years of each other.

Does that help, too? Having a cluster that’s grown up together?

It certainly does.

Take recruiting, for example. When you’re hiring someone it helps prospective employees to know that there are other options. Clearpath has hired quite a few people from outside the region and if you’re taking a risk and moving a few thousand kilometers to join us, it’s good to know that there are other companies around, whether it’s for future growth opportunities, shared interests, or participating in knowledge transfer. For prospective employees, knowing that you can get back on your feet if you need to, or that your partner can find a job in the region, is very useful.

There is also something to be said for being part of a startup community. We all know each other, and we try to mentor each other. Even if you’re not working in exactly the same market, you can look at other startups and see them succeeding – and this is evidence that you can do it too.

A new book, Learning Robotics using Python, takes a different approach to teaching the Robotics Operating System (ROS). Written by Lentin Joseph, founder and CEO of Indian robotics startup Qbotics Labs, it enables you to learn by building an interactive, autonomous, mobile robot, and is the result of his research while designing the company’s autonomous robot prototype, Chefbot. It features Artificial Intelligence, vision capabilities, speech recognition and synthesis.

Chefbot is designed primarily to serve food in hotels and restaurants and Qbotics plans to release the robot as open hardware to develop a community focussed on improving the design. The company also provides assistance for clients autonomous machines.

Learning Robotics Using Python uses ROS Indigo, a stable version of ROS, installed on the latest Ubuntu L.T.S 14.04.02. The code is also compatible with ROS Jade. After a basic introduction, it looks at the mechanical design of robots using open source tools. ROS is introduced following the building of 2D and 3D models, and Chefbot is simulated using ROS and Gazebo. The Chefbot simulation is adapted from a Turtlebot simulator.

With a successful simulation in Gazebo, the book turns to the hardware element, the selection of the electronic components needed to build the robot. The sensor and actuator interfacing is explored and interfacing embedded boards to ROS using Python.

The Chefbot has (Kinect) sensors, ultrasonics and sound sensors, as well as motor encoders. The book looks at the detailed interfacing of each sensor with ROS and Kinect, and how to work with depth to pointcloud package.

In addition, it examines speech recognition and synthesis in robots, using ROS and Python, and implementing a version of Artificial Intelligence using ROS and Python.

After all this interfacing, the final integration is possible, both in hardware and software, resulting in the complete robot.

It needs a GUI controller, built using PyQt. Debugging is enabled via ROS rqt. Once the GUI has been constructed, it goes into the calibration and testing of the robot before its deployment.

Learning robotics using Python is available on the PACKT website and also Amazon.com.

If you liked this article, you may also be interested in:

• Book Review: Implementation of Robot Systems, by Mike Wilson
• Arduino for Makers #2: Deconstructing Arduino programs
• Arduino for Makers #1: Setting up a development station
• MATLAB Robotics System Toolbox and ROS

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In this episode, Audrow Nash speaks to several robotics companies at the company showcase at RoboBusiness 2014, which took place in Boston, Massachusetts.

The following is an ordered list of the people interviewed.

1. Norm Domholt (1m 40s), CEO of NPC Robotics,  talks about their modular remote operate vehicle Black Dog.
2. Annemarie Laudanski (6m 20s), Lead Field Applications Engineer at Xsens, demos their inertial motion capture system.
3. Carl Vause (9m 14s), CEO of Soft Robotics, speaks about the capability of the soft robotic manipulators, which can be used to pick fruit, among other applications.
4. Konrad Fagertun (13m 56s), COO of NLinks, tells about their new robot which drills holes in ceilings – a laborious and difficult task for humans.
5. Asim Ikram (15m 51s), CEO of Neo Robotics, discusses the market for robots in the Gulf Cooperation Council region.
6. Harri Valpola (21m 40s), Head of Technology at Zen Robotics, talks about the companies waste sorting robotic system.
7. Dick Zhang (26m 15s), CEO of Identified Technology, speaks about their self-piloting aerial drone systems for automated energy, mining, and construction site monitoring, surveying, and mapping.

Links:

• Download mp3 (18.8 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

Transcript

Norm Domholt (1m 40s), CEO of NPC Robotics:

Audrow Nash: Hi, can you introduce yourself and your company?

Norm Domholt: Yes, thank you. My name is Norm Domholt and I’m president of NPC Robotics. Today, at the show, we’re featuring and debuting our latest invention. We’ve been working on it for three years and it’s called the Black Dog.

Audrow Nash: Now, you have the Black Dog here. Can you describe it?

Norm Domholt: Black Dog is a remote control mobile platform. It’s designed modularly, so that it can be repaired in the field without probing with any sort of instruments and with the use of one particular tool – a quarter inch hex wrench. All the parts can be taken apart and replaced in minutes.

Audrow Nash: Can you tell me a bit more about how it’s modular?

Norm Domholt: We realized, through our experience with Battle Box, that the teams that seemed to win were the teams that designed their robots to be repaired quickly and without the need to take apart wiring.

It’s something that can be plugged in and replaced quickly. So, we decided that there should be around eight components in here that can be replaced with the use of this one tool and they should all be modular. There are no internal wires that you have to worry about, so everything unplugs and plugs together very quickly.

Audrow Nash:  So it’s a trunk-treaded robot that has two motors driving everything and it’s three feet long, a foot tall and two feet wide?

Norm Domholt: Yes, 42 inches long, 22 inches wide and 12 inches tall. It weighs 200 pounds and it goes four miles an hour. It’s got a lot of torque. Each motor is two-and-a-half horsepower – that’s a 24volt lithium battery.

Audrow Nash: Can you give me some examples of how it could be fitted with different components in the field?

Norm Domholt: Yes. Let’s say a SWAT team buys our robot power base; there are many different applications and mission-specific needs that they could have throughout a day. For example, they go to a house with something suspicious going on inside. They can put a telescoping camera, called RibbonLift, on the robot, within – say – ten seconds. They can then maneuver it to the building from a safe distance. If they hear shots fired and see there’s no need for the camera, they can pull the robot back within seconds, take their camera off and put a 300 pound bullet-proof shield on instead. Then, a SWAT person could operate this with a thumb throttle, while crouching behind the shield, and approach the house from a safe position. He can direct it to come back and hurl a teargas canister through the window. It could be adapted to any particular use very quickly.

Audrow Nash: Very good, and can you tell me at what stage of production your company is?

Norm Domholt: We’re debuting this today. We were hoping it would be a little further along and we could operate it. Some of the electronics are not completed yet; the mechanics are almost done and the modularity is complete. The technology that’s going in is already in place and, in another two months, we’ll be testing at facilities – breaking down doors, going through hallways, up and down stairs. We’ll be testing to see how far – wirelessly – this thing can go into a building.

Audrow Nash: Who would be interested in this product?

Norm Domholt: The same people who would be interested in a multi-purpose tool. Police can use it, the home-owner… Who would use a power-base? SWAT, the oil industry, hazardous waste…

Audrow Nash: So, lots of applications?

Norm Domholt: A lot of applications, yes.

Audrow Nash: Thank you.

Norm Domholt: You’re welcome, thank you.

Annemarie Laudanski (6m 20s), Lead Field Applications Engineer at Xsens:

Audrow Nash: Hi, can you introduce yourself and tell me about your company?

Annemarie Laudanski: Hi, my name is Annemarie Laudanski and I work for Xsens, we do animation promotion capture.

Audrow Nash: Can you tell me about what you have set up at this booth and describe it?

Annemarie Laudanski: Today we’re trying out our inertial motion capture system. It’s a full body- measure-capture system built of 17 inertial sensors. In each of these, there are 3-D accelerometers, gyroscopes and magnetometers. They’re used to measure the movement of arms and legs and send information wirelessly to the computer, where it’s translated into a bio-chemical model. So, we can measure exactly what someone is doing and portray that on our software.

Audrow Nash: At the booth you have a dancer?

Annemarie Laudanski: We do, yes. We hired Ashley. She’s the cheerleader for the Boston Celtics and she’s with us during the conference to showcase the use of our system.

Audrow Nash: You mentioned you were using exonorometers, gyroscopes and Megnatometers. So, it doesn’t require any external visual system to track them? You’re just taking the raw data from all of these sensors, running that through, and then feeding it to a model, to develop a person?

Annemarie Laudanski: That’s right. The exonorometers measure how fast each limb segment is moving, the gyroscopes measure the rate of turn of the limb and then we use the megnatometers to correct the drift in the other sensors. A fusion of these three sets of sensor data is sent wirelessly to the computer where it’s combined with our calibration procedure to build a human model. The calibration takes an average reading from the sensors, while the subject is standing still for five-seconds. This is a known position for taking sensor data, that then informs each of the trackers, so that the model is built correctly.

Audrow Nash: I see. As the dancer moves, it’s happening in real time.

Annemarie Laudanski: Yes, all of it is done in real time so all of the sensors are sending data at 30 bites per second to the computer and everything is shown with a latency of, at most, ten milliseconds, which is mostly due to the Microsoft operating system we are working with.

Audrow Nash: What are some applications?

Annemarie Laudanski: The applications are really wide for this type of technology. They go from entertainment purposes, where people are using it for movie and game design, to movement science, where it’s used for sport, rehabilitation or ergonomics and also human-robot interaction. So, you can even have someone in the system and driving a robot, or building up a database of human movements that can be used to program one and allow it to move like a human would.

Audrow Nash: Thank you.

Annemarie Laudanski: Thank you.

Carl Vause (9m 14s), CEO of Soft Robotics:

Audrow Nash: Hi, can you introduce yourself and the company?

Carl Vause: I’m Carl Vause and this is Soft Robotics Incorporated. We’re a Boston-based company that got spun out as a research group at Harvard about a year ago. We’re taking soft robotics technology and making it real, producing commercial products to solve new problems across a variety of markets.

Audrow Nash: Can you tell me about what you’ve displayed at the booth?

Carl Vause: At the booth today we have two soft hands: A very large hand for handling pumpkins, shoe bags and objects of very weighty geometry, and a small-handed, or short, robotic arm. What we wanted to show here is that, currently, there are a lot of problems with robotic manipulation. Manipulators don’t do well with soft, curvy, easily damaged objects and they don’t do well in unstructured gripping scenarios.

What we have solved using soft actuators is the ability to pick up anything with this arm, from a prickly cactus, which you would never pick up with your bare hand, to a racket ball or a cupcake. We want to show that, with almost no control system, building a very clever, soft actuator can solve a problem that has been vexing robotics: How to develop a gripper that is as adaptable and conformal as a human hand. People are spending millions of dollars to solve this problem using traditional hard robotics.

Audrow Nash: Can you describe the hand and the smaller robotic arm?

Carl Vause: One is a four-finger hand. It’s based on proprietary actuator geometry. We use vacuum to curl the fingers out of the way and they pressurize to create a grasp. We have a three-finger gripper as well. It actually has fingernails, so it can access a low profile object like a circuit board.

Audrow Nash: Can you tell me some potential applications of these soft hands? I know there are a lot.

Carl Vause: A lot, but how do you interface a robot with something that has an unknown shape? Or when the position is unknown? Or something easily damaged? Agricultural processing is interesting. Picking lettuces heads, for example. There’s a danger of introducing E. coli into the food chain, so washing and cleaning lettuce. Robotics companies and the US Food and Drug Administration want to automate that task. But robots today can’t handle lettuce, because their heads vary in size, shape and weight, and they are delicate objects. So, one of the things that we really wanted to do with our soft robotic technology is unlock that market for automation, and automate data handling.

They can even work in wet conditions; there are no electrical connections here. You can sterilize them and wash them down with bleach at the end of the day. It really does bring automation into the agricultural process. It makes it a reality.

There’s also a light-weight version for working side-by-side with a human collaborator. In a lab, for example where you need to be careful what you touch. A manipulator can pick up a range of lab instruments, or tools, and hand them safely to you. It has the credentials of collaboration.

Audrow Nash: The ones that you have on display are on open feedback loops. As in, they don’t sense what they’re picking up, they just actuate. You’ve talked a little bit about embedding sensors into the fingers in the hands. Can you tell me more about that?

Carl Vause: We can easily embed sensors to give an object grasp detection. We can embed it in our proprietary manufacturing process, in simple off-the-shelf sensors and feed them back into something very basic. If you want it very simple, take an Arduino and provide passive grasp detection. That’s very simple.

Audrow Nash: Thank you.

Carl Vause: Alright.

Konrad Fagertun (13m 56s), COO of NLinks:

Audrow Nash: Hi, can you introduce yourself and tell me about your company?

Konrad Fagertun: Yes, my name is Konrad Fagertun. I’m a co-founder of NLink Robotics. We’ve come all the way from Norway to Boston, to showcase the world’s first mobile drilling robot. We’re excited to be here at Robobusiness, showing what we’ve been working on for the last two years.

Audrow Nash: Can you describe the robot to me?

Konrad Fagertun: Yes. The challenge in the construction industry is hard labor – manual work – and the hardest of all is drilling holes in concrete ceilings. So, we made a mobile drilling robot. It comprises a mobile lift – that can drive around – and, on top of that, is a robotic arm holding a rotary hammer drill. It thrusts that up to the ceiling and can drill holes wherever you want. It uses lasers and cameras to position itself, so it can drill to a millimeter’s precision. This can be done hundreds of thousands of times, say for a large shopping mall or an office building. It’s repetitive work.

The robot is a little bit bigger than a man, and can reach several feet high, so you don’t have to get up on scaffolding or stairs to do the work; the operator can stand on the ground, safely at a distance. But, if he wants to get close, no problem, the robot is certified to work with humans, so it doesn’t matter if it crashes into your arm, it won’t hurt you and it’ll stop. It’s useful for electrical contractors, plumbers, mechanical engineers…

Audrow Nash: Thank you.

Konrad Fagertun: Thanks.

Asim Ikram (15m 51s), CEO of Neo Robotics:

Audrow Nash: Can you introduce yourself and your company?

Asim Ikram: Yes, my name is Asim Ikram and I’m the CEO of Neo Robotics in Dubai.

Audrow Nash: What do you guys do?

Asim Ikram: We’re a new start up in Dubai, not limited to robotics. In that region, they are not exposed to robots at the moment. We are starting at the lower level, to introduce smart apps and ICT, Information Communication Technologies. Then, we want to switch to robots, because the customers are the same.

Audrow Nash: So, basically you’re acting as a liaison between companies in the US and outside for bringing robots into Dubai?

Asim Ikram: Into the GCC region – the Gulf Cooperating Countries. There are six of them in the Middle East:  Saudi Arabia, Bahrain, Oman, Kuwait, Emirates and Qatar. They have a strong focus on technology, they’re teaching robotics and megatronics and have labs at almost every university. The industry will develop but, at the moment, there’s no pull from for robots.

Audrow Nash: What are some of the more popular technologies that are being brought to Dubai?

Asim Ikram: Not so much industrial robotics, but there are new production lines coming up that have industrial robots in the blueprint. Those lines should start operating in six months. The financing and planning is already done. Healthcare and education, that’s much further ahead. On the education side, there’s big competition between the private schools that are teaching things like systems programming. Now they’re also trying to get robots into the classrooms and teach with them. There’s a lot of movement there.

Audrow Nash: What robotics opportunities do you think there are in Dubai for engineers and entrepreneurs from the rest of the world?

Asim Ikram: There’s a lot of work in real-world drone applications. Already, highway police are using drones. They’ve also started making some national identity card deliveries with drones. Real applications of technology are moving fast. On the security side, I’ve seen a lot of civil defense robots for bomb disposal.

Audrow Nash: How did you decide to take this opportunity and form a company?

Asim Ikram: I’ve been working in this market for some time and I’ve been seeing a lot of pull in different areas. Healthcare and education are moving very fast and the industrial side will come up in the next one or two years. There are a few robotics service companies, but not many locally so I found a gap. It’s a good time to start something.

Harri Valpola(21m 40s), Head of Technology at Zen Robotics:

Audrow Nash: Can you introduce yourself and your company?

Harri Valpola: Yes, hello. I’m Harri Holopainen. I’m one of the founders of Zen Robotics Company in Helsinki, Finland. Currently, our main application is robots that have learnt to pick up waste. A little bit like WALL-E…

Audrow Nash: It looks a little different than WALL-E. Can you describe the robotic system you have?

Harri Valpola: Whereas WALL-E is cute, a small Hollywood robot, our robot is more like an large industrial scale model. It doesn’t look like a human; it’s not cute, but it works very efficiently. It’s powerful. It can lift and throw objects that weigh up to 20 kilos – 40 pounds – like they were nothing.

Audrow Nash: It looks like a conveyor belt and has robotic arms that move around and pick objects off of the conveyor belt and throw them to sort them. Am I correct?

Harri Valpola: Yes. It works in a similar way to human hands. All around the waist is a conveyor belt. The robot recognizes the objects and sorts them, exactly as a human would do, but faster. It’s also stronger and it doesn’t break.

Audrow Nash: What kind of sensors are you using to identify the material? What waste are you sorting now?

Harri Valpola: We are currently focused on construction and demolition waste. The sensors we are using are tailored for that particular waste. The most important material we’re sorting is  wood, followed by mineral-based parts and metal. We’re using sensors that can discriminate between these. They include metal detectors and spectrometers.

Audrow Nash: Where have you implemented the system and what sort of feedback are you getting from customers?

Harri Valpola: Currently, we have three customers in Europe; two in Finland and one in Holland. We’ve been developing this system for two years and now it’s working and customers are happy. We’re ready to scale up, now we have the product, so that other customers can start making use of this solution.

Audrow Nash: What’s the benefit of sorting the waste?

Harri Valpola: In Europe, at least, there are many companies that are floating in waste. Population densities are so high, there’s no place to put it. The only real option is to sort the waste. There are many valuable raw materials there. Sorting stones is not important but there are valuable metals, like copper and aluminum, and also wood. That’s particularly useful in Holland because they don’t have forests, and it can be recycled and used as a construction material.

Audrow Nash: Thank you.

Harri Valpola: Thank you.

Dick Zhang (26m 15s), CEO of Identified Technology:

Audrow Nash: Interviewer:  Can you introduce yourself and tell me about your company?

Dick Zhang: My name is Dick Zhang from Identified Technologies. We build and scale drones for commercial data capture. We work with construction and energy industry clients and help deliver data and insights. We get this information either by deploying robots on the ground, or by utilizing commercial planes and helicopters. It’s very expensive, it’s very dangerous and it’s very slow.

We deliver insights by collecting massive scale amounts of data on what has happened in the past, what’s going on now and what’s going to happen in the future, through predictive capabilities.

We collect the data via a three-part system: an autonomous drone, a dock station, that swaps batteries and uploads data, and a client dashboard that can be used to interact with and control the system from the cloud, from anywhere in real time.

We’ve been working with really exciting partners in energy and construction for the last six months. I run the business, look for exposure and develop partner and client relationships.

Audrow Nash: Can you tell me a bit about the demonstration – the drone, the quadcopter – you have on your booth?

Dick Zhang: The drone itself, everything inside, in terms of hardware, is really off-the-shelf components. Our dock station, that swaps batteries and outputs data, that’s all proprietary mechanical design and hardware. The software system is also proprietary. So a lot goes into flight control – safety checks, process checks, maintenance, things like that.

Audrow Nash: Swapping the batteries, is that automated?

Dick Zhang: Yes, that is automated. On the docking station, there’s a carrousel with six batteries that are always charged. It inserts another battery when the drone needs to fly.

To tell you a story, what inspired all of this was a talk with one of our clients. They run 5,000 job sites around the world. They bought an Internet tool but had no idea how they were going to train 1,500 people to use it. So I automated a system and made it self-operating and self-sufficient. We were able to employ 1,500 really rapidly across the country and across the world.

Audrow Nash: Is it totally self-sustaining?

Dick Zhang: Yes, right now all you need is a standard power plug. Future generations will evolve generators, solar panels and wind fans. But, apart from the source of power, everything happens on its own. It flies itself, captures data and swaps batteries.

Audrow Nash: What kind of navigation do you use on the drones?

Dick Zhang: Navigation in drone technology has become really sophisticated over the last couple of years. We have a combination of GPS and inertial sensors on board, so that, three or four hundred times every second, the drone is saying to itself: “I need to be in position XYZ. Right now I’m not there, but I’m close. Let me get there.” So, four hundred times every second, it’s making these corrections and adjustments so that it can fly along the path that the computer programmed.

Audrow Nash: Other than the IMU and the GPS, what other sensors do you use? Do you have a vision system for landing on the platform that swaps the batteries, and some other sensors?

Dick Zhang: Yes. Now we are working with cameras. We’ve done some early tests with our partners on infrared and gas effect. We do a lot of work in energy. As part of this start up growth, we’re going through a crafty process of crawl, walk and run. It’s the vision that enables the most capabilities, to help our clients right now, with the resources that we have. Now, to touch on how we land, that’s a huge problem. So, we built a proprietary vision system based on ultrasound. There’s a beacon and a dock and a transceiver, where you listen and triangulate.

Audrow Nash: Okay, so you use GPS to get it close enough…

Dick Zhang: Exactly.

Audrow Nash: Then the ultrasound takes over by localizing the sound somehow?

Dick Zhang: Yes, exactly.

Audrow Nash: Okay, thank you.

Dick Zhang: That was awesome. Thank you.

All audio interviews are transcribed and edited for clarity with great care, however, we cannot assume responsibility for their accuracy.

Fig. 1. The OpenBionics prosthetic hand.

OpenΒionics is an open-source initiative for the development of affordable, light-weight, modular robot hands and prosthetic devices, and can be easily reproduced using off-the-shelf materials.

Most current prosthetic hands cost tens of thousands of US dollars. The OpenBionics initiative presents a new open-source design for highly functional, affordable prosthetic hands. The proposed hands (see Fig. 1) cost less than $200, weigh less than 300g and can be easily fabricated with off-the-shelf materials and rapid prototyping techniques. A novel selectively lockable differential mechanism (based on the whiffletree) allows the user to block the motion of each finger independently, and facilitates the execution of 144 different postures with only one motor. To our knowledge this is the most affordable and light-weight prosthetic hand ever built. The design is based on parametric models of hand anthropometry studies, so it can easily be personalized to specific patients. Anyone interested should fill out the appropriate web-form, providing the desired specifications (hand type: left or right, hand length and breadth) and they will receive the personalized design via email.

Nowadays most amputees express their disappointment for: 1) the large cost of buying and maintaining a prosthesis, 2) the increased weight of the device and 3) the difficulties they face with repairs [1]. Moreover the fear of damaging the prosthesis leads most of the amputees to avoid using them in everyday life tasks and use instead simple hooks or grippers. On the other hand, when the amputees are involved in the selection and preparation of the prosthesis (e.g., replication of an open-source design), the likelihood of prosthesis acceptance is increased 8 times. These findings confirm that what amputees need is a highly functional, personalized, affordable and light-weight prosthesis that can be easily developed and repaired.

The OpenBionics initiative has developed and freely distributed an open-source design for creating anthropomorphic, under-actuated prosthetic hands of low complexity and cost. Our hands use a novel differential mechanism (a variation of the whiffletree or “seesaw mechanism” [2]) that blocks the motion of each finger, allowing the user to select between multiple grasping postures and gestures with only one motor. Human-likeness of both robot structure and motion is achieved by employing an index of anthropomorphism in the design process [3] that uses parametric models derived from hand anthropometry studies [4]. The proposed hands can be easily reproduced with off-the-shelf materials and can be fabricated with rapid prototyping techniques.

Hand Design

Robot Fingers – In our design, anthropomorphism is reflected in two specific design choices: 1) the use of an anthropomorphic kinematic model [3] and 2) the use of a bio-inspired finger actuation and transmission system. The bio-inspired finger design structurally reproduces the flexion movements of the human hand with tendons driven through low-friction tubes and the extension using elastomer materials. The backbone of the finger can be constructed with Plexiglas (acrylic) or ABS, and the flexure joints with silicone or polyurethane sheets.

Thumb Mechanism – For the thumb we use a selectively lockable, toothed mechanism that can implement 9 different opposition configurations. The mechanism substitutes the three Degrees of Freedom (DoF) that implement the human thumb opposition with only one rotational DoF and is completely stiff when it is locked, thus it is not affected by torsional forces inherent in dynamic or unstructured environments. A separate tendon routing system is used for the thumb, and the thumb tendon is terminated to a separate servo pulley.

Selectively Lockable Differential Mechanism – We developed a selectively lockable differential mechanism that can block the motion of each finger independently. Such a mechanism allows the user to intuitively select the desired finger combinations. The top two bars of the whiffletree (see Fig. 2) have appropriately designed holes, and the palm accommodates a series of buttons that are elongated upon pressing. The idea is that, when each button is pressed, the elongated part fills the appropriate finger hole and the motion of this particular finger is constrained. A total of 16 different index, middle, ring and pinky combinations can be implemented using the differential mechanism and a single motor. These can be combined with the 9 discrete positions of the thumb, to produce a total of 144 different grasping postures or gestures with a single motor.

Fig. 2: The left figure presents the inner side of the palm. The three bars of the modified whiffletree are depicted with red color. The holes of the upper two bars are filled by the elongated parts of the corresponding buttons, constraining the motion of different fingers. In this instance the motion of the index and pinky fingers is constrained, resulting to a change at the inclination of the differential mechanism finger bars. The right figure depicts the thumb mechanism in red and golden colors.

Personalized Designs – Τhe use of parametric models derived from human hand anthropometry studies allows for the development of personalized prostheses (see Fig. 3). The only parameters required to derive the finger phalanges lengths and the personalized finger base frames positions and orientations are the human hand length (HL) and the human hand breadth (HB).

The proposed hands can be fabricated using off-the-shelf, low-cost materials and rapid prototyping techniques (3D printing) or standard machinery tools (CNC, laser cutters etc). All required materials can be easily found in hardware stores around the world.

You can use the following link to order your personalized design (.CAD files): Web Form for Personalized Designs. All cad files (Solidworks .sldasm, .sldprt and .dwg, .dxf, .stl), required for the replication of the proposed design are distributed through our website.

Fig. 3: Personalized designs. Two different robot hand models are depicted. The left hand version was created with a total length of 17.8cm, while the right hand version with a total length of 19cm.

Fig. 4. The prosthetic hand developed by the OpenBionics team.

The proposed design is fully described in the following paper:

George P. Kontoudis, Minas V. Liarokapis, Agisilaos G. Zisimatos, Christoforos I. Mavrogiannis and Kostas J. Kyriakopoulos, “Open-Source, Anthropomorphic, Underactuated Robot Hands with a Selectively Lockable Differential Mechanism: Towards Affordable Prostheses”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg (Germany), 2015 (under review).

The OpenBionics initiative provides also designs for open-source, affordable (<100$), light-weight (<200gr), under-actuated robot hands.

Experiments conducted with the proposed hands can be found at the following videos:

More information can be found in our website, http://www.openbionics.org, Facebook, Twitter:

References:
[1] D. Edeer and C. W. Martin, “Upper limb prostheses – a review of the literature with a focus on myoelectric hands,” 2013. [Online]. Available: www.worksafebc.com/evidence
[2] L. Birglen and C. M. Gosselin, “Force analysis of connected differential mechanisms: application to grasping,” The International Journal of Robotics Research, vol. 25, no. 10, pp. 1033–1046, 2006.
[3] M. V. Liarokapis, P. K. Artemiadis, and K. J. Kyriakopoulos, “Quantifying anthropomorphism of robot hands,” in IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2013, pp. 2041–2046.
[4] B. Buchholz, T. J. Armstrong, and S. A. Goldstein, “Anthropometric data for describing the kinematics of the human hand.” Ergonomics, vol. 35, no. 3, pp. 261–73, 1992.

Recent Publications:

Agisilaos G. Zisimatos, Minas V. Liarokapis, Christoforos I. Mavrogiannis and Kostas J. Kyriakopoulos, “Open-Source, Affordable, Modular, Light-Weight, Underactuated Robot Hands”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago (USA), 2014. [BibTeX] [PDF]

Minas V. Liarokapis, Panagiotis K. Artemiadis and Kostas J. Kyriakopoulos, “Quantifying Anthropomorphism of Robot Hands”, IEEE International Conference on Robotics and Automation
(ICRA), Karlsruhe (Germany), 2013. [BibTeX] [PDF]

Agisilaos G. Zisimatos, Minas V. Liarokapis, Christoforos I. Mavrogiannis, George P. Kontoudis and Kostas J. Kyriakopoulos, “How to Create Affordable, Modular, Light-Weight, Underactuated, Compliant Robot Hands”, Control Systems Lab, School of Mechanical Engineering, National Technical University of Athens, January 2015. [BibTeX] [PDF]

****

Editor’s note:
The OpenBionics initiative is not to be confused with Open Bionics Ltd (ex featured here).

• DARPA’s HAPTIX project hopes to provide prosthetic hands with sense of touch
• Lifehand 2 prosthetic grips and senses like a real hand
• Revolutionary implant enables broken spinal cord to function again
• Introducing the Cybathlon

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

I love hacking – i.e. modifying everyday items you already have at home or that can be purchased easily. And my daughter loves air hockey! This fun project combines standard RepRap 3D printer parts (NEMA17 stepper motors, drivers, Arduino Mega, RAMPS, belts, bearings, rods, printed pieces …) with video detection and motor control to make an air hockey robot that you can play against.

The main advantage to using these parts is that they are inexpensive and easily available. I wanted something easily transportable and comfortable to play, but at the same time, easy to replicate and open source.

After some weeks scratching my head and problem solving, I can proudly state: It’s air hockey time!

Developing the robot

In the end I decided to use a 3-motor design (2 for the Y axis and one for the X axis). After several design iterations, and some material changes to minimize the weight and inertia (and thus increase the accelerations of the robot), I got a fairly operational design. One of the most important changes is the replacement of the X-axis rods with carbon tubes (from kites), which works very well on PLA-printed bushings and saves a lot of weight.

The code

I started studying the code for Marlin (typical RepRap firmware) software, but in the end I decided to start from scratch. This was because I don’t need a gcode interpreter, and also because the software of a 3D printer has a motion planning algorithm and this is not the way an air hockey robot must work. 3D printers plan movements for smooth paths through all the points, but an air hockey robot should move immediately with every new command cancelling the previous one, because we want the robot to move as quickly as possible to the new position.

I began developing the driver for the stepper motors using Arduino interruptions and making the code as efficient as possible. I implemented position, velocity and acceleration control using an acceleration ramp with a simplified S-profile. The interrupts can generate pulses up to 25Khz, which is the maximum speed that I can achieve with the steppers. The position control works at 1 kHz.

Puck detection

For the puck detection problem I decided to use a computer vision system. Initially I planned to use the CMUCAM5 (Pixy) sensor, which fits very well in this project (and in near future I will prepare a version for this sensor), but the camera is not available yet, so instead I used the PS3 EYE camera with a PC to develop the vision system. I developed the vision system in C using OpenCV libraries for capturing, thresholding, filtering and segmentation.

The system detects the puck (it must have a special color) and the result is sent to the Arduino by serial port. The vision system sends information packets containing the position of the puck and the robot in pixels within the image. The camera is running at 60Hz at a resolution of 320×240. Fortunately the camera has very little distortion and I don´t need to implement a lens distortion correction. The software records a video on the PC – very useful for debugging.

One of the main problems I’ve had is that the power of the NEMA17 motors are pretty fair for this project and the disadvantage of using stepper motors is that if we lose steps (which can easily happen, for example when the robot hits the puck against the wall) and we do not take into account the missing steps, we lose the position of the robot. In order to fix this, I use the vision system to detect the robot as well (with a different color) so we can detect when the robot has lost steps and fix it.

What remained was the implementation (in the Arduino) of the trajectory prediction system and the strategy of the robot. Once we have detected the puck in two consecutive frames we can calculate the trajectory (equation of the line through two points, this is college math!). The trajectory prediction takes into acount that the puck can rebound against a side wall. All these calculations are accesible to the strategy subsystem that decides what the robot will do: defense, defense+attack, and preparing for a new attack.

A nice feature of this project is that the strategy subsystem is fully insulated and is very easy to modify and reprogram, so you can develop your own strategy algorithms from the complexities of motor control, vision system and prediction code.

Although it seems like a complex project on the surface, it’s a surprisingly affordable and “doable” project.

• ROS 101: Intro to the Robot Operating System
• 3D video game with robots
• Arduino for Makers #1: Setting up a development station
• New open source Turtlebot tutorial aims to bring robotic development to the masses
• 20 great books to hook kids and teens on robotics

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

3D Robotics has unveiled DroneKit, an open-source development kit for app development. This is a clear invite for the wider development community to build upon 3DR’s flight control software.

The SDK and web API allow developers to build apps for the air, ground or cloud. Exciting times for anyone who is a 3DR customer. 3DR’s main competitor DJI also has a development kit available for the developer community.

Benefits of becoming a 3DR developer include:

• UNIVERSAL PLATFORM – Develop your drone apps for Android, laptops, embedded computers and web.
• ROBUST AND POWERFUL – DroneKit builds on the work of thousands of contributors and years of R&D.
• VEHICLE AGNOSTIC – DroneKit works with copters, planes, rovers and even blimps!
• FREE AND OPEN – No contracts to sign, no hoops to jump through. DroneKit is available to all.
• EASY TO USE – Set up and take control of your drone in minutes, not days.
• EXTENSIBLE – DroneKit can be extended to handle additional sensors and actuators.

DroneKit features include:

• Fly paths with waypoints
• Fly in spline path with fine grain control over vehicle velocity and position
• Have the drone follow a GPS target
• Control the camera and gimbal with regions of interest points
• Access full telemetry from the drone over 3DR Radio, Bluetooth, Wi-Fi, or over the internet
• View playbacks and log analysis of any mission

DroneKit powers the most successful flight control programs in the world:

• Tower (formerly Droidplanner), hands-down the best flight planning mobile app out there, was built on DroneKit for Android
• Droneshare, the global social network for drone pilots, is built on DroneKit web services
• Project Tango Indoor Navigation is built on Pixhawk, APM and Tower
• IMSI/Design TurboSite aerial reporting app for construction

You can visit the dedicated website at DroneKit.

The tutorials take developers from the initial setup of TurtleBot through programming a web app to control it. The final demo is to bring coffee to people around an office, but my ultimate goal is for creative new users to learn from this simple idea and modify at their whim to generate inventive uses and expansions.

I produced this free series in hopes of bringing robotics technology to the web development community. As we recently saw with 3D printers, web developers have a long tradition of embracing emerging technologies. My goal, as a member of both the open-source robotics and web development communities, is to encourage web developers to start learning robotics so that we can see the same sort of creativity and popularity take off.

The way I see it, if we can give developers a taste of ROS’ power, they will invest the time to learn ROS properly. Today robotics stands on the foundation built by many brilliant people and engaged, innovative communities; now it is time to mine the untapped potential of developers not previously involved with robotics. By sparking developers’ interest and teaching them the basics of ROS, I hope to break down even more barriers and put us that much closer to making the robot revolution a reality.

(You might recall that we also recently built the open electronics and firmware for a new robot hand.)

UPDATE: New video of a collaborative, cloud-based mapping experiment. Mapping is essential for mobile robots and a cornerstone of many more robotics applications that require a robot to interact with its physical environment. It is widely considered the most difficult perceptual problem in robotics, both from an algorithmic but also from a computational perspective. Mapping essentially requires solving a huge optimization problem over a large amount of images and their extracted features. This requires beefy computers and high-end graphics cards – resulting in power-hungry and expensive robots.

Update

Here is a video of a collaborative mapping experiment using Rapyuta, the open source cloud robotics framework. Instead of just compressing each frame of the stream, we now run a dense visual odometry algorithm to produce keyframes and these keyframes are eventually sent to the cloud for optimization. This technique is a very natural way to compress since keyframes are produced only when the robot has moved a certain distance or angle. The Cloud, in addition to optimization, now also does merging of maps produced by multiple robots. The code is open under the Apache 2.0 license at https://github.com/rapyuta/rapyuta-mapping.

Together with our RoboEarth colleagues Dominique Hunziker, Dhananjay Sathe, Mayank Singh, we have set up an inexpensive, light weight robot so that it can perform full 3D mapping in real-time by offloading heavy computation to the RoboEarth Cloud Engine (nickname: Rapyuta) (see previous Robohub article). In the video below Master student Vlad Usenko shows how robots can sidestep this problem.

How it works
What actually happens here? As you can see from the video and the figure below, the robot is equipped with an RGBD sensor (ASUS Xtion PRO), an ARM-based single board computer (Odriod-U2), and an off-the-shelf wireless dongle. The robot transmits RGB images (QVGA resolution) and depth images (QVGA resolution) at 30 FPS to Rapyuta with a frame by frame compression (JPEG for RGB, and PNG for depth images). The single board computer is only used for compression, communication, and low-level control of the robot – the heavy computation all happens off-board. The optimized map shown at the end of the video is built by manually triggering the optimization process in the cloud after the demo (i.e., not in real time).

The main purpose of this demonstrator is to showcase that today’s wireless technologies allow robots to outsource computationally heavy data processing to the cloud. As shown in the figure below, and as I’ve laid out in an earlier article on Cloud Robotics, bandwidth is a key driver for cloud-based robot services and applications.

Most actual mapping scenarios call for a slightly different split in work-load: You’d want to perform visual odometry locally and only send keyframes to Rapyuta for global optimization. The good news is that we already supports this setup: Rapyuta provides the corresponding software packages to run on ODroid boards along with the necessary cloud computing infrastructure.

What’s next?
This demonstration stops short of actually using the data returned from the cloud: A map is created and available in real time, but not actively used to control the robot. Using the map to control the robot, e.g. to improve exploration, is one logical extension.

Another natural next step would be to use the computational resources in the cloud for localizing in an existing map. Imagine a robot entering a new building and trying to get its bearings. Such localization in an existing map requires performing a search over the entire map – essentially a very large set of images and their features – to find the best match between what the robot is seeing now and a part of the map.

But the cloud can be used to do much more interesting things than improve single robot navigation. Multiple robots can instantaneously share their maps, so applications like multi-robot mapping become much easier. This implies using the cloud as more than a pure computational resource – it becomes a medium to store, share, and collectively improve data. This is interesting for static scenes, but things get even more interesting for dynamic scenes, for example when multiple robots track dynamic objects in the map. With WIRE, some of my RoboEarth colleagues have developed algorithms to do just that. We are currently working on integrating this functionality with Rapyuta and the RoboEarth database and will make it available as a Rapyuta cloud service later this year.

Many of the key advantages of using this Cloud Robotics approach to mapping will only become apparent when maps are brought together with the vast amounts of knowledge required to understand environments. Performing mapping in the cloud not only allows the creation of maps but also the ability to understand them: By bringing computation close to the knowledge required to make sense of all of a robot’s sensor information, Cloud Robotics offers robots a very powerful way to understand the world around them. When working together, cloud computing frameworks like Rapyuta and online knowledge bases like RoboEarth can allow us to make robots that are both smart and inexpensive.

Additional information
For further information, see:

• Technical details and tutorial for this setup (available early July)
• Video of recent live Cloud Robotics Mapping demo at ROSCon 2013
• Cloud Robotics Workshop at IROS 2013 in November, Tokyo
• Homepage of the RoboEarth Cloud Engine (codename “Rapyuta”)
• RoboEarth homepage

Questions?
Feel free to post in the comments below.

Aisoy, a spanish robotics startup, is motivated by the goal of building intelligent, personal, “social” robots, which make our lives easier and funnier. Their robot, the Aisoy1, is their first step towards achieving this vision. Robohub recently caught up with the team, to talk about social robotics, the Aisoy1, and the startup culture in Spain.

Can you briefly give the readers an overview of the Aisoy1 robot, and the audience/experience level that it is targeting?

Aisoy1 is an affordable, social-robotics platform designed for anyone over the age of 7 years, who wants to experiment with personal, emotional and social robotics, or just explore robotics and have fun. Our audience includes kids, adults, educators, artists and creatives who want to learn about social robotics technology.

Aisoy1 is based on ROS and uses the Raspberry Pi (other single-board computers will be available soon). Beginners can program the Aisoy1 using AIDIA (a web based graphical environment) or using MIT’s Scratch programming environment. For advanced users who want to delve deeper, programming in Python or C++ is also an option.

What do you mean by “social robotics”?

Social robotics is a deep concept. For us, a social robot is a robot that is:

• Intelligent in the sense that it can understand our desires and can have fun with us.
• Emotional in the sense that it can interact with us in a natural way, while also being able to express emotion and give the impression of feeling and evolving.
• Personal in the sense that it provides a different and unique experience for every person.

What vision or idea motivated the development of the Aisoy robot?

Aisoy is motivated by the idea of creating “life” with emotional (social) robots. We want to build emotional links between us and our physical and digital worlds, by giving emotions to the objects and services that we use every day. The first step we’ve taken is the creation of the Aisoy1, a robot that can interact with us in a way that we as children only dreamed of.

In the creation of the Aisoy1 (our first robot), we wanted to build a platform that included cutting-edge technologies in areas such as pattern recognition, unsupervised learning, emotional and decision frameworks, natural language processing for dialogue management, automatic speech recognition, text-to-speech, and computer vision.

For me, one of the most interesting features of the Aisoy is the integration with cloud-based services, such as audio processing and a text-to-speech engine. Is offloading complex services to the cloud a future direction you see for robotics? What are the next services you have planned for Aisoy?

The cloud is right now a key part of the ecosystem of our life and in the near future, we believe cloud robotics will be fundamental in the development of more advanced and capable robots. We are seeing the first steps in this direction already: Amazon are developing cloud-connected drones for usage in deliveries, and Google is taking a stance in the robotics world. Two companies whose major businesses lie in the cloud – it will be very interesting to watch how their access to cloud-computing resources inspire their robotic developments. In the not-too-distant future, we can certainly imagine ourselves living among connected robots, each of them sharing knowledge with us and the other robots, in a kind of shared intelligence.

For a more concrete example, consider cloud services, such as speech recognition and text-to-speech, which have become commodities for our smartphones – common features, like Siri or Shazam, couldn’t exist without them. The same is happening with robots: there are capabilities, such as SLAM or object recognition, which are computationally very expensive (currently too expensive for the majority of microprocessor-based robots), but can be delivered effectively through cloud integration. Such capabilities, we believe, will become standard for future robots.

We’ve bet on this since the beginning, and have already integrated cloud-based speech recognition and text-to-speech. We’re currently migrating our emotion and decision engines to the cloud so that you can use it from your robots and from other connected devices. By porting these services to the cloud, we want to give people a new kind of experience using social robotics.

Basing the Aisoy on standard, (partially) open source hardware and software such as the RaspberryPI and ROS makes the Aisoy a very customizable platform. What is the coolest thing you’ve seen done with the Aisoy?   

There are many people working with Aisoy1 and their inventions constantly surprise us, for example: a Star Wars Encyclopedia programmed in AIDIA or the integration of Aisoy with turtlebot for making an assistant guide inside a shop.

Socially, one of the coolest projects is taking place in Alicante (Spain), where several schools are preparing a stage play where the main characters are Aisoy1, programmed by children under 11 using Scratch.

We were also very proud when Aisoy1 was the host of Telefonica Talentum and introduced the CEO. It was amazing.

What are the next steps in Aisoy development?

Our development efforts are guided largely by the feedback and requests we get from our customers. A few things they are asking for (and we are currently working on) are:

• Sophisticated personalities with clear focused functions (e.g. security or energy saving)
• Visual development tools for users with low technological background (this is motivating our Scratch integration)
• A mobility device for Aisoy1
• Integration with more cloud services

Now I’d like to change the focus and look at Aisoy as a company. We often hear stories of US-based startups, but very little from the rest of the world, hence I think it would be very interesting to get an insight into the startup culture in Spain.

Is the Spanish ecosystem (funding, mentoring, etc) supportive of startup companies with new and novel ideas?

Spain still has a long way to go in this area. Companies such as Facebook, Apple or Google, could never have been founded in Spain – their ideas were too innovative, and too unproven. Spanish investors usually invest in companies whose technology has proved successful in existing markets and who have equally successful business models – minimal-risk investments. They are happy to invest once the company has an income, but are extremely conservative in supporting young companies with innovative ideas to get off the ground.

What were the major difficulties in the early stages of the company, and how / with what resources did you overcome these challenges?

In the early stages of Aisoy, we had two major challenges: Finding funding, and surviving the economy.

Finding funding was difficult for us. In Spain it’s easier to reach 200K€ in a “Friends, Family &Fools” first round, or even 10M€ for well-established companies with proven business model, than it was for a company like us, in the end-phase of a long-term technological development, to reach 2M€. We are very thankful for our first funding from the founders and CDTI (The Centre for Industrial Technological Development, a Spanish Public Business Entity) through the Neotec program for technology startups.

Relatedly, our first years of development and production were during the high-point of the Spanish economic crisis and the Fukushima disaster (which affected hardware supplies). It was not easy for us to survive during these years, but we love robotics, believe in our vision, and have fought hard to keep pushing Aisoy forward. We’re now very proud to see the results of our efforts over the last years!

In the last few years we’ve seen the development of many different strategies and frameworks for running a startup. Did you take guidance from such resources, or find them helpful in the early stages of AISoy?

Nowadays people are using or talking about these methods (eg. Steve Blank’s “The Startup Owners Manual”, Alexander Osterwalder’s “The Business Model Canvas”, Eric Ries’ “The Lean Startup”), but five years ago such guidelines were hard to come by. In our case, we used similar methodologies without knowing that we were using their principles. In the end, most of it is just common sense.

Looking back on the last five years, what knowledge/experience that you have now would have been useful to have when you started? What would you do differently, and what would you do the same?

Jeez, where should we begin… Experience is experience! But if we were to do it all again, we would:

• Use affordable 3D printers (such as the Prusa i3 we have recently built), to drastically improved the speed of prototyping, thus reducing risks by testing more and reaching the market faster.
• Make the hardware more modular so that we can reduce the risk of component obsolescence or supply problems affecting the whole system.
• Develop the SDK and developers oriented tools first, making it easier for us to develop for the system

In the end though, these things are a matter of continuous evaluation and improvement!

Any closing words?

Thank you for giving us this opportunity to reach your audience (we’re among them). We’re really looking forward to showing you the upcoming tools we’re developing, such as the new AIDIA and our latest development, a Chrome App to bridge with Scratch and cloud services. Stay tuned!

Thank you to the Aisoy team! If you are interested in more information, check out their homepage!

In academia, where robotics researchers look to open source as a means of advancing community knowledge, the answer is perhaps more obvious. But in business, it’s clearly a balancing act. And so, ‘To be open, or not to be open?’ — that is the question for our panelists this month.

We asked Frank Tobe, Robert Morris and Brian Gerkey to weigh in. Here’s what they have to say …

Brian Gerkey on “Is open source a good business model for robotics?”

The IT economy has powerfully demonstrated what happens when companies can leverage open source infrastructure when they build new products and services.  A company like Google would never have come into existence had they not been able to rely from the beginning on solid open source tools like Python and GCC.  IBM would arguably have not been able to make its immensely successful pivot from products to services without Linux.  How many startups these days begin as a cloud-hosted machine running some derivative of the venerable LAMP stack? …

Read more →

Robert Morris on “Is open source a good business model for robotics?”

The premise of this question is that robotics companies are manufacturers and that there is choice between an open source and closed source business model.  Robotics companies are best thought of as service companies (even manufacturers, especially when moving beyond early adopters) and openness is not an ‘either/or’ choice, but rather a continuum.  In this day and age the question is, ‘What do you need to keep open create value for your customers?’ …

Read more →

Frank Tobe on “Is open source a good business model for robotics?”

Certainly robotics has its share of proprietary software and control systems. Each robot manufacturer markets their products based on the need for secure, proprietary and un-shared systems so that they can ensure stability and control. Whole industries have been set up to bridge those proprietary barriers so that multi-vendor solutions can happen …

Read more →

While arguing by analogy is fraught with peril, I believe that the similarities between robotics and the rest of the IT world are strong enough to justify it.  In robotics, we have many shared problems to solve when developing a product or service, from low-level drivers to high-level capabilities, and all the developer libraries and tools in between.  I have yet to see a successful robotics business for whom any of that stuff is the competitive advantage.  Rather, success comes from the innovative composition and application of that technology in a form that somebody will pay for.  The hard part is figuring out what the robot should *do*.  By working together on the common underlying problems, we end up with better, more reliable solutions, and we free ourselves to spend more time at the application level, which is where we can differentiate ourselves.

In other words, I believe that open source is a great model for the robotics business as a whole.  Now, is it a good model for any individual company?  It certainly can be.  As examples, we see small-to-medium companies, such as Clearpath Robotics, Rethink Robotics, and Yujin Robot, which use ROS directly in their products. And we see larger companies, such as Bosch and Toyota, using ROS in R&D and prototyping efforts.  These are all profit-motivated companies making what is presumably a rational economic decision to rely on open source software.  They’re each holding something back that is their “special sauce,” whether that’s higher level application software, configuration data, customizations to the open source code, or the designs for the hardware.  And that’s expected: unless you’re in a pure consulting business (selling your time), then you need to own and control something that forms the basis of your product or service offering (to allow you to sell something other than your time).

Fortunately, open source software is entirely compatible with such business models.  In fact, it was our hope to one day see such commercial users of ROS that led us to choose a permissive license (BSD, or Apache 2) for the code that we developed.  We’re now witnessing, with the debut of so many new robotics companies, the fruits of those earlier labors in building a shared development platform.

Read more answers →

This is the third tutorial in the Up and flying with the AR.Drone and ROS series.

In this tutorial we will:

1. Learn about the AR.Drone’s state feedback (and how it is handled by ROS)
2. Learn about the AR.Drone’s tag detection
3. Program our first ROS nodes: A subscriber and a publisher

In the previous tutorials we:

1. Installed ROS, the AR.Drone driver and AR.Drone keyboard controller then flew the AR.Drone using the provided keyboard controller (link)
2. Learned about ROS communication, setup and then flew the AR.Drone using a joystick (link)

In the next tutorials, we will:

• Write a controller to enable tag following
• Write a controller for the drone which allows us to control drone velocity, rather than body angle

What you’ll need for the tutorial

1. An AR.Drone!
2. A computer or laptop with WiFi
3. Linux, either the ARDroneUbuntu virtual machine, or a custom installation
4. Either the keyboard or joystick controller from the previous tutorials

I also assume that you’ve completed either the keyboard or joystick tutorial, which detail how to setup the software, connect to the drone, etc, and provide a method of controlling the AR.Drone from your computer.

Updating

If you have not completed the joystick tutorial:

Please go back and update your installation according to the instructions listed there.

If you downloaded the ARDroneUbuntu virtual machine:

To update your installation for this tutorial, click the “Update Tutorials” button in the ARDroneUbuntu Virtual Machine

If you use a custom installation:

At the terminal, enter the following:

$ roscd ardrone_autonomy
$ git pull
$ roscd ardrone_tutorials
$ git pull
$ roscd
$ rosmake ardrone_tutorials

AR.Drone State Feedback – A First Look

Preparation:

1. Find the Orange-[Green|Yellow|Blue]-Orange tag that came with the AR.Drone.
2. In your ARDroneUbunutu virtual machine (or custom installation), open a new terminal window and type:

   $ roscd ardrone_tutorials/launch

3. If you intend to fly with the keyboard, type:

   $ geany keyboard_controller_with_tags.launch

   Otherwise, if you intend to fly with the joystick:

   $ geany joystick_controller.launch

4. Now find the line:

   <param name="enemy_colors" value="3" />

5. And change the value to “1”, “2”, or “3”, depending on whether your tags are Orange-Green-Orange, Orange-Yellow-Orange or Orange-Blue-Orange respectively.

Tag Detection and Feedback Data

1. Run the system (but don’t takeoff with the drone!) using either the keyboard_controller:

   $ roslaunch ardrone_tutorials keyboard_controller_with_tags.launch

   or the joystick_controller:

   $ roslaunch ardrone_tutorials joystick_controller.launch

2. Open a new terminal window and type:

   $ rostopic list

   • This list shows all the active ROS “topics” used for communication to and from the AR.Drone
3. In the same terminal window, type:

   $ rostopic echo /ardrone/navdata

   • This echoes the AR.Drone’s feedback data into the terminal window.
   • Note that many items will be zero because the AR.Drone is not yet flying
4. Move the Orange-Color-Orange tag in front of the AR.Drone’s front camera. It should be detected, and the console output (from the previous step), should update to reflect this detection.
   • Note the tags_xc and tags_yc values. These represent the x/y location of the tag within the image.
   • The video display window will show a dot at these coordinates, and the distance measurement above the dot
5. Now place the tag somewhere that can be seen by the flying drone. After this is done, take off with the drone.
6. Watch the console output. What happens if you fly forwards? sideways? What variable changes if you rotate the drone?

Plotting the AR.Drone’s State

1. A more intuitive way of representing this feedback data is using a plot. PressCtrl-C to terminate the echo process from the last stage, and then type:

   $ rxplot /ardrone/navdata/rotX,/ardrone/navdata/rotY

   • This launches a plotting window showing the AR.Drone’s estimated roll (rotX) and pitch (rotY).
   • Similar plots could, for example, be generated for yaw (rotZ), x and y velocity (vx and vy), altitude (altd), etc.
2. Similarly to displaying the AR.Drone’s feedback data, we can use the plotting method to investigate how we are controlling the drone:

   $ rxplot /cmd_vel/linear/y,/cmd_vel/linear/x

   • This plots the roll (linear/y) and pitch (linear/x) commands.
   • Note that these are scaled with a value of $\pm1$ being the maximum commandable angle (set by the euler_angle_max in the launch file).
   • Note also that the direction of AR.Drone movement is commanded (ie. move along the x-axis = /cmd_vel/linear/x = commanded pitch), whereas the AR.Drone’s feedback data is named with reference to axis of rotation (ie. tilt around y-axis = /ardrone/navdata/rotY = feedback pitch).

Logging Data

1. To log data, we can use the inbuilt ROS command rosbag:

   $ rosbag record -O ARDroneFlight.bag /ardrone/navdata /cmd_vel

   • This will record the topics /ardrone/navdata and /cmd_vel to a “bag-file”, stamped with the current date and time.
   • Leave this process running for a few seconds, fly the AR.Drone around, and when finished press Ctrl-C to terminate.
   • Recording all topics is possible with rosbag record -a, however this also records the AR.Drone’s HD video stream, thus bag-files will quickly become very large
   • The advantage of using rosbag to log data, is that bag-files can be replayed at a later time, and interact very nicely with the existing ROS infrastructure.
2. Land the AR.Drone and close any windows / terminate any processes relating to the controller. Now, into the terminal window that ran the AR.Drone controller, type:

   $ roscore

   Now open a new terminal window and type:

   $ rxplot /cmd_vel/linear/y,/cmd_vel/linear/x

   Now finally (ignoring the warning from the previous step), switch back to the terminal window that you used to run the rosbag command and type:

   $ rosbag play ARDroneFlight.bag

   Does the flight look familiar? It should! This is the flight you recorded in the last step! Use Ctrl-C to terminate the above processes.

Custom Data Handling

In the last section, we investigated the AR.Drone’s feedback data by using ROS’s inbuilt plotting (rxplot) and logging (rosbag) commands. In this section, you will learn how we get this feedback data into your own application by writing a custom ROS Node. This node will be used to compare the commanded roll and pitch angles with the AR.Drone’s roll and pitch estimates, thus laying the foundations for the next tutorial, where we will use this feedback data to programmatically control the AR.Drone.

Messages, Topics and Nodes

In ROS terminology, a program is known as a Node. To interact with other nodes (for example the AR.Drone driver), each node will have some subscribers, which listen for information messages on a specific topic (communications channel); and some publishers, which send messages to a specific topic.

These messages can be anything. An example that we saw in the previous section was the Navdata message, which is sent over the /ardrone/navdata topic and contains feedback data from the AR.Drone.

We will now investigate this message a little further. Open up a new terminal window and type

rosmsg show ardrone_autonomy/Navdata

You should be presented with the following output:

$ rosmsg show ardrone_autonomy/Navdata
std_msgs/Header header
  uint32 seq
  time stamp
  string frame_id
float32 batteryPercent
uint32 state
int32 magX
int32 magY
int32 magZ
int32 pressure
int32 temp
float32 wind_speed
float32 wind_angle
float32 wind_comp_angle
float32 rotX
float32 rotY
float32 rotZ
int32 altd
float32 vx
float32 vy
float32 vz
float32 ax
float32 ay
float32 az
uint32 tags_count
uint32[] tags_type
uint32[] tags_xc
uint32[] tags_yc
uint32[] tags_width
uint32[] tags_height
float32[] tags_orientation
float32[] tags_distance
float32 tm

This listing shows us the various members of the Navdata message, including their datatypes. Note that the Navdata message has a header, which includes the time the message is sent.

Minimum Viable Subscriber

Open up a new terminal window and type:

$ roscd ardrone_tutorials
$ geany src/minimum_viable_subscriber.py

Up will pop a geany window, ready for your to edit the new file. Enter the following into the new geany window, then save and close the file:

import roslib
import rospy

roslib.load_manifest('ardrone_tutorials')

from ardrone_autonomy.msg import Navdata

def ReceiveData(data):
  print '[{0:.3f}] Pitch: {1:.3f}'.format(data.header.stamp.to_sec(), data.rotY)

rospy.init_node('minimum_viable_subscriber')
sub_Navdata = rospy.Subscriber('/ardrone/navdata', Navdata, ReceiveData)

while not rospy.is_shutdown():
  pass

We will now use the previously recorded bag-file to test the node that we’ve just written.

1. In one terminal window, enter:

   $ roscore

   Which begins the roscore process, which coordinates the interactions between nodes.

2. Now in a new terminal window type:

   $ roscd ardrone_tutorials
   $ python src/minimum_viable_subscriber.py

   This will launch the node that we’ve just programmed.

3. Finally, in a third terminal window, begin the bag-file playback with:

   $ rosbag play ARDroneFlight.bag

   • Note you must be in the same directory as the bag-file for the above command to work
4. If you switch back to the terminal window in which our new node is running, you should see a stream of time-stamped printouts with the drone’s pitch, for example:

   [1353596639.491] Pitch: 4.375
   [1353596639.511] Pitch: 4.128
   [1353596639.531] Pitch: 3.750
   [1353596639.552] Pitch: 3.304
   [1353596639.571] Pitch: 3.063

Minimum Viable Subscriber: A Closer Look

If you made it through the previous section, congratulations, you have successfully written and tested a ROS node, which listens for Navdata measurements from the AR.Drone (or equivalently: a recorded flight) and outputs a time-stamped pitch measurement. But how does it all work?

Lets take a look at the source-code from the previous section, line by line:

import roslib
import rospy

With these two lines, we import the standard ROS packages, required for every ROS node.

roslib.load_manifest('ardrone_tutorials')

Once the required packages are imported, load the manifest file. This file details the dependencies of the current project (for example, that ardrone_tutorials is dependent on the AR.Drone driver, ardrone_autonomy), which through the above command are loaded into the path.

from ardrone_autonomy.msg import Navdata

Once the project dependencies have been loaded, we can import those which are needed by the current node. In the above case, we load the Navdata message definition from the ardrone_autonomy package.

def ReceiveData(data):
  print '[{0:.3f}] Pitch: {1:.3f}'.format(data.header.stamp.to_sec(), data.rotY)

Here we define a callback function to handle the Navdata messages. This function prints a formatted string containing the message’s timestamp and the AR.Drone’s pitch angle. This callback function will be used later in the program.

rospy.init_node('minimum_viable_subscriber')

Before issuing any commands to the rospy package, we need to initialize a ROS node.

sub_Navdata = rospy.Subscriber('/ardrone/navdata', Navdata, ReceiveData)

We can now subscribe the node to various topics. Above we subscribe the node to the /ardrone/navdata topic, which carries Navdata messages. Received messages should then be handled by the previously-defined ReceiveData function.

while not rospy.is_shutdown():
  pass

Because our program is relying on callbacks for data processing, we need to stop the program from terminating. The above defines a loop that runs until the node is shut down (for example with Ctrl-C). In most situations the above loop would perform some sort of processing, however in our example, we simply pass (do nothing).

Publishing Data

In this section we will introduce the concept of publishing data to a topic (the opposite of subscribing). We will be using this more extensively in the next tutorial, where we will be programmatically controlling the AR.Drone by publishing messages to the /cmd_vel topic.

Minimum Viable Publisher

The goal of this section is to create a minimum_viable_publisher.py file such that Navdata messages are averaged over a second and the average pitch published to the (new) /ardrone/average_pitch topic. Modify the file as follows:

import roslib
import rospy

roslib.load_manifest('ardrone_tutorials')

from ardrone_autonomy.msg import Navdata
from std_msgs.msg import String

messages = []

def ReceiveData(data):
  messages.append(data)

rospy.init_node('minimum_viable_publisher')
sub_Navdata = rospy.Subscriber('/ardrone/navdata', Navdata, ReceiveData)
pub_Average = rospy.Publisher('/ardrone/average_pitch', String)

r = rospy.Rate(1)
while not rospy.is_shutdown():
  if len(messages)>0:
    avg = sum([m.rotY for m in messages])/len(messages)
    messages = []

    avgmsg = String()
    avgmsg.data = 'Average Pitch: {0:.3f}'.format(avg)
    pub_Average.publish(avgmsg)
  r.sleep()

You can run the program using the following series of commands:

1. In one terminal, run:

   $ roscore

2. In the next terminal, run the new program

   $ roscd ardrone_tutorials
   $ python src/minimum_viable_publisher.py

3. We now echo the published data to the topic we’ve created:

   $ rostopic echo /ardrone/average_pitch

4. And finally, like we’ve done twice before, we begin the playback of our recorded flight

   $ rosbag play ARDroneFlight.bag

5. Switching back to the terminal in which we ranrostopic echo, you should see a new message with the average pitch appearing once a second:

   data: Average Pitch: 0.649
   ---
   data: Average Pitch: 1.078
   ---
   data: Average Pitch: 3.781
   ---
   data: Average Pitch: 3.285
   ---

Minimum Viable Publisher: A Closer Look

Comparing minimum_viable_publisher.py to minimum_viable_subscriber.py from the last section, we can see many similarities:

from ardrone_autonomy.msg import Navdata
from std_msgs.msg import String

We additionally import the String message, required for our new topic.

messages = []
def ReceiveData(data):
  messages.append(data)

We have updated the navdata callback function to save the messages in a queue, rather than process them straight away.

Note: because we’re accessing the messages list in both a callback function and the main loop, we should be using Python’s threading.Lock to control access to the list and prevent race conditions. See ardrone_tutorials/src/drone_video_display.py for an example.

pub_Average = rospy.Publisher('/ardrone/average_pitch', String)

We create a new publisher, responsible for publishing messages of the String type to the /ardrone/average_pitch topic. Note that we do not need to explicitly initialize this topic, it will be initialized automatically on first usage.

The biggest change is in the program’s main loop, which I will now cover in a little more detail:

r = rospy.Rate(1)

Firstly we define a Rate object. We use this to control the frequency of the main processing loop. In our case, we want to print a message every second, thus we want a frequency of $1 \text{Hz}$.

while not rospy.is_shutdown():
  if len(messages)>0:
    avg = sum([m.rotY for m in messages])/len(messages)
    messages = []

    avgmsg = String()
    avgmsg.data = 'Average Pitch: {0:.3f}'.format(avg)
    pub_Average.publish(avgmsg)

If messages have been received, we calculate the average pitch using a python list comprehension to extract the pitch (.rotY) from every Navdata message. We then empty the messages queue.

We then construct a new String message and set its data property, before publishing it to the topic we previously created.

r.sleep()

Finally, we tell the rate object to sleep. This command will calculate how long it needs to sleep, such that the desired loop frequency is achieved.

Conclusions

I hope you’ve enjoyed working through this tutorial and learning about data flow in the ROS environment. In the next tutorial, which I will release in a month’s time, we will use what we have learned today to programmatically control the AR.Drone, while at the same time learning about PID control.

If you have any questions about the tutorial, please leave me a comment below and I will get back to you as soon as possible!

You can see all the tutorials in here: Up and flying with the AR.Drone and ROS.

If you liked this article, you may also be interested in:

• Parrot AR.Drone app harnesses crowd power to fast-track vision learning in robotic spacecraft
• Taking your first steps in robotics with the Thymio II
• Discovering social robotics with the Aisoy1
• Classroom robotics: Motivating independent learning and discovery

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

This post is part of Robohub’s We Robot coverage.

Paper: A Licensing Approach to Regulation of Open Robotics
Author: Diana Marina Cooper
Moderator: Michael Froomkin

Last year when Willow Garage announced that the company will be changing its funding model there was a few hours of confusion where some people thought the open source robotics company was going to shut down. The robotics community was shocked to hear the news– later found to be erroneous. Reaction from the research community in particular was that of a panic.

That is because a lot of robotics researchers have been depending on continued existence of the company’s products, especially the open source Robot Operating System (ROS). Currently there are over 90 known institutions all over the world contributing repositories to the ROS community alone, and the number of contributions to the community has been increasing exponentially. And that’s just considering ROS and not the other open robotics communities that exist throughout the world.

Given the popularity of open source communities and its rapid growth it is not surprising that open source was a hot topic of discussion at the We Robot 2013 conference last month. Intellectual property of open source systems was discussed during a panel, power of open source was emphasized during the Robotics Industry Roundtable, and a new license for open robotics was proposed. Part of the discussions involved open robotics as a cause of societal and legal concerns.

Typically, in research institutes, there are a set of rules in place to prevent people from damaging equipment, harming others, and harming people who work within the facility. Outside institutional hierarchy, however, only few things restrict or prevent creative and technically inclined individuals from taking open and easily accessible robotics technologies and creating whatever they wish. That is even if the product is meant to serve horrific ends.

For example, apart from my own set of moral principles, there’s nothing stopping me from taking a LEGO Mindstorm kit, attaching a set of sensors, motors, gears, and a knife together to make a dangerous, yet autonomous tabbing machine of some sort. Not to make things too dramatic, but anyone above the age of 10 can play with LEGO Mindstorms and build robots.

That makes liability issues a big headache in open robotics, where shared design/code is meant to be modified by end-users, as compared to closed robotics. This liability issue also means that manufacturers are turned off from developing open systems.

Is there a way we can foster open robotics and open source communities while addressing these concerns?

At We Robot 2013 Diana Cooper, a JD Candidate at the University of Ottawa, presented her attempt to tackle the open source headache by proposing a new license called the Ethical Robot License (ERL). In her paper, A Licensing Approach to Regulation of Open Robotics, Cooper presents ERL as “a licensing approach to allocate liability between manufacturers and users and promote ethical and non-harmful use of open robots”. You can watch the full panel discussion from We Robot above.

Image from http://presta-ecommerce.com/

Included in the license are positive obligations of designers such as having default privacy setting where people need to opt-in for data collection, disclosure, and use of personal information. The current draft of the license envisions people registering their work to a central database and to keep it up to date of the modifications.

There are also restrictions that are aimed to address the ‘open source robotics for horrific end’ scenario above. Under ERL, you may not make robots that have self-defence capabilities, use or modify robots as a weapon, or make robots that perform certain law enforcement functions, such as surveillance, interrogation and executions. Certain restrictions on military/defense use of robots are also outlined in the license.

As Brad Templeton pointed out during the panel, open source communities value the freedom to share, modify and use technologies. It is curious whether the license that imposes added restrictions will be well received by the communities.

Cooper acknowledges that there’s still more work to be done on the license itself before it can be operationalized. She also agrees that this license doesn’t stop people from doing things that the license is meant to restrict. Nonetheless, given the fact that so many roboticists and hobbyists already benefit from open robotics and – I believe – do not intend their shared work to be used for building robots that harm people and society, the ERL could act as a mechanism to overcome various ethical, legal, and societal concerns pertaining to open robotics.

What do you think about the Ethical Robot License or the idea of regulating open robotics through licensing approach? Let us know in the comments below.

See all the We Robot coverage on Robohub →

Rethink Robotics’ Baxter and Aldebaran Robotics’ NAO robots

Aldebaran Robotics’ launched their ASK NAO (Autism Solution for Kids) initiative for use in special education and for children with autism. More humanoid than Baxter, NAO is kid-sized, the new program provides educators with planning, monitoring and analyzing interactions to help set up lessons with NAO. Read the press release.

Although Baxter and NAO have different capabilities, the ability for researchers and educators to easily program applications – and share those apps with others – because of the open source nature of the software and SDK – is hoped by both companies (Aldebaran and Rethink) to accelerate new capabilities and apps for commercial use.

What about Cloud Robotics?

The following video briefly explains the general motivation:

Generally, a robotics application interacts with its environment using sensors and actuators. This implies that the data produced by the sensors has to be processed and is normally used as feedback to determine the next action. For actions without hard real-time requirements, processing can be moved to the cloud. This reduces the necessary computing power on the robot, which can improve operation duration for mobile systems and may decrease costs.

Simplified cloud model: Each robot is connected to its own clone in the cloud (rectangular boxes), which are tightly interconnected with each other and with the databases (circular disks).

Many common high-level robotics tasks, such as planning or mapping, don’t require hard real-time. Mapping, for example, involves storing a reusable description of the environment to allow the robot to localize a landmark or itself. Additionally, maps can be used on a large scale to plan trajectories for moving to a landmark or on a small scale to grasp an object, which in turn might include querying an object database for grasp points to get the best position to grab the object. Because planning can be done ahead of time (i.e. act instead of react), it has no hard real-time constraints.

The advantages of offloading computation to the cloud can be even more pronounced when considering collaborative robot scenarios. For example, sharing data when creating a common map, or negotiating a cooperative plan when performing a common task can be more easily achieved with a centralized architecture with powerful computational capabilities.

RoboEarth allows robots to share knowledge in a centralized knowledge base. This centralized approach has the advantage of avoiding duplicate information and the necessity to constantly exchange newly generated knowledge to avoid duplicate learning. However, to benefit from the full potential of this approach not only requires a centralized storage of knowledge, but also centralized reasoning, i.e. a framework for cloud computation for robots.

The missing piece

Until now, RoboEarth only provided data storage for robots. With the introduction of Rapyuta, the RoboEarth Cloud Engine, the missing piece in the cloud infrastructure for robots is added. Rapyuta provides a platform where robots can create computing environments to offload there heavy computation into the cloud. These computing environments can either function as a personal cloud for individual robots or be shared by multiple robots. Additionally, computing environments have high-bandwidth connections to the RoboEarth database, allowing their processes to quickly access knowledge.

Technical aspects

Simplified overview of Rapyuta: The robots are connected to Rapyuta’s communications module in the cloud. Similarly, the cloud applications running inside the computing environments (LXC) are also connected to the communications module. The module manages communications from and to all connected elements.

To simplify the deployment and interaction of the cloud applications Rapyuta, similar to the Google App Engine, defines a protocol for cloud applications. However, in contrast to Google’s model where only web requests are used to communicate directly with the cloud application, Rapyuta introduces an additional layer. This means, a robot does not communicate directly with a cloud application, but uses Rapyuta as a medium. This model gives more flexibility and control over the system. For example, Rapyuta can be used as a relay to communicate directly between robots without the creation of a computing environment. Furthermore, the additional layer implies that Rapyuta allows to run multiple processes using sockets to communicate with each other. Google’s App Engine, however, prevents this, because web applications have no need for communicating processes to service web requests.

On the robot side Rapyuta uses common web protocols that allow access via a web browser. Like rosbridge and ros.js, Rapyuta uses WebSockets for persistent connections and JavaScript Object Notation (JSON) compatible strings for messages. However, because rosbridge provides only a subset of the functionality of Rapyuta the two protocols are not compatible. The main reason for this is that Rapyuta allows the client to interact with more than one ROS environment, i.e. computing environments, over the same connection, which leads to a more complex protocol for Rapyuta.

On the cloud application side the message system defined by the Robot Operating System (ROS) is used. This allows to deploy and run most of the more than 3’000 existing ROS packages (i.e., ROS applications) with minimal to no changes in the RoboEarth Cloud Engine.

To separate the ROS nodes, i.e. the cloud applications, for different users, Rapyuta provides individual computing environments. A robot can have many computing environments and a computing environment can be used by many robots, which allows a variety of deployment models. Linux Containers (LXC) were chosen for the implementation of computing environment to securely control all operational aspects such as communication as well as CPU, RAM, and disk usage. Additionally, Rapyuta allows different computing environments to communicate with each other using the ROS messaging system.

Rapyuta is open source. The Rapyuta Alpha release is available on github under the Apache License, Version 2.0.

For more information have a look at the Rapyuta documentation.

Further reading:

• Analysis: Robot learning in the cloud

Making robots that are able to localize, track and separate multiple sound sources, even in noisy places, is essential for their deployment in our everyday environments. This could for example allow them to process human speech, even in crowded places, or identify noises of interest and where they came from. Unlike vision however, there are few software and hardware tools that can easily be integrated to robotic platforms.

The ManyEars open source framework allows users to easily experiment with robot audition. The software, which can be downloaded here, is compatible with ROS (Robot Operating System). Its modular design makes it possible to interface with different microphone configurations and hardware, thereby allowing the same software package to be used for different robots. A Graphical User Interface is provided for tuning parameters and visualizing information about the sound sources in real-time. The figure below illustrates the architecture of the software library. It is composed of five modules: Preprocessing, Localization, Tracking, Separation and Postprocessing.

To make use of the software, a computer, a sound card and microphones are required. ManyEars can be used with commercially available sound cards and microphones. However, commercial sound cards present limitations when used for embedded robotic applications: they can be expensive and have functionalities which are not required for robot audition. They also require significant amount of power and size. For these reasons, the authors introduce a customized microphone board and sound card available as an open hardware solution that can be used on your robot and interfaced with the software package. The board uses an array of microphones, instead of only one or two, thereby allowing a robot to localize, track, and separate multiple sound sources.

The framework is demonstrated using the microphones array on the IRL-1 robot shown below. The placement of the microphones is marked by red circles. Results show that the robot is able to track two human speakers producing uninterrupted speech sequences, even when they are moving, and crossing paths. For videos of the IRL-1, check out the lab’s YouTube Channel.

For more information, you can read the following paper:
The ManyEars open framework, F. Grondin, D. Létourneau, F. Ferland, V. Rousseau, F. Michaud, Autonomous Robots – Springer US, Feb 2013

Gumstix’ announcement of official ROS support accompanies the introduction of a bootable microSD card featuring Linaro 12.11 LTS and ROS preloaded, as well as an image available for download. ROS binaries were previously only available for Intel, AMD or ARMEL architectures, requiring all ARMHF-based implementations to be compiled from source, with build time being a limitation for ROS implementations on an architecture ideally suited to robotics development. While Gumstix users are still able to build ROS from source with relative ease on an existing Overo with Ubuntu system, users wanting a quick, simple solution are now also able to flash an Ubuntu with ROS image to their Overo COM, thus eliminating a serious impediment to ROS development.

Since ROS was first released in November 2007, there has been a growing trend of implementing ROS on Ubuntu-based systems to power robotic devices. By October 2009, 50 organizations or individuals had publicly released ROS software in its indexed repositories, a number which has grown in the last three years to over 175 organizations or individuals. Gumstix support for ROS satisfies its own customers’ growing demand for the ability to implement this increasingly popular open source solution with ease and flexibility as part of a broader trend in ROS development.

“Last June, Gumstix announced its TurtleCore expansion board as a platform for ROS development on the iRobot Create,” said Dr. W. Gordon Kruberg , president and CEO of Gumstix. “The ability to deploy an available ROS binary on our Overo line of COMs complements the goal initially set forth with the introduction of the TurtleCore: to provide the most powerful development system for roboticists in the smallest form factor. This image and the microSD release now provide that software with minimum development time and greater ease.”

“We welcome the support of Gumstix in our efforts to expand the open source robotics community,” said Tully Foote, ROS Platform Manager at Willow Garage. “We love what Gumstix has done to date, and look forward to working with them in the future.”

Developers using customized robotics platforms powered by the Overo will be equally advantaged by ROS, needing only to add packages specific to their solution on top of the standard ROS installation. More information about using Robot Operating System and Ubuntu with Gumstix, including the prebuilt image, is available at the Gumstix Users Wiki, while the $45 bootable microSD with ROS is available at www.gumstix.com.

About Gumstix, Inc.
Since developing the first Linux®-based computer-on-module in 2003, Gumstix has grown to become the premier provider of Linux®-based COMs and expansion boards, with over 15,000 diverse customers in more than 40 countries. Gumstix’ commitment to providing the best, standard platform for ubiquitous, intelligent devices with flexible and open-source design results in less internal development time and faster time-to-market for its customers’ products.

RELATED LINKS
http://www.gumstix.com
PR Newswire (http://s.tt/1yUDg)